Least Squares Solutions/Data Fitting (4/16/16)

Question:  Find the least squares solution to 
x + y = 1
x - 3y = 4
2x + y = 3.

(*Here are the matrices a, b*)
a = {{1,1},{1,-3},{2,1}};
b = {{1},{4},{3}};
MatrixForm[a]
MatrixForm[b]

(*Here are two ways of obtaining the transpose.*)
Transpose[a] // MatrixForm
a\[Transpose] // MatrixForm

(soln = Inverse[Transpose[a].a].Transpose[a].b ) // MatrixForm

The least squares solution is y = (11/6)x - 8/11.

N[a.soln]

Note that the output values obtained from our solution are very close to the values of b in the original problem.

Question: Find the line y=ax + b that best fits the data (-1,1), (1,2), (2,-1).

a = {{-1,1},{1,1},{2,1}};
MatrixForm[a]
b = {1,2,-1};
MatrixForm[b]

Inverse[Transpose[a].a].Transpose[a].b  // MatrixForm

The line y = -1/2 x + 1 is the best fit line.

Question: Find the parabola that best fits the data points (-1,8), (0,8), (1,4), (2, 16).

Here, I will illustrate how Mathematica has built-in commmands for the methods we are using.

data = {{-1,8},{0,8},{1,4},{2,16}};
MatrixForm[data]

quad = Fit[data, {1, x, x^2}, x]

The parabola that best fits the data is 3x^2 - x + 5.  Here is a graph with the data points and the quadratic function.

Show[Plot[quad, {x, -2, 3}], ListPlot[data, PlotStyle -> Red]]

Note that we can find a cubic function going through all 4 data points.

cubic = Fit[data, {x^3,x^2, x, 1}, x]
Show[Plot[{quad, cubic}, {x, -2, 3}], ListPlot[data, PlotStyle -> Red]]

Finally, here are some tricks for producing and manipulating matrices.

To access the (i,j)th entry in a matrix, you use [[i,j]].  For example, the various entries of data can be accessed as follows.

(data = {{-1,8},{0,8},{1,4},{2,16}})// MatrixForm
data[[2,2]] (*Row 2, Column 2*)
data[[3,1]] (*Row 3, Column 1*)
data[[3,3]]  (*Row 3, Column 3, which does not exist*)

Here is a way we can use the data table to create matrices a, b.

(xvals = Table[data[[i,1]], {i, 1, 4}]) 
(b = Table[data[[i,2]], {i, 1, 4}]) // MatrixForm
(a =Table[ { i^2, i, 1}  , {i, xvals}]) // MatrixForm

Inverse[Transpose[a].a].Transpose[a].b  // MatrixForm

or, there is a built-in command for LeastSquares.

LeastSquares[a,b]

If you are interested in Problem 5:  Here is an example of creating a list of (x,y) values for the curve y = x^2 - 1, from x=1 to x=3, with step size of 1/5.

Table[{x, x^2-1}, {x, 1, 3, .2}]

Here is an example of (x,y) values for y=sin(x), from x=1 to x=3 with step size of .2.

Table[{x, Sin[x]}, {x, 1, 3, .2}]