Least-Squares Fit

We can reformulate this problem into a matrix problem as follows: Let A be the n by 2 matrix like this:

A = [ xand let_{1}1 x_{2}1 x_{3}1 x_{4}1 ... x_{n}1]

As discussed in class and in the text (pg. 333) this is called a
least-squares problem and the solution comes from realizing that
the residual must be orthogonal to the column space of A or
that A^{T}r = 0. This works out to form the normal equations:

AIf A has full rank (in this case, 2, which means that we have at least 2 distinct x values), then A^{T}Ac = A^{T}y.

x = [2 5 7 8]'; % column of x-values y = [1 2 3 3]'; % column of y-values A = [x, ones(4,1)]; % form matrix A, add column of 4 1s ATA = A'*A; % form Athen c(1) would be the slope (should be 0.3571) and c(2) the intercept (0.2857).^{T}A for normal equations and b = A'*y; % form A^{T}y c = inv(ATA)*b;

Once we have A and y, we can do this more efficiently with

c = (A'*A)\(A'*y);If you want to see the data and the results you can plot the points and the curve as follows:

predy = A*c; % compute predicted values plot(x,y,'o',x,predy) % plot data as circles and draw best-fit line

Here's how it works for a quadratic (and, in general, for the others).
The residual is still observed - predicted and it is still orthogonal
to the column space of a matrix, so we first look at the residual: **r** = **y** -
Ac where **y** is the column vector of y-values, c is the column
vector of coefficients we're solving for (for the quadratic, c = [a b c]'),
and A is the n by 3 matrix of the form

A = [ xNote that if you take a row of A and multiply it by the column c you get ax_{1}^{2}x_{1}1 x_{2}^{2}x_{2}1 x_{3}^{2}x_{3}1 ... x_{n}^{2}x_{n}1]

Since the residual is orthogonal to A, we again get the normal equations:

A^{T}A c = A^{T}y

How do we do a fit to other types of equations? The key thing to note
is that the columns in A correspond to the functions in the equation
we are trying to fit. For example, for the line the functions are
x and 1 and the first column of A is x_{i} and the 2nd column are
all 1s. Thus if we wanted to fit the crazy function above to some data,
we'd form the matrix A with columns corresponding to the functions
x, 1, sin(x), cos(2x), log(x) and e^{x}.

Once we have the matrix A, we just use the normal equations to solve for
the coefficients.

x = [0 1 2 3 4 5 6 7 8]' y = [1 1 2 2 3 3 4 5 7]'

2. Take the same data and find the best-fit quadratic curve to it.

3. One application of these curves is in approximating other functions. For the data take 11 equally spaced points from to 0 to 2pi and the value of cosine at each point:

x = 2*pi*[0:10]'/10; % 11 equally spaced points y = cos(x);Find the polynomials of degree 2 (quadratic) and 4 (quartic) that best fit this data.

4. Sales models are usually linear functions, but it makes more sense to include some seasonal fluctuations. Suppose over a year, monthly sales amounts (in $1K) are recorded as follows:

s = [20 15 12 8 13 22 28 33 35 45 56 62]and we think the data follows the function