Beau,

Another follow-up about the least squares regression issue.

Quoting from the matlab website.
"Use the MATLAB® backslash operator (mldivide) to solve a system of
simultaneous linear equations for unknown coefficients. Because
inverting X'*X can lead to unacceptable rounding errors, the backslash
operator uses QR decomposition with pivoting, which is a very stable
algorithm numerically."

That about seems to cover the problem.

Cheers,

Kurt

--- In omatrix@yahoogroups.com, "kurtforrester" <kurtforrester@...> wrote:
>
> Beau,
>
> Thanks for the response.
>
> Both methods are supposed to be minimising the sum of squared error
> (SSE) sum(observed_i - predicted_i)^2. Therefore comparing the
> parameter values/confidence interval is not the measure of difference
> in the methods, it should be the SSE.
>
> For the backslash division:
> SSE = 0.759546
>
> For the full wiki method implemented in O-Matrix:
> SSE = 0.96827
>
> Even within computation tolerances the SSE for both methods are far
> from the same (> 25% variation) and it appears that O-Matrix is not
> minimising the objective function to an appropriate tolerance. I am
> just trying to reconcile the differences in the methods.
>
> Yours,
>
> Kurt
>
>
> --- In omatrix@yahoogroups.com, "Harmonic Software" <beau@> wrote:
> >
> > Kurt,
> >
> > Your solutions bhat, i.e. beta hat in the wiki article I believe
> are within
> > the stated confidence intervals? I believe that the stated bhat
> values in
> > the wiki article are the confidence intervals, of which the O-Matrix
> > solution gives the estimate. Will look into that further for you.
> >
> >
> >
> > Beau
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > From: omatrix@yahoogroups.com [mailto:omatrix@yahoogroups.com] On
> Behalf Of
> > kurtforrester
> > Sent: Thursday, April 24, 2008 11:42 AM
> > To: omatrix@yahoogroups.com
> > Subject: [omatrix] Linear Algebra
> >
> >
> >
> > Trying to replicate the example from wikipedia for linear regression
> > using the following O-Matrix script
> > ##### Code Start ###################################
> > height = {1.47, 1.5, 1.52, 1.55, 1.57, 1.60, 1.63, 1.65, 1.68, 1.7,
> > 1.73, 1.75, 1.78, 1.8, 1.83}
> > weight = {52.21, 53.12, 54.48, 55.84, 57.2, 58.57, 59.93, 61.29,
> > 63.11, 64.47, 66.28, 68.1, 69.92, 72.19, 74.46}
> >
> > X = [ones(rowdim(height),1), height, height^2]
> >
> > s = inv(X'*X)
> > bhat = X\weight # Backslash division
> > bhat2 = s*X'*weight # As shown
> > http://en.wikipedia.org/wiki/Linear_regression
> >
> > print bhat
> > print bhat2
> > ##### Code end #####################################
> > # O-Matrix responds
> > ##### Command window output begin ##################
> > {
> > 128.807
> > -143.155
> > 61.9582
> > }
> >
> > {
> > 132.617
> > -147.417
> > 63.0986
> > }
> > ##### Command window output end ##################
> >
> > Why if they are both implementations of linear regression (although
> > different methods) are the results so different?
> >
> > Cheers,
> >
> > Kurt
> >
>