Curve fitting question (OT)



#15

Hi experts,

So far, I thought I know some statistical data analysis, but here's something strange leaving me confused:

Fitting a model curve to measured data points, i.e. data with errors > 0 , reduced chi square (RC2) is one method to assess the goodness of fit. Browsing some textbooks and the net, I find just one-sided tests for RC2. I.e. it is said to be a good fit model if RC2 is less than or approximately equal to 1, and a bad fit model if RC2 >> 1. Why isn't this a two-sided test?

Reasoning: The model curve hitting all data points right in the middle is very improbable for physical data. So it should be a two-sided test for RC2, excluding very low values of RC2, too. OTOH, fitting is done minimizing the sum of squared distances from all data points to the curve, so zero would be optimum.

You see me confused. What's wrong? Help is appreciated.

Walter


#16

Hi Walter,

Most books use R-square and the F statistics to measure goodness of fit. One popular tool is found in the Excel Data Analysis add-in. It generates an ANOVA table that shows the F statistics and its significance. Also, the tool calculates and shows the R-square and its adjusted value. Finally the output also shows the standard error, student-t, and significance of student-t for each regression coefficient (and intercept).

I rely on the regression ANOVA table and other statistics displayed by the Excel Data Analysis add-in more than just the Chi-square.

If you want to examine the relevance of yoru curve fit I highly recommend Excel's tool and the richness of the output.

Namir


#17

Hi Namir,

Thanks for your friendly advise. I know and use this tool already. It is just that almost every author I see demonstrating simple Chi Square tests uses single sided tests to measure goodness of fit, while the same authors have no problems with double sided t-tests, for example. That's what puzzles me. Maybe a bit academic, but I just want to understand it.

Walter

#18

Quote:
Why isn't this a two-sided test?

You're squaring the differences so the closer to zero you get the better the fit. Thus no need for the other side. Or am I missing something in the question?


- Pauli


#19

Hi Pauli,

Quote:
Or am I missing something in the question?

Maybe. Please check my "reasoning" above and Dave's answer below. IMHO a contradiction remains.

Walter

Edited: 30 July 2009, 3:03 a.m.


#20

Missed that bit for some reason -- probably my own stupidity and haste :-(

I've never heard of a test for a too-good fit of the data. That has always been a cause for great rejoicing not concern. However, if you are interested in that why not do another test at the 99.9999% (or whatever) level and reject if that accepts the fit?


- Pauli

#21

Quote:
so zero would be optimum.

Not really! Reduced chi-square includes the estimated size of your errors, so if you have estimated your errors correctly (sometimes a task in itself for real-world data), the fit value should be near 1. If it is much below one, either your error estimates are not good (too pessimistic), or you are fitting to too many variables or the wrong equation(s) and fooling yourself that you know what is going on!


#22

Hi Dave,

What you state matches what I remember from the old days, too. But it doesn't seem to be very popular, does it? So maybe I miss an important reason for single sided testing here, being so simple nobody thinks its worth explaining?

#23

The Reduced Chi Square test is the Chi Square devided by the degrees of freedom. So it seems to me to be a "normalised" Chi Square, hence 1 being the optimum value. This makes it easier to present results without the reader having to worry about D.O.F., number of samples etc.

The Chi Square test itself can be either a one sided (upper or lower) test, or a two sided test. However, only the upper one-sided test lends itself to the Reduced Chi Square.

Bart


#24

Hi Bart,

I agree with everything you wrote but the last sentence. Nothing prevents me from dividing a lower Chi Square by its DF.

Walter


#25

Hi Walter

The problem at the lower tail of the CS is that except for very large DOF, CS/DOF < 1, even if it is a bad fit.

Bart


#26

Hi Bart,

CS/DOF will *always* stay <1 for the lower tail, as well as it will stay >1 for the upper tail d8) Both values will approach 1 for large DOF.

Walter


#27

Hi Walter,

Sorry, I seem to have the wrong end of the "tail", RC2 is more suited for a lower tail GOF check, if <=1 is to be the acceptance criteria.
However, as CS/DOF=1 is around the 60th percentile, the RC2 seems a very loose check for goodness of fit. As the 5th percentile is a more acceptable criteria, then RC2 < 0.78 for 100 DOF or <0.5 for 20 DOF.

Let me add that I've used the Chi2 and looked into the RC2 when I saw your question this morning.


#28

Just to illustrate what we are talking about: This is a diagram of Reduced Chi Square over Degrees Of Freedom. The lines are for different confidence levels and number of sides, and "einseitig" means single sided, while "zweiseitig" means (surprise!) double sided. Solid lines are for single sided, blue lines are for 99%, so broken lines are for double sided, and red lines are for 95%.


Possibly Related Threads...
Thread Author Replies Views Last Post
  Entering,Saving,and Analysis /Fitting X Y Data on the Prime Harold A Climer 6 659 10-26-2013, 01:54 PM
Last Post: Tim Wessman
  Challenge(?): Intersection curve between two cylinders in a specific position Pier Aiello 15 1,019 09-17-2013, 05:58 PM
Last Post: Pier Aiello
  HP Prime : geometry & curve Mic 0 220 09-15-2013, 02:31 PM
Last Post: Mic
  HP 32S-II Vertical Curve Program Ron Cardwell 2 367 05-20-2013, 07:54 AM
Last Post: Thomas Klemm
  OT: A question about the Sharp EL-8131 Marcel Samek 11 749 02-13-2013, 03:10 PM
Last Post: Massimo Gnerucci (Italy)
  OT--APF Mark 55 Question Matt Agajanian 6 521 07-19-2012, 10:51 PM
Last Post: Matt Agajanian
  Legible version of 29C Curve Fitting program Matt Agajanian 6 521 03-21-2012, 07:46 PM
Last Post: Matt Agajanian
  HP 32sII Integration Error of Standard Normal Curve Anthony (USA) 4 423 03-14-2012, 03:18 AM
Last Post: Nick_S
  OT - Length of day question Cristian Arezzini 20 1,135 12-14-2011, 09:28 AM
Last Post: M. Joury
  Algorithm for fitting a logistic curve? Tim Wessman 5 417 11-13-2011, 01:22 AM
Last Post: Wes Loewer

Forum Jump: