Comments on: Statistics: tell me about r-square for nonlinear models

Question: Statistics: tell me about r-square for nonlinear models

jonesor — Thu, 21 Jun 2007 02:36:24 -0800

StatisticsFilter: Can someone explain - with examples - why R^2 (r-square) values are not appropriate for use with non-linear regression models. I am told that the reason that r-square values are not recommended for non-linear regression is that they can mislead. R^2 for linear models are bounded between 0 and 1 and can be interpreted as the proportion of variance explained by the model. R^2 for non-linear models, I am told, can be outside these bounds and therefore cannot be interpreted in the same way.
I'd like to know how this exceeding the bounding thing works. (I am using R if that makes giving examples easier). Many thanks!!

By: singingfish

singingfish — Thu, 21 Jun 2007 04:10:31 -0800

OK, for one thing the assumption for a linear regression is that the variance is approximately equal at every point along the scale (homoscedascity). If I understand it correctly given the mathematical procedures used to compute non linear regression this is not possilbe so you get heteroscedascity. I'm no expert in non-linear regression, but I'd expect that you have procedures in place to allow for some violations of assumptions but this "screwing around with the maths" doesn't help produce a valid estimate for r^2

Nice simple example of non-linear regression is logistic regression - a special case of non-linear regression. maybe you should have a look at that to help get your head around it a bit better.

By: desjardins

desjardins — Thu, 21 Jun 2007 05:31:38 -0800

If there is no linear relationship between the two variables, the correlation coefficient [r] is close to 0. [This] does not mean that there isn't any type of relationship between two variables; it is possible for two variables to have an r of 0 and be strongly related in a non-linear way.

SPSS 12.0 Guide to Data Analysis, Marija J. Norusis

I just closed the damn book so I don't have the page # for the quote.

By: ROU_Xenophobe

ROU_Xenophobe — Thu, 21 Jun 2007 08:27:08 -0800

You mean R2 in models like logit or probit, then yeah.

From dim memory and quick googling:

R2 is 1- sum of squared residuals / sum of squared deviations, or 1-SSR/SSD.

This works in OLS because the total sum of squares (SDD) is the regression sum of squares plus the sum of squared deviations, or Total SSD = regression SSD + SSR.

But this identity doesn't hold in nonlinear models, so R2 gets fucked up. Simple as that.

Don't despair. If you want a rough goodness of fit measure for an MLE model, the chi-2 statistic (the ratio of a null model's likelihood to the full model's likelihood) can serve similar to an OLS model's F statistic. Or you can just use more direct measures of predictive success. Or, MLE procedures in many statistical packages can generate pseudo-R2s that are varying degrees of crap.

R2 is overrated anyway. While the common interpretation is that it's the percent of variance that's explained (understandable given the identity), a more accurate interpretation is that it's how much more variation you're explaining than a null model with only a constant would explain. That is, how much better is your model than a model that explains the DV with its own mean. But there are two parts here -- how good your model is, and how good the DV's mean does at explaining the DV. The better the mean does, the worse your R2 will look even though your model is good.

Also, R2 always increases with added terms (at least in OLS). Want a higher R2? Add junk regressors. Want an R2 of 1.00? Have N-1 regressors (IIRC) and leave the dataset fully identified.

By: jonesor

jonesor — Thu, 21 Jun 2007 10:18:37 -0800

Thanks ROU! That's just what I wanted.
I will use cross-validation instead of an R^2.

By: math_junkie

math_junkie — Tue, 28 Aug 2007 13:32:00 -0800

for non-linear models, the R2 can become negative - not exactly intuitive but nevertheless useless. Another possibility is to examine the standard error of any fit - linear or not - to see if additional parameters improve the model