Comments on: What is the non-parametric version of two-way ANOVA?
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA/
Comments on Ask MetaFilter post What is the non-parametric version of two-way ANOVA?Wed, 03 Oct 2007 18:40:51 -0800Wed, 03 Oct 2007 18:40:51 -0800en-ushttp://blogs.law.harvard.edu/tech/rss60Question: What is the non-parametric version of two-way ANOVA?
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA
How do you examine the interaction of two factors on a non-parametric dependent variable (e.g. nonparametric version of the 2-way ANOVA)? <br /><br /> Asking for a friend who needs some stats help-<br>
<br>
He conducted a study in which two groups of individuals (one visually impaired group and one control group) were administered a visual task. Each group has an equal sample size and an equal number of males and females.<br>
<br>
He initially conducted a 2x2 ANOVA in order to examine the interaction between group condition and sex on visual task performance. He has been asked to re-analyze the data given that the dependent variable (performance on visual task; #of errors) is not normally distributed. How can he examine the <b>interaction of sex and group condition on visual task performance using a nonparametric statistical method</b>?<br>
<br>
And specifically, can this analysis be conducted in SPSS and, if so, how would it be done? If not in SPSS, what program?post:ask.metafilter.com,2007:site.72982Wed, 03 Oct 2007 17:36:37 -0800cashmanANOVAstatisticsBy: parudox
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086821
The <a href="http://en.wikipedia.org/wiki/Mann-Whitney_U">Mann-Whitney <em>U</em></a> test looks relevant.comment:ask.metafilter.com,2007:site.72982-1086821Wed, 03 Oct 2007 18:40:51 -0800parudoxBy: RichardP
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086822
Perhaps a Friedman two-way analysis of variance? UCLA has a <a href="http://www.ats.ucla.edu/STAT/mult_pkg/whatstat/default.htm">nice table</a> for choosing an appropriate statistical analysis with links showing how to do each test using SAS, Stata and SPSScomment:ask.metafilter.com,2007:site.72982-1086822Wed, 03 Oct 2007 18:40:58 -0800RichardPBy: thrako
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086823
I'm not an expert, but I would think about doing a series pairwise comparisons with the Wilcoxon-Mann-Whitney test.comment:ask.metafilter.com,2007:site.72982-1086823Wed, 03 Oct 2007 18:41:46 -0800thrakoBy: a robot made out of meat
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086827
Two standard ways to do this would be<br>
1) transform the data to normality. Try a box-cox transform.<br>
2) transform the data to (global) ranks and do a standard anova. It's very robust.<br>
<br>
It's hard to say exactly what the best thing to do is without knowing how the outcomes are distributed. But really, transforming it to approximate normality (even if it's discrete) is a good place to start.<br>
<br>
for review:<br>
The Rank Transform Method in Some Two-Factor Designs<br>
Michael G. Akritas<br>
Journal of the American Statistical Association, Vol. 85, No. 409 (Mar., 1990), pp. 73-78comment:ask.metafilter.com,2007:site.72982-1086827Wed, 03 Oct 2007 18:43:19 -0800a robot made out of meatBy: cashman
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086868
Thank you so much! He's reviewing and trying to figure out if these are applicable. Definitely appreciated and I'll mark a best as soon as I get the word.<br>
<br>
Any replies still welcome.comment:ask.metafilter.com,2007:site.72982-1086868Wed, 03 Oct 2007 19:31:32 -0800cashmanBy: a robot made out of meat
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086876
cash: a quick word about what the data are like would help. For example, if the outcomes are ordinal (like 1,2,3,4,5) then the many non-parametric rank tests aren't going to work.comment:ask.metafilter.com,2007:site.72982-1086876Wed, 03 Oct 2007 19:38:18 -0800a robot made out of meatBy: fshgrl
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086890
If you have access to PRIMER it is set up to do this kind of thing.comment:ask.metafilter.com,2007:site.72982-1086890Wed, 03 Oct 2007 19:47:42 -0800fshgrlBy: cashman
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086893
Not sure if I'm answering your question correctly, but I think the data might qualify as ordinal since the values range from 1 to 5 errors. Its skewed towards most people getting most of the items correct.comment:ask.metafilter.com,2007:site.72982-1086893Wed, 03 Oct 2007 19:48:52 -0800cashmanBy: ikkyu2
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086894
I am a little rusty on this, but I seem to recall that for these kinds of 2x2 designs with non-parametric data the Mantel-Haenszel chi-square test is the way to go. If you're cramming this into a 2x2 table then you must be stratifying the dependent results (# of errors on visual testing) into PassVisualTest and FailVisualTest categories. This may not be the best way to go about this.<br>
<br>
The (Wilcoxon)-Mann-Whitney U test is a special or degenerate case of this test so it may also be appropriate.<br>
<br>
It's worth knowing that you have to have a pretty large sample size for either of these tests to have a decent chance of detecting a significant result. If any of the 2x2 cells contain less than 5 subjects, the chi-square test is likely to produce a type II error (failure to detect a true difference between the groups); the Mann-Whitney test likewise requires a fairly large sample, 50-100 or so in each wing to detect reasonable differences.<br>
<br>
I would strongly caution you about transforming data to fit a normal distribution, without the help of a trained statistician and a decent theoretical explanation of why your transform is valid. Squinting at the data and saying "let's log-transform this so it looks more bell-shaped" in order to apply a parametric analysis is one of the easiest ways to destroy validity that I know of. <br>
<br>
I don't know why I bother with these questions, actually; I suggest just waiting until ROU_Xenophobe weighs in with his answer.comment:ask.metafilter.com,2007:site.72982-1086894Wed, 03 Oct 2007 19:48:56 -0800ikkyu2By: ikkyu2
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086922
OK, so I actually cracked a book here. The case and control groups here have equal <i>n</i> and equal number of men and women. Were the cases and controls actually intentionally matched? If they were, you can gain a great deal of power by using a paired analysis, which in this case would be the Wilcoxon matched pairs signed rank sum test. The data doesn't have to be parametric but it does have to be roughly symmetric within groups around the within-group mean.<br>
<br>
I think I'd want to eyeball the data before I suggested anything further.comment:ask.metafilter.com,2007:site.72982-1086922Wed, 03 Oct 2007 20:10:38 -0800ikkyu2By: cashman
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1086941
Wow, thanks a lot ikkyu2! The data was not intentionally matched [via my friend], though retrospectively they are well matched on age, race and sex.<br>
<br>
I really appreciate all the help, especially since most of this is going over my head [and into his]. Hopefully by tomorrow this will all be figured out.comment:ask.metafilter.com,2007:site.72982-1086941Wed, 03 Oct 2007 20:25:51 -0800cashmanBy: ROU_Xenophobe
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1087047
<i>I don't know why I bother with these questions, actually; I suggest just waiting until ROU_Xenophobe weighs in with his answer.</i><br>
<br>
Heh. Thanks.<br>
<br>
But this sounds like actual science, with experiments and everything, and I live in a different world from that; I almost never have to deal with anova except for the F or Chi2 statistics embedded in regression output.<br>
<br>
My first reaction: Is there a strong reason why you can't just run an event count model -- a poisson regression or negative binomial regression? But I come from a world where the first reaction is to find a regression-like tool. The only times I've run event count models were in classes, though, so I can't offer much help on them. Both make assumptions about the dependent variable, but I forget which.<br>
<br>
That said, the tactically right answer will come from your friend's discipline. Not much sense in fighting a battle for your results <i>and</i> an entirely separate battle to get your methods accepted, at least not unless you're already securely tenured or otherwise safe. Better, in the immediate sense, to use the methods that your discipline accepts.comment:ask.metafilter.com,2007:site.72982-1087047Wed, 03 Oct 2007 22:22:06 -0800ROU_XenophobeBy: ikkyu2
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1087093
First eyeball a histogram of the test scores of the cases and a similar histogram of the test scores of the controls. Ideally each histogram would be pretty symmetric (not skewed to the left or right) and the two histograms would be somewhat similar in overall shape, with the blind folks' translated over towards more errors. If they are skewed a transform can be applied to help correct the skew and make them more symmetric about their means. The same transform has to be applied to each group.<br>
<br>
Then, I would have someone who's blind (in the statistical sense, not the visual sense) to the outcome data match the cases and controls retrospectively. If you're matching on just age, gender and race, which I agree is reasonable, you might even be able to get a computer program to figure out the optimal match, or just do it by hand.<br>
<br>
Next, apply the Wilcoxon signed rank sum test. This test will tell you if there's a statistically significant difference in test scores between cases and controls, and it can generate a confidence interval relating to the magnitude of that difference.<br>
<br>
May I put in a plug for Douglas G. Altman's <i>Practical Statistics for Medical Research</i>. Chapter 9, sections 4 through 8 deal with these tests.comment:ask.metafilter.com,2007:site.72982-1087093Wed, 03 Oct 2007 23:51:18 -0800ikkyu2By: ikkyu2
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1087095
Which event would you count, ROU_Xenophobe? Missing a test question, or passing a fail threshold?comment:ask.metafilter.com,2007:site.72982-1087095Wed, 03 Oct 2007 23:52:37 -0800ikkyu2By: a robot made out of meat
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1087173
Ok, since you say pseudo-ordinal, I give you http://www.whitemoldresearch.com/files/022706-01.pdf <br>
<br>
as a reference, skip to the discussion for summary.. Apparently the rank transform is something that people used to do in this case, but don't recommend anymore (in favor of distributional tests). Someone said Friedman test up above. There might be a way to get that test to do what you want, but it's normally thought of as a complete block design / repeated measures test.comment:ask.metafilter.com,2007:site.72982-1087173Thu, 04 Oct 2007 04:46:56 -0800a robot made out of meatBy: nixxon
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1087203
If ikkyu2 is right and your participants are either passing or failing the test, I'd do a logistic regression with a sex*condition interaction term. But I agree with ROU_Xenophobe that you should do whatever is standard practice in your field.comment:ask.metafilter.com,2007:site.72982-1087203Thu, 04 Oct 2007 06:01:27 -0800nixxonBy: ROU_Xenophobe
http://ask.metafilter.com/72982/What-is-the-nonparametric-version-of-twoway-ANOVA#1087255
<i>Which event would you count, ROU_Xenophobe? Missing a test question, or passing a fail threshold?</i><br>
<br>
Number of questions missed.<br>
<br>
If it's pass/fail, then you have a binary DV and can do logit or probit (or gompit or scobit or...)<br>
<br>
<i>I'd do a logistic regression with a sex*condition interaction term.</i><br>
<br>
It might be easier to just use dummies for impaired men, impaired women, and unimpaired men (OR otherwise any 3 of the 4 categories in your 2x2), and then interpret the differences in the coefficients on each dummy. This is easy enough by hand if you output the variance-covariance matrix, or some software (stata) can do this for you. And it won't use any more DF than a dummy for sex, a dummy for condition, and their interaction.<br>
<br>
But again, you want to reflect practice in your field, not mine. I know nothing about the different kinds of sophistication that get used to assess experimental outcomes, because we can hardly ever run experiments (and I never do).comment:ask.metafilter.com,2007:site.72982-1087255Thu, 04 Oct 2007 07:13:56 -0800ROU_Xenophobe