How do I report Bonferonni corrected pairwise comparisons in APA style?
January 11, 2011 2:47 PM Subscribe
What statistics do I report for Bonferonni corrected post hoc tests?
I ran a one-way repeated measures ANOVA and found that my overall F test for my independent variable (facial expression @ 3 levels, approving, neutral, and disapproving) was significant.
Next, I ran Bonferonni corrected post hoc tests and found that all of my pairwise comparisons were significant (i.e, 1 versus 2, 2 versus 3, and 1 versus 3).
My very specific questions are:
1. What statistics do I report for these post hoc test? Just the means and standard deviations for each level of the independent variable? In my case, the facial expressions of approving (1), neutral (2), and disapproving (3)? SPSS gives me a significance level for each pairwise comparison, but I do not know if there are F tests to report for each of these (along with degrees of freedom).
2. If it is already "corrected" using the Bonferroni correction, would a significance level of .032 still be significant? Typically, this would fall below the .05 threshold and be significant. I just thought that Bonferonni was lowering the significance level on the basis of the number of tests. Is this already "built" into the correction such that .032 would still be considered significant?
I hope this makes sense without knowing all the details of the experiment.
Any help and guidance for reporting these findings in APA style is appreciated.
I ran a one-way repeated measures ANOVA and found that my overall F test for my independent variable (facial expression @ 3 levels, approving, neutral, and disapproving) was significant.
Next, I ran Bonferonni corrected post hoc tests and found that all of my pairwise comparisons were significant (i.e, 1 versus 2, 2 versus 3, and 1 versus 3).
My very specific questions are:
1. What statistics do I report for these post hoc test? Just the means and standard deviations for each level of the independent variable? In my case, the facial expressions of approving (1), neutral (2), and disapproving (3)? SPSS gives me a significance level for each pairwise comparison, but I do not know if there are F tests to report for each of these (along with degrees of freedom).
2. If it is already "corrected" using the Bonferroni correction, would a significance level of .032 still be significant? Typically, this would fall below the .05 threshold and be significant. I just thought that Bonferonni was lowering the significance level on the basis of the number of tests. Is this already "built" into the correction such that .032 would still be considered significant?
I hope this makes sense without knowing all the details of the experiment.
Any help and guidance for reporting these findings in APA style is appreciated.
As I recall, when you click the "Bonferroni" option in the ANOVA post-hoc menu in SPSS, it runs entirely too many contrasts (some of which you might not actually care about) and then cruelly overcorrects for familywise error. Do you have specific predictions about which levels of your factor should be different?
Typically, the simplest way to run a limited number of contrasts is through SPSS syntax. Set up your ANOVA in the point-and-click menu, and then click paste. SPSS will dump your instructions into a syntax window. Add a line with the following sub-command (make sure it comes before the line with the period that marks the end of the ANOVA command)
/contrast(pred) = special (1 -1 0)
where "pred" is your predictor variable in SPSS and the numbers in the parentheses are weights specifying your contrast. So, if there are three levels, "1 -1 0" specifies a pairwise comparison between level one and level two, "0 1 -1" is between two and three, "1 -1/2 -1/2" between one and the mean of two and three, and so on. Then you click the run button and SPSS will give you some output, including the results of your contrasts. The p-values in the contrasts will be uncorrected, so you're at your leisure to apply the Bonferroni correction. You can read more here.
It's also a good idea to learn about orthogonal and nonorthogonal contrasts, and how the difference might affect your approach to data analysis. An introductory applied stats textbook will have a chapter dedicated to a simple discussion of these topics.
posted by Nomyte at 6:58 PM on January 11, 2011 [1 favorite]
Typically, the simplest way to run a limited number of contrasts is through SPSS syntax. Set up your ANOVA in the point-and-click menu, and then click paste. SPSS will dump your instructions into a syntax window. Add a line with the following sub-command (make sure it comes before the line with the period that marks the end of the ANOVA command)
/contrast(pred) = special (1 -1 0)
where "pred" is your predictor variable in SPSS and the numbers in the parentheses are weights specifying your contrast. So, if there are three levels, "1 -1 0" specifies a pairwise comparison between level one and level two, "0 1 -1" is between two and three, "1 -1/2 -1/2" between one and the mean of two and three, and so on. Then you click the run button and SPSS will give you some output, including the results of your contrasts. The p-values in the contrasts will be uncorrected, so you're at your leisure to apply the Bonferroni correction. You can read more here.
It's also a good idea to learn about orthogonal and nonorthogonal contrasts, and how the difference might affect your approach to data analysis. An introductory applied stats textbook will have a chapter dedicated to a simple discussion of these topics.
posted by Nomyte at 6:58 PM on January 11, 2011 [1 favorite]
This thread is closed to new comments.
I would take all this to your professor or journal and ask how they want you to report this. The idea of multiple correction (Bonferroni or otherwise) is so incredibly thorny and complex that it is hard to give general advice.
I would say report the tests P-values as given, and note which ones survive after correction. My own personal axe is to generally distrust almost everything about F-tests, since it is hard to verify the assumptions (of the distribution of the input data) that they rely on :). Others are more tolerant.
posted by gregglind at 2:54 PM on January 11, 2011 [1 favorite]