Mad social scientist seeks statistical test...

May 7, 2008 4:07 PM

StatsFilter: I want to compare quantitatively the results of subjecting two different groups of people, A and B, to two different "programs". Which statistical test do I use to see which "program" had a greater effect?

I have a questionnaire with various 1-5 scale type responses. I want to get these responses, add them all up (or otherwise quantify them), and then have these people engage in one of two kinds of activities, and then take the survey again to see if their responses change. What I'm interested in is which activity had a greater effect. Thanks!

posted by R_Nebblesworth to science & nature (5 comments total)

You want a paired t-test.
posted by djb at 4:23 PM on May 7, 2008 [1 favorite]

Thank you! That was fast.
posted by R_Nebblesworth at 4:25 PM on May 7, 2008

Wait - not so fast!

It depends on the data itself which test you need to pick. Questionnaire data with discrete reponses would often be analysed with non-parametric tests. I would use the Mann-Whitney U test to compare the change in scores from baseline between groups.

If after adding up all the reponses from each questionnaire into a single score, you find that score is roughly normally distributed, you can still use the t-test. You are however less likely to be criticised if you use the Mann-Whitney here.
posted by roofus at 2:28 AM on May 8, 2008

I'm with roofus on this one. The responses that you have are on an ordinal measurement scale at best. You need measures on an interval or ratio scale to use a t-test. Also, as roofus points out, this isn't a hard and fast rule, but the most correct one, so you'll be less likely to be criticized.
posted by mausburger at 9:00 AM on May 8, 2008

The more practical problem with the paired t-test is that it won't actually tell you what you want to know. The paired t-test will compare the before vs. after aspect, but to also do the between-subjects part (ie, which activity has a greater effect), you'll need a 2x2 mixed factorial design. This basically tells you whether the (within-subjects) change is different for the two (between-subjects) groups.
posted by svenx at 9:52 AM on May 8, 2008