Help with statistics!
December 30, 2011 10:28 PM Subscribe
Statistics help - What test should I use? Help needed for a novice.
I have a sample of 75 people. I have their scores for different opinions on a 0-10 likert type scale. I've combined 6 of these related scores to give me a scale from 0-60 to be my dependent variable. I know enough to know this is an ordinal variable. But that's about all I know and need help on the next steps.
For bivariate comparisons - with ordinal dv.
1) What test do I use if the independent variable is categorical? Does it matter if it is 2 categories, vs 3 or 4?
2) What test do I use for another ordinal variable? Is there a difference if I use an interval variable?
For multivariate - with ordinal dv.
3) OLS? Ordinal Lobit? This is a longer term project - I'll read up more once I get started but where should I start reading?
The last time I took a stats class was back in 2007 and I do qualitative social science work now - I've read through old questions about books (and even have suggested some myself in the past) - but I need a refresher.
I have a sample of 75 people. I have their scores for different opinions on a 0-10 likert type scale. I've combined 6 of these related scores to give me a scale from 0-60 to be my dependent variable. I know enough to know this is an ordinal variable. But that's about all I know and need help on the next steps.
For bivariate comparisons - with ordinal dv.
1) What test do I use if the independent variable is categorical? Does it matter if it is 2 categories, vs 3 or 4?
2) What test do I use for another ordinal variable? Is there a difference if I use an interval variable?
For multivariate - with ordinal dv.
3) OLS? Ordinal Lobit? This is a longer term project - I'll read up more once I get started but where should I start reading?
The last time I took a stats class was back in 2007 and I do qualitative social science work now - I've read through old questions about books (and even have suggested some myself in the past) - but I need a refresher.
These kinds of data are usually treated as interval, not ordinal level data. The fact that you've combined the scores (presumably adding them?) already implies an interval level of measurement.
If the IV has 2 levels a t-test is typical. More than 2, ANOVA is typical.
A lot of these questions depend on your research question, however. Feel free to MeMail, and I can try to point you in the right direction
posted by jasper411 at 11:46 PM on December 30, 2011 [1 favorite]
If the IV has 2 levels a t-test is typical. More than 2, ANOVA is typical.
A lot of these questions depend on your research question, however. Feel free to MeMail, and I can try to point you in the right direction
posted by jasper411 at 11:46 PM on December 30, 2011 [1 favorite]
That you've decided to add them (meaning that differences are all comparable) and it's 0-60 (meaning discreteness / boundedness is probably negligible) strongly suggests you can go ahead and treat it like a continuous number (t-tests, OLS). How confident are you in your summed score as reliable and valid? Some variant of robust standard errors (like bootstrap) is good to show that the violations of OLS assumptions aren't throwing the significance estimates off.
You can probably also do the conditioning the other direction and predict the IV using the answers if that makes the setup easier (if you are not doing a designed experiment or some other trick which allows causal inference). For example, for a factor IV with no a priori relationship between the levels you know it's a multinomial regression.
Psychometrics has a long history of dealing with this kind of data which can be as complicated as you like, but much of that is looking at aspects of the data which you may not care about. Your actual question and the audience to which you intend to present (!!) matters a lot for the "right" analysis choice. Often doing whatever is conventional in the field / audience appropriate is more important than slight differences in the robustness of the technique.
posted by a robot made out of meat at 10:55 AM on December 31, 2011 [1 favorite]
You can probably also do the conditioning the other direction and predict the IV using the answers if that makes the setup easier (if you are not doing a designed experiment or some other trick which allows causal inference). For example, for a factor IV with no a priori relationship between the levels you know it's a multinomial regression.
Psychometrics has a long history of dealing with this kind of data which can be as complicated as you like, but much of that is looking at aspects of the data which you may not care about. Your actual question and the audience to which you intend to present (!!) matters a lot for the "right" analysis choice. Often doing whatever is conventional in the field / audience appropriate is more important than slight differences in the robustness of the technique.
posted by a robot made out of meat at 10:55 AM on December 31, 2011 [1 favorite]
a robot made out of meat provides good insight into this. The individual scales are certainly ordinal, but when you add/combine the data and create a composite score, it's hard to call that data ordinal, continuous or anything else for that matter unless it has been validated. Say two subjects have a composite score of 20 but obtained it with different sums of sub-scores. Is it really fair to say that they should be treated as the same? Or for that matter that a composite of 21 that varies widely from another composite of 20 in its sub-scores should really be higher on the scale at all? Do you have good reason to (or not to) weight the sub-scores in some fashion prior to combining them? Without realizing it, you've already made lots of assumptions about the data by combining it in the way you have chosen.
The rigorous way to do this is its own entire field of statistics that looks to validate such scales both internally, across different populations, and sometimes different languages/cultures. From what you've described, though, I suspect at the level of inquiry you're considering, you can probably just as well treat this data as continuous. This is because if its remotely valid to just add up the scores the way you have, they basically must behave in a near continuous fashion, otherwise they would require individual weighting. Either way, composite scores generally get treated continuously by design. Perhaps good practice would be to make sure the distribution of composite scores is reasonably approximates a normal distribution if you plan to use standard parametric tests (t-test/ANOVA).
posted by drpynchon at 11:33 AM on December 31, 2011
The rigorous way to do this is its own entire field of statistics that looks to validate such scales both internally, across different populations, and sometimes different languages/cultures. From what you've described, though, I suspect at the level of inquiry you're considering, you can probably just as well treat this data as continuous. This is because if its remotely valid to just add up the scores the way you have, they basically must behave in a near continuous fashion, otherwise they would require individual weighting. Either way, composite scores generally get treated continuously by design. Perhaps good practice would be to make sure the distribution of composite scores is reasonably approximates a normal distribution if you plan to use standard parametric tests (t-test/ANOVA).
posted by drpynchon at 11:33 AM on December 31, 2011
Response by poster: Thanks everyone - you have definitely got me started on the right track.
The skewness is -.468 and kurtosis is -.192 - so am I safe in assuming it is normally distributed?
posted by quodlibet at 11:42 AM on December 31, 2011
The skewness is -.468 and kurtosis is -.192 - so am I safe in assuming it is normally distributed?
posted by quodlibet at 11:42 AM on December 31, 2011
Response by poster: Sorry - with Std Error of .277 (skewness) and .548 (kurtosis).
posted by quodlibet at 11:44 AM on December 31, 2011
posted by quodlibet at 11:44 AM on December 31, 2011
Response by poster: But a Shapiro-Wilk significance score of .049. Looks like I have problems?
posted by quodlibet at 11:50 AM on December 31, 2011
posted by quodlibet at 11:50 AM on December 31, 2011
It depends on what you are doing. For example, t-test do not require normality of the underlying variable to be valid. OLS tends to be very resilient to non-normal outcomes; the procedure is optimal under normality but applicable without that assumption. In general if you (nearly) correctly specify the relationship between the mean of a group of observations and its variance you are fine (the theory for this goes under the name quasi-likelihood). Also remember that the usual normality assumption is on the residuals, not the outcome. Normality tests are not very useful for exploratory analysis; make some plots.
posted by a robot made out of meat at 7:31 PM on December 31, 2011
posted by a robot made out of meat at 7:31 PM on December 31, 2011
This thread is closed to new comments.
For the rest of them, you can probably use OLS. The assumptions of the usual model aren't quite satisfied with an ordinal dependent variable, but if this a big concern for you, you can use a bootstrap or similar approach to determine whether the coefficients are significant.
It's hard to say more without more knowledge of your data and exactly what it is you're trying to determine.
posted by mikeand1 at 10:59 PM on December 30, 2011