Analyzing ranking data
March 6, 2006 10:26 AM Subscribe
StatsFilter: How to analyze a set of ranking data?
I ran a survey. I have a set of rankings: 10 items where each respondent uniquely ranked each item 1-10. But... I don't really know what to do with it. How do I generate a group ranking based on the individual rankings? Total the ranking vales for each item? Average them? Also, any references to a text that describes how to analyze this kind of data are welcome.
I ran a survey. I have a set of rankings: 10 items where each respondent uniquely ranked each item 1-10. But... I don't really know what to do with it. How do I generate a group ranking based on the individual rankings? Total the ranking vales for each item? Average them? Also, any references to a text that describes how to analyze this kind of data are welcome.
Response by poster: I saw that thread, thanks. But I only have one set of data and I don't need to correlate it with anything else. My main desire is to translate a set of individual preferences into a group preference in a statistically valid way.
posted by GuyZero at 11:05 AM on March 6, 2006
posted by GuyZero at 11:05 AM on March 6, 2006
Response by poster: To half-answer my own question, anyone know how to do a one-dimensional least-squares fit in Excel?
posted by GuyZero at 11:17 AM on March 6, 2006
posted by GuyZero at 11:17 AM on March 6, 2006
Response by poster: To further answer my own question, please don't mock me for taking half an hour to realize that a one-dimensional least-squares is equivalent to the average. *sigh*
posted by GuyZero at 11:28 AM on March 6, 2006
posted by GuyZero at 11:28 AM on March 6, 2006
Best answer: You can use an election method like the Condorcet method to generate a group ranking. Assuming there are no ties or "circular ties," the Condorcet method, the group ranking has the following property: If A is ranked before B in the group ranking, then a majority of the individuals also rank A before B.
Other election methods like Single Transferable Vote may produce different results; it can be useful to try more than one and then analyze any differences in the results.
posted by mbrubeck at 12:41 PM on March 6, 2006
Other election methods like Single Transferable Vote may produce different results; it can be useful to try more than one and then analyze any differences in the results.
posted by mbrubeck at 12:41 PM on March 6, 2006
Best answer: This is what you seem to be asking:
If 10 people each have their own individual preferences, what are their group preferences? What is the correct way to aggregate their preferences?
It turns out that this is a fundamentally insoluble problem -- there is no globally correct way to aggregate individual preferences; every possible way is terrible in at least one respect. Ken Arrow got a Nobel for showing this.
But that's at a more philosophical or technical level. For your purposes, I would do two things:
(1) Average ranking
(2) Borda count. This runs sort of like a car-racing championship. Each alternative get N-1 points for being at the top of someone's preference ordering, N-2 for being in second place, etc, and 0 for being in last place. Then the winner is the alternative with the most points.
Do both; neither should be difficult to do in Excel. If they come out the same, great. If not, it might well depend on the particulars of the things that people are ranking what sort of method makes most sense.
posted by ROU_Xenophobe at 12:57 PM on March 6, 2006
If 10 people each have their own individual preferences, what are their group preferences? What is the correct way to aggregate their preferences?
It turns out that this is a fundamentally insoluble problem -- there is no globally correct way to aggregate individual preferences; every possible way is terrible in at least one respect. Ken Arrow got a Nobel for showing this.
But that's at a more philosophical or technical level. For your purposes, I would do two things:
(1) Average ranking
(2) Borda count. This runs sort of like a car-racing championship. Each alternative get N-1 points for being at the top of someone's preference ordering, N-2 for being in second place, etc, and 0 for being in last place. Then the winner is the alternative with the most points.
Do both; neither should be difficult to do in Excel. If they come out the same, great. If not, it might well depend on the particulars of the things that people are ranking what sort of method makes most sense.
posted by ROU_Xenophobe at 12:57 PM on March 6, 2006
If my math is right, the average ranking and the Borda count will produce the same ranking, as long as every individual has uniquely ranked every item.
posted by mbrubeck at 2:22 PM on March 6, 2006
posted by mbrubeck at 2:22 PM on March 6, 2006
This thread is closed to new comments.
If you want to find out how related the rankings are you can use correlation tests - this can be used to determine whether people or groups of people give similar answers to questions. For ranked data you want Spearman's Rho (wikipedia) or Kendall's Tau (wikipedia). Both of these are testable for significance, if you need that sort of thing.
"Nonparametric statistics for the behavioural sciences" is a book which covers all sorts of tests, but it's a library visit rather than a book anyone in their right mind would want to purchase.
posted by handee at 10:37 AM on March 6, 2006