How can I more easily create a ranked list?

April 21, 2009 12:33 PM

Is there a simple program that will help a person rank their preferences of a list of items by presenting them with a series of A vs. B matchups?

Suppose I have a list of items and I want to rank them according to personal preference. I'm looking for a program that would ask me "Which do you prefer: thing X or thing Q?" Then, after asking a series of those questions, would give me a list of things ranked from 1 to 100 (or whatever the number is.)

It would have to have some ability, I imagine, to resolve inconsistencies, in case the person answering the questions ranks A above B, B above C, and C above A.

Does this make sense to everybody? Anybody know of a program that would do this? (Mac preferred, but PC is fine.)

posted by soonertbone to computers & internet (7 comments total)

I should add that Yahoo has an example of something I'd like. They use it to rank college football teams.
posted by soonertbone at 12:42 PM on April 21

Does this make sense to everybody?

Yeah, I don't know of any pre-packaged software that will do this, but I like organizing and ranking things so I've done this in various ways myself.

The easiest, most low-tech way of doing something like this is to use a tournament structure. First, make a randomized list of your things (I've done it with music, so that's just a matter of creating a playlist and doing sort->randomize). Then go through them two by two, keeping the better and deleting the worst. Once you make it through the list, repeat until you get down to the best ones. Although this method works with a minimum amount of comparisons and doesn't require any special software, the downside is that you only end up with shortlist of the best at the end rather than an actual ranked list. Also, depending on luck of the draw, some of the best of the list will get knocked out because they are up against even better ones.

I also wrote a custom program for a project once that would display two random images side-by-side, along with a slider at the bottom. Moving the slider all the way to the left would give the left image a score of 10 and the right image a score of 0, putting it in the center would give them both 5, moving it all the way to the right would give the right 10 and the left 0, etc. The end result of that was that after many comparisons, each image would have a 0-10 score based on how it was ranked against the others. I could have just used single images and a method to give each of them a 0-10 score individually, but I think the comparison method helps make sure that the scoring is objective (it's hard to keep track of what a 5 versus a 6 is when scoring a single image, but it's easy to say this one is better than that one). It takes a lot of comparisons to use that method, but the nice thing is you'll get a rough set of rankings right away and the more comparisons you do the more accurate the results will be.
posted by burnmp3s at 1:41 PM on April 21

Strangely enough, AskMe covered this in January.
posted by nicwolff at 1:55 PM on April 21 [1 favorite has favorites]

Ah, crap. I tried searching.

Still, though, the previous thread doesn't quite answer my question. I'm not inclined to figure out the math/programming and make something myself. Hoping that someone's done the work for me.
posted by soonertbone at 2:15 PM on April 21

Funny answer: Bring a list of all your options to your next eye exam. Eye doctors are good at comparing 2 variables. (which is better A or B... A or B?)

Serious answer: Are you doing this by just your own personal personal preferences, or does this involve a set of survey-style data? If its survey data, you could start with some sort of zoomerang survey.

A friend of mine used to eat M&Ms by taking 2, smacking them into eachother and eating the one that broke first, then repeating until he had only one M&M left...which he would then put in an envelope and mail back to M&M/Mars with the instructions to breed that M&M...

But, you're trying toget a short list of inputs to compare, and then apply some sort of algorithm to get an Ace-beats-King rulesset... then you're talking something way way more complex (in the style of Pandora's song selection, and Netflix's reccommendation generation. For that you need to have categorical variables which each contains.. then you just effectively score your preferences by scoring your subset of data - then setting how far you are willing to deviate from your personal peference.... that'd be a few grand in software development costs... gotta buy more M&Ms somehow...
posted by Nanukthedog at 2:18 PM on April 21

You're describing The Prioritizer. First, you add up to 15 items. Then, you answer A or B questions until you've ranked all the items. Very cool. It probably uses some sort of bubble or insertion sort to present the choices.

My firm is developing something based on that plus some Agile concepts on Prioritisation, but it will not be ready for public consumption for some time.
posted by Cobalt at 2:30 PM on April 21 [2 favorites has favorites]

For the record, in case anybody in the future finds this post and wants to do the same thing: there's a new website called FlickChart which does what I'm after, at least for movies.
posted by soonertbone at 9:42 PM on June 25