Of cakes and counts
August 26, 2011 2:02 AM Subscribe
I run an online cake contest (link in profile) with a judging panel. Is my scoring methodology the most efficient/accurate?
Today, I'm announcing the winners to my online cake contest. Since we had around 300 entries and tons of visitors, I push the judging duties off to a panel of cake-happy judges from all around.
There are two categories of cake: 2D and 3D. There are 5 winning places in the contest per category.
My current methodology is as such:
I ask each judge to supply a list of their top TEN cakes for each category, ranked in order from 1 to 10. I then invert their rank, assign that cake that inverse number of points, and tally everyone's entries up.
I've asked a more math-inclined friend about this before, who agreed that it was as fair as most other options, but I'm wondering if there are other ways to approach this. Every judge has an equal say. No judge communicates with any other judge when they give me their scores. I compile them, and I've seen an entry leapfrog another from just one ballot, because a judge gave a 1st or 2nd place (10 or 9 points) to an entry and none to the entry it leapfrogged.
Since we only have about 10-15 judges in a given year, this means that things can shift dramatically.
I figured that going out to 10 places would help smooth things out, reduce the risk of ties, and frankly keep us from going insane. (Some of the cakes are amazingly good.)
Is there a better way to handle this? Would it make sense to apply a weight to the number of votes for a cake? If a cake gets 10 "9th place" votes, it currently equals a cake getting 2 "1st place" votes... but I don't know if that's a flaw, really. It's just hard to say.
To be clear, the system has worked just fine. I'm more curious from an academic perspective.
Today, I'm announcing the winners to my online cake contest. Since we had around 300 entries and tons of visitors, I push the judging duties off to a panel of cake-happy judges from all around.
There are two categories of cake: 2D and 3D. There are 5 winning places in the contest per category.
My current methodology is as such:
I ask each judge to supply a list of their top TEN cakes for each category, ranked in order from 1 to 10. I then invert their rank, assign that cake that inverse number of points, and tally everyone's entries up.
I've asked a more math-inclined friend about this before, who agreed that it was as fair as most other options, but I'm wondering if there are other ways to approach this. Every judge has an equal say. No judge communicates with any other judge when they give me their scores. I compile them, and I've seen an entry leapfrog another from just one ballot, because a judge gave a 1st or 2nd place (10 or 9 points) to an entry and none to the entry it leapfrogged.
Since we only have about 10-15 judges in a given year, this means that things can shift dramatically.
I figured that going out to 10 places would help smooth things out, reduce the risk of ties, and frankly keep us from going insane. (Some of the cakes are amazingly good.)
Is there a better way to handle this? Would it make sense to apply a weight to the number of votes for a cake? If a cake gets 10 "9th place" votes, it currently equals a cake getting 2 "1st place" votes... but I don't know if that's a flaw, really. It's just hard to say.
To be clear, the system has worked just fine. I'm more curious from an academic perspective.
Best answer: There is, in fact, no "perfect" voting system for any election - Arrow's Impossibility Theorem made three pretty sensible "fairness" claims and then showed that no voting system could fulfil them all.
There's tons of reading you can do on this, and I might even have some old course notes on it somewhere - election theory is a pretty huge field.
posted by The Shiny Thing at 2:56 AM on August 26, 2011
There's tons of reading you can do on this, and I might even have some old course notes on it somewhere - election theory is a pretty huge field.
posted by The Shiny Thing at 2:56 AM on August 26, 2011
This thread is closed to new comments.
You may want to read about some of the other systems as well to understand the pros and cons of each method.
posted by vacapinta at 2:20 AM on August 26, 2011 [1 favorite]