AlgorithmFilter
December 10, 2006 4:05 AM Subscribe
How do you fairly score a pub quiz?
The quiz I run always has wildly varying team sizes - from one guy on his own to groups of eight or nine. It seems obvious to me that a sole participant who scores 95% has done better than a group of ten who score 100%, but I'd like an algorithm that will calculate a fair score for every team. I've tried to create one using exemplar data but I just don't seem to be able to hit the right combination of terms. My basic data set consists of (number of team members) and (score out of 100).
Any statisticians out there with a solution?
Bonus points if the algorithm is easily understandable to the general public.
You have to write in html to get returns and other fancy stuff. HTML basics
posted by gregoryc at 4:18 AM on December 10, 2006
posted by gregoryc at 4:18 AM on December 10, 2006
Response by poster: I did, I promise! Trust me, I know how to use HTML. Everything looked fine in preview and then out plopped this ugly block of text.
posted by alby at 4:23 AM on December 10, 2006
posted by alby at 4:23 AM on December 10, 2006
The pub quizzes I go to don't bother with anything so complex. Instead they just institute rules about team size. One says that if you have a team of more than N people, you can't win the first place prize. Another says you can't win a prize at all. I guess that isn't effective if you're not giving out a prize, though.
posted by brett at 4:24 AM on December 10, 2006
posted by brett at 4:24 AM on December 10, 2006
Googling for handicapping for different team sizes gets surprisingly good results. And that's not even a snark; the first link is the paper cited below.
Here is a business school paper on handicapping bar quizzes based on team size (source PDF) but it might have more math than you want. Team size advantage is also a common problem common in golf and soccer, so it may be worth seeing how they deal with it.
posted by ardgedee at 4:40 AM on December 10, 2006
Here is a business school paper on handicapping bar quizzes based on team size (source PDF) but it might have more math than you want. Team size advantage is also a common problem common in golf and soccer, so it may be worth seeing how they deal with it.
posted by ardgedee at 4:40 AM on December 10, 2006
Response by poster: (I never even considered "handicapping" as a search term.)
The paper is good, but involves the quizmaster (i.e. me) having some idea of how participants are going to score (i.e. a history of results) before the quiz begins. As the teams change each week that data isn't available to me - plus I'd like to be able to use the same algorithm (with the same constants, if constants are necessary) week in, week out.
posted by alby at 5:16 AM on December 10, 2006
The paper is good, but involves the quizmaster (i.e. me) having some idea of how participants are going to score (i.e. a history of results) before the quiz begins. As the teams change each week that data isn't available to me - plus I'd like to be able to use the same algorithm (with the same constants, if constants are necessary) week in, week out.
posted by alby at 5:16 AM on December 10, 2006
Went to a pub quiz a few weeks ago where team size served as a tie-breaker.
posted by Carol O at 5:17 AM on December 10, 2006
posted by Carol O at 5:17 AM on December 10, 2006
You would have to adjust a bit to get exactly the right balance, but how about a simple rule like: your score is your percentage minus a 2% penalty for the number of members on your team. A solo person that gets 95% would get a final score of 93%, and the group of ten with a score of 100% would get a final score of 90%.
posted by underwater at 6:34 AM on December 10, 2006
posted by underwater at 6:34 AM on December 10, 2006
I run a weekly pub quiz in New York, and there are two reasons why I find this is not a big issue.
First of all, we keep a cap of five people per team. (We tell people four is the max, but someone's girlfriend or brother always shows up late or whatever). The point of activities like pub quizzes is to be social, not to win big bucks. People do come and play solo (and often do quite well), but we made an honest attempt to foster a commnity by actively encouraging the breaking up of larger groups and the bringing-together of people who come solo.
We encourage this by giving out lots of smaller prizes for non-winning teams. We give out prizes for things like Best Team Name, Best Dressed Player, Best Wrong Answer, a random 6th place prize, that sort of thing. And the prizes are things like snacks, trinkets, some dollar-store stuff, promotional swag from the bar... it takes the edge off not being one of the winning teams.
Second of all, the larger a team is, the weaker the drawing power of any one prize. We have quite a few sponsors who give us some decent top prizes at this point, but like my co-presenter makes a point of saying, it's not real estate. If nine people were to win (for example) a $25 bar tab prize or two tickets to a show, or even $100 cash, how many drinks is that per person? Who gets to go to the show, and what do the other seven people actually win? Would that really be worth it?
When I see a large group come in and I want them to break into smaller groups, I would have no problem telling them all of this (and I often do), but I lead with telling them they'd win twice as many prizes if they played two teams. Which is generally the truth, and does the trick without having to put the bad-cop hat on.
That may not answer the question, but I figured I'd share our way of getting around the team-size issue without resorting to algebra.
posted by chicobangs at 7:19 AM on December 10, 2006
First of all, we keep a cap of five people per team. (We tell people four is the max, but someone's girlfriend or brother always shows up late or whatever). The point of activities like pub quizzes is to be social, not to win big bucks. People do come and play solo (and often do quite well), but we made an honest attempt to foster a commnity by actively encouraging the breaking up of larger groups and the bringing-together of people who come solo.
We encourage this by giving out lots of smaller prizes for non-winning teams. We give out prizes for things like Best Team Name, Best Dressed Player, Best Wrong Answer, a random 6th place prize, that sort of thing. And the prizes are things like snacks, trinkets, some dollar-store stuff, promotional swag from the bar... it takes the edge off not being one of the winning teams.
Second of all, the larger a team is, the weaker the drawing power of any one prize. We have quite a few sponsors who give us some decent top prizes at this point, but like my co-presenter makes a point of saying, it's not real estate. If nine people were to win (for example) a $25 bar tab prize or two tickets to a show, or even $100 cash, how many drinks is that per person? Who gets to go to the show, and what do the other seven people actually win? Would that really be worth it?
When I see a large group come in and I want them to break into smaller groups, I would have no problem telling them all of this (and I often do), but I lead with telling them they'd win twice as many prizes if they played two teams. Which is generally the truth, and does the trick without having to put the bad-cop hat on.
That may not answer the question, but I figured I'd share our way of getting around the team-size issue without resorting to algebra.
posted by chicobangs at 7:19 AM on December 10, 2006
The pub I go to handles it by having a prize for best small team. The highest-scoring team wins regardless of team size.
posted by kirkaracha at 7:40 AM on December 10, 2006
posted by kirkaracha at 7:40 AM on December 10, 2006
To make an algorythm it would be interesting to know in general the effect of the number of team members on the quality of the outcome, there is I suspect a point above which the additional participants interfere with good decisions, and confuse the thinking, sometimes just adding noise.
What about a limit on the space the team can occupy, marking the floor with chalk, that would be interesting for the social dynamic, a mix between student games and pub quiz. Teams of one are dubious, and if as is the case in the UK you pay in on a percapita basis, they should be disciuraged.
To make it more sociable, why not have an acceptable team size band, say 3-5 and any larger get one member re-allocated to an undersized team, selecting the candidate at random, that'll sort em out?
posted by SwissTommy at 7:53 AM on December 10, 2006
What about a limit on the space the team can occupy, marking the floor with chalk, that would be interesting for the social dynamic, a mix between student games and pub quiz. Teams of one are dubious, and if as is the case in the UK you pay in on a percapita basis, they should be disciuraged.
To make it more sociable, why not have an acceptable team size band, say 3-5 and any larger get one member re-allocated to an undersized team, selecting the candidate at random, that'll sort em out?
posted by SwissTommy at 7:53 AM on December 10, 2006
Here's a thought:
If a team of eight people gets a question wrong, that's eight people who couldn't figure it out. They should score -8. So, if a large group of 8 scores 19/20, I think it's fair to penalize them 8 points for their wrong answer. They score 12/20.
This would be equivalent to a group of 4 getting 18/20, or a group of two scoring 16/20. Seems pretty fair.
posted by MotorNeuron at 7:59 AM on December 10, 2006
If a team of eight people gets a question wrong, that's eight people who couldn't figure it out. They should score -8. So, if a large group of 8 scores 19/20, I think it's fair to penalize them 8 points for their wrong answer. They score 12/20.
This would be equivalent to a group of 4 getting 18/20, or a group of two scoring 16/20. Seems pretty fair.
posted by MotorNeuron at 7:59 AM on December 10, 2006
The hard part is not coming up with a working algorithm -- whatever it is, as long as you apply it evenly, it is what it is -- it's explaining to drunk people so that they understand it and won't feel the need to complain about it.
What about teams where two people "are just there to hang out and aren't answering questions, honest, we don't know anything?" What about teams where someone comes in halfway or leaves early? How do you score them? More importantly, how do you explain to them how they've been scored?
Avoid this mess if you at all can. It'll make you crazy, and make the night seem a whole lot like work..
posted by chicobangs at 8:08 AM on December 10, 2006
What about teams where two people "are just there to hang out and aren't answering questions, honest, we don't know anything?" What about teams where someone comes in halfway or leaves early? How do you score them? More importantly, how do you explain to them how they've been scored?
Avoid this mess if you at all can. It'll make you crazy, and make the night seem a whole lot like work..
posted by chicobangs at 8:08 AM on December 10, 2006
As a similar but slightly different alternative to what underwater writes, how about adding points for small teams?
Instead of penalizing a large team for getting the wrong answer, reward a small team more than a large team for getting the right answer. Say if everynoe gets 10 points for each correct answer, you get 1 point for each person your team is UNDER the maximum, or UNDER some established number as the upper limit for a large team.
For example, a team of 1 person would get 20 points for a correct answer, a team of 2 people 19 points, and a team fo 10 people only 10 points.
posted by jckll at 8:27 AM on December 10, 2006
Instead of penalizing a large team for getting the wrong answer, reward a small team more than a large team for getting the right answer. Say if everynoe gets 10 points for each correct answer, you get 1 point for each person your team is UNDER the maximum, or UNDER some established number as the upper limit for a large team.
For example, a team of 1 person would get 20 points for a correct answer, a team of 2 people 19 points, and a team fo 10 people only 10 points.
posted by jckll at 8:27 AM on December 10, 2006
Ick, don't give a bonus.
The lone player who scores 95% didn't do better, because he forgot to bring someone along who knows the stuff he doesn't know. That's as much a failure of strategy as not paying attention to current events.
It also encourages socialization and gets more people into the pub.
What's more, if a small team beats a larger team due to some pity bonus, I think that would take away some of the fun of the victory. Scoring should be completely objective.
posted by fleacircus at 9:43 AM on December 10, 2006
The lone player who scores 95% didn't do better, because he forgot to bring someone along who knows the stuff he doesn't know. That's as much a failure of strategy as not paying attention to current events.
It also encourages socialization and gets more people into the pub.
What's more, if a small team beats a larger team due to some pity bonus, I think that would take away some of the fun of the victory. Scoring should be completely objective.
posted by fleacircus at 9:43 AM on December 10, 2006
Collect data on team performance as a function of the number of players. Work out the mean score for teams of each size. Call that M(t) for t being the size of the team. Work out the average of all the M(t) values, and call that A. Adjust the scores of future games by A - M(t).
Here's an example with only two team sizes: Suppose that 2-man teams score 80 points on average and 3-man teams score 90 points on average. M(2)=80 and M(3)=90. A=85. Now play another game. The 2-man team scores 79 and the three man team scores 82. Adjusting the scores gives the 2-man team 84 points and the three-man team 77 points, so the 2-man team wins despite the lower actual score.
There's a good chance that there isn't a significant difference between team scores. If that's the case then you can skip this whole exercise. I believe you'd use ANOVA to perform this test. MS excel has ANOVA built in, but it's a good idea to read up on it first so you know what yo'ure doing.
posted by jewzilla at 10:29 AM on December 10, 2006
Here's an example with only two team sizes: Suppose that 2-man teams score 80 points on average and 3-man teams score 90 points on average. M(2)=80 and M(3)=90. A=85. Now play another game. The 2-man team scores 79 and the three man team scores 82. Adjusting the scores gives the 2-man team 84 points and the three-man team 77 points, so the 2-man team wins despite the lower actual score.
There's a good chance that there isn't a significant difference between team scores. If that's the case then you can skip this whole exercise. I believe you'd use ANOVA to perform this test. MS excel has ANOVA built in, but it's a good idea to read up on it first so you know what yo'ure doing.
posted by jewzilla at 10:29 AM on December 10, 2006
My experience with pub quizzes is that large teams, past a certain point, are actually a handicap.
If there's someone on the team who's sure, then fine.
But for every question where you're not sure, you've got a ton of different opinions on what the answer is and a bitter argument going on.
I think there's something of an exponential curve whereby a six-person team is four times more likely to get that question wrong than a three-person team, and 36 times more likely to get into a big fight and sulk about it afterward.
The suggestion to use team size as a tie-breaker is useful, but otherwise I don't think you should sweat it.
posted by AmbroseChapel at 12:21 PM on December 10, 2006
If there's someone on the team who's sure, then fine.
But for every question where you're not sure, you've got a ton of different opinions on what the answer is and a bitter argument going on.
I think there's something of an exponential curve whereby a six-person team is four times more likely to get that question wrong than a three-person team, and 36 times more likely to get into a big fight and sulk about it afterward.
The suggestion to use team size as a tie-breaker is useful, but otherwise I don't think you should sweat it.
posted by AmbroseChapel at 12:21 PM on December 10, 2006
I've participated in quite a few pub quizzes, and the best arrangement I've seen for this was that you could have as many people as you wanted, but for each person above 6, your total was penalised 1 point.
Of course, this will be more or less effective depending how many points you're giving out overall... the good thing about it is that it's simple, it works, everyone understands it and it doesn't lead to having to break up teams or having people be narky over team-size violations.
posted by pompomtom at 2:46 PM on December 10, 2006
Of course, this will be more or less effective depending how many points you're giving out overall... the good thing about it is that it's simple, it works, everyone understands it and it doesn't lead to having to break up teams or having people be narky over team-size violations.
posted by pompomtom at 2:46 PM on December 10, 2006
My experience with pub quizzes is that large teams, past a certain point, are actually a handicap.
This is very, very true. Large teams hinder the "blink" factor, where somebody knows the right answer without knowing why. What happens in large groups (and I consider five or six a large group) is that you get people who shouldn't contribute railroading their answers through, or just suggesting answers that they're really not sure about.
The last time I did a pub quiz, I kept two sets of scores: the scores for our team as a whole, and a score of the answers I would have submitted if I were on my own. I won.
posted by jdroth at 3:53 PM on December 10, 2006
This is very, very true. Large teams hinder the "blink" factor, where somebody knows the right answer without knowing why. What happens in large groups (and I consider five or six a large group) is that you get people who shouldn't contribute railroading their answers through, or just suggesting answers that they're really not sure about.
The last time I did a pub quiz, I kept two sets of scores: the scores for our team as a whole, and a score of the answers I would have submitted if I were on my own. I won.
posted by jdroth at 3:53 PM on December 10, 2006
If I went to a pub quiz where they suggested some sort of team-load handicapping system and it took more than 2 seconds to explain (anyone who chooses to participate all by themselves gets 5 bonus points, for example) I would roll my eyes and very likely not go back.
I'm there for something to do socially with a few of my friends. I think you would be better served (and serve your audience better) by spending the time encouraging people to find teams and taking a second to say "anyone here have (max number - 1) and willing to take on a single? Any singles looking for a team??
posted by phearlez at 12:13 PM on December 11, 2006
I'm there for something to do socially with a few of my friends. I think you would be better served (and serve your audience better) by spending the time encouraging people to find teams and taking a second to say "anyone here have (max number - 1) and willing to take on a single? Any singles looking for a team??
posted by phearlez at 12:13 PM on December 11, 2006
This thread is closed to new comments.
posted by alby at 4:07 AM on December 10, 2006