Statistically prove the world isn't fair
February 10, 2020 1:18 PM Subscribe
I have two basketball hoops that, in theory, electronically record baskets made. The hoops blatantly under-report. With a big spreadsheet of points earned on each basket, but not shots attempted, by teams in a league can I statistically show the likelihood of one hoop being worse at registering shots than the other?
I'm in a casual sports league that has the above setup with electronic score keeping. (But what about 3 pointers or free throws, you might ask? That's a great question. In theory the ref adjusts the scoreboard as needed. In practice both the shots and the manual correcting are all very rare.)
It's a given that not every basket is actually recorded. This is accepted by everyone as part of the game because the speed and direction of the ball seem to affect the likelihood of the scoring mechanism picking it up. What's not fine is everyone's suspicion that one basket is worse at recording shots than the other.
The league addressed concerns last week by pointing out that the north basket had 231 points recorded and the south basket had 220 points. Seems close enough, it must be fair, right? But there are some very good teams and very bad teams so we all don't think the number of points actually tells us anything.
I have a big spreadsheet containing all the scores of all games played so far, with points broken down by quarter, as well as data for which direction the team was playing towards. The one caveat here is that teams don't switch directions at the half so I don't have control data for each pairing - a given pairing only has one team per basket.
I can envision showing that a given team tends to score less on one basket than the other but this is confounded by the fact that they might be playing a more difficult team that week. Is there a way to control for that with the data I have so that I can reasonably show the southern basket might be worse?
There are obviously a lot of goofy things going on in this league, and I am not worried about changing those. I really just want to know if the stupid southern hoop cost us a close match against an undefeated team last week.
I'm in a casual sports league that has the above setup with electronic score keeping. (But what about 3 pointers or free throws, you might ask? That's a great question. In theory the ref adjusts the scoreboard as needed. In practice both the shots and the manual correcting are all very rare.)
It's a given that not every basket is actually recorded. This is accepted by everyone as part of the game because the speed and direction of the ball seem to affect the likelihood of the scoring mechanism picking it up. What's not fine is everyone's suspicion that one basket is worse at recording shots than the other.
The league addressed concerns last week by pointing out that the north basket had 231 points recorded and the south basket had 220 points. Seems close enough, it must be fair, right? But there are some very good teams and very bad teams so we all don't think the number of points actually tells us anything.
I have a big spreadsheet containing all the scores of all games played so far, with points broken down by quarter, as well as data for which direction the team was playing towards. The one caveat here is that teams don't switch directions at the half so I don't have control data for each pairing - a given pairing only has one team per basket.
I can envision showing that a given team tends to score less on one basket than the other but this is confounded by the fact that they might be playing a more difficult team that week. Is there a way to control for that with the data I have so that I can reasonably show the southern basket might be worse?
There are obviously a lot of goofy things going on in this league, and I am not worried about changing those. I really just want to know if the stupid southern hoop cost us a close match against an undefeated team last week.
IANAStatistician but it seems to me that what you want to do is calculate the correlation between hoops and wins versus expectation (caveat: correlation does not imply causation). In other words, what is the odds ratio between being the south team and being the losing team (or equivalently, between being the north team and being the winning team). Then also calculate the odds ratio between being the winning team and being the better team (approximate that by using the rankings to indicate the expected outcome of the game). If the second OR is much higher, it suggests that the relative difficulty of the matchup swamps out the basket-ness, whereas if they are close then it would indicate that drawing the southern basket would have a noticeable effect on the outcome. Obviously there are a million other factors you could point at (Johnny Big Game was out that week due to a pulled hamstring, whatever) but the point is to get a rough estimate without much effort.
Frankly, I doubt this will have the outcome you want, which is to tell you the reason you lost a close match against the undefeated team was that dastardly southern hoop.
posted by axiom at 1:47 PM on February 10, 2020
Frankly, I doubt this will have the outcome you want, which is to tell you the reason you lost a close match against the undefeated team was that dastardly southern hoop.
posted by axiom at 1:47 PM on February 10, 2020
I can envision showing that a given team tends to score less on one basket than the other but this is confounded by the fact that they might be playing a more difficult team that week. Is there a way to control for that with the data I have so that I can reasonably show the southern basket might be worse?
I dont think you can "control" for that unless you only compared teams' performance against the exact same team while playing in opposite directions or without some sort of outside measure of opponent-quality which would then parse out the differences in points scored due to a weaker/stronger opponent vs direction of play.
Without a measure of opponent quality, and without narrowing your analysis, the thing you want is more data. lots of it. over enough iterations teams would play against opponents of all qualities and the effect of opponent strength would be washed out.
posted by Exceptional_Hubris at 2:15 PM on February 10, 2020 [1 favorite]
I dont think you can "control" for that unless you only compared teams' performance against the exact same team while playing in opposite directions or without some sort of outside measure of opponent-quality which would then parse out the differences in points scored due to a weaker/stronger opponent vs direction of play.
Without a measure of opponent quality, and without narrowing your analysis, the thing you want is more data. lots of it. over enough iterations teams would play against opponents of all qualities and the effect of opponent strength would be washed out.
posted by Exceptional_Hubris at 2:15 PM on February 10, 2020 [1 favorite]
Yes, calculate the mean for each goal per game over a wide enough sample size and you'll see if it is miscounting with a reasonable guess.
If it got messed up at a certain point though, that would be hard to narrow down. I'd be looking at it per season probably.
posted by OnTheLastCastle at 2:35 PM on February 10, 2020
If it got messed up at a certain point though, that would be hard to narrow down. I'd be looking at it per season probably.
posted by OnTheLastCastle at 2:35 PM on February 10, 2020
This is a bit of a fool’s errand. Sure you can craft all sorts of recipes for evidence but statistics don’t even prove things about controlled experiments, and it all gets murkier with observations of the natural world.
If you want to convince someone who wants to believe they are fair, and you’re not dealing with scientists and statisticians, the most reasonable way forward is to drop a few hundred balls (perhaps from various heights and angles) through each and get good controlled data.
Then you might convince someone that there is a widespread and persistent difference in recording.
posted by SaltySalticid at 2:42 PM on February 10, 2020 [5 favorites]
If you want to convince someone who wants to believe they are fair, and you’re not dealing with scientists and statisticians, the most reasonable way forward is to drop a few hundred balls (perhaps from various heights and angles) through each and get good controlled data.
Then you might convince someone that there is a widespread and persistent difference in recording.
posted by SaltySalticid at 2:42 PM on February 10, 2020 [5 favorites]
Can the teams swap sides halfway through? Then it wouldn't matter if one end had an advantage.
posted by tracer at 3:01 PM on February 10, 2020 [3 favorites]
posted by tracer at 3:01 PM on February 10, 2020 [3 favorites]
Swapping sides only solves the problems if both teams make the same exact same kinds of shots and both hoops have the exact same kind of error. Otherwise systematic bias can easily come through, in principle.
posted by SaltySalticid at 3:08 PM on February 10, 2020
posted by SaltySalticid at 3:08 PM on February 10, 2020
Best answer: If the rate at which the electronic counter is failing is high enough, and you have some time on your hands, then you could perform the following experiment: have one or more people shoot at each basket until they successfully sink n shots into each, and count how many times the electronic counter fails. Assume that there is a fixed probability for counter failure for each basket and perform a hypothesis test in which the null hypothesis is that both baskets have the same failure rate. Compute your test statistic et voilà.
Why does the failure rate have to be high enough? If the failure rate is too close to 0, then you'll have to collect a large sample in order to detect a statistical difference. If you have a guess about the failure rate (0.1? 0.01?) and an opinion on how big the difference would have to be for you to care about it, then you can theoretically work backwards from the test statistic to determine how big of an n you need to detect that difference with high probability.
posted by jomato at 3:10 PM on February 10, 2020 [1 favorite]
Why does the failure rate have to be high enough? If the failure rate is too close to 0, then you'll have to collect a large sample in order to detect a statistical difference. If you have a guess about the failure rate (0.1? 0.01?) and an opinion on how big the difference would have to be for you to care about it, then you can theoretically work backwards from the test statistic to determine how big of an n you need to detect that difference with high probability.
posted by jomato at 3:10 PM on February 10, 2020 [1 favorite]
Best answer: I would definitely approach this through experiment rather than going back into old records. If you go back through old records you'll be actually trying to make a case for something you think is happening rather than just straightforwardly finding out if it's happening or not. If you do an event where people just throw baskets, have a manual score taker and compare it to what the sensor tells you, you'll have incontrovertible proof one way or the other, with no bias.
posted by bleep at 3:24 PM on February 10, 2020 [2 favorites]
posted by bleep at 3:24 PM on February 10, 2020 [2 favorites]
I think experimenting would be the way to go.
Yes there are absolutely ways to account for the impact of better and worse teams, but for that stuff to mean anything you need high sample sizes, and team rosters change over time (presumably). I suspect you don't want to go down the road of tracking performance for every single player.
posted by juv3nal at 5:06 PM on February 10, 2020
Yes there are absolutely ways to account for the impact of better and worse teams, but for that stuff to mean anything you need high sample sizes, and team rosters change over time (presumably). I suspect you don't want to go down the road of tracking performance for every single player.
posted by juv3nal at 5:06 PM on February 10, 2020
NB even if you assume team rosters do not change, players do get better (with experience, say) or worse (age, injury) over time too.
posted by juv3nal at 5:07 PM on February 10, 2020
posted by juv3nal at 5:07 PM on February 10, 2020
I mean, assuming that the baskets are randomly assigned (eg by a coin flip) and you have a large enough sample, the total number of points scored should be ~the same on each basket.
You can get fancier if you want, but that's where I'd start.
posted by Betelgeuse at 5:08 PM on February 10, 2020 [2 favorites]
You can get fancier if you want, but that's where I'd start.
posted by Betelgeuse at 5:08 PM on February 10, 2020 [2 favorites]
Assuming that answering this question is purely recreational:
Can you load up your data in R or a similarly sophisticated stats package (ie, better than Excel)? This is a good case for a mixed-effects model. Assume the number of points per game per team is Poisson-distributed, and let team ID have random effects in the model, both on their own score (offense) and the opponent's (defense). Let hoop ID have a fixed effect on team score. See if the fixed effect is statistically significant. Or, better, run two versions of the model, one with and one without a fixed effect of hoop ID, and compare them using Akaike's Information Criterion to see if the one with hoop ID performs better.
If the above seems like more trouble than it's worth, it probably is. OTOH if the above seems like a fun way to learn some statistics that's slightly more sophisticated than intro-level stuff, it probably is. I'd say it's also probably the most principled way to answer your question given the data you have, but even if you find a positive result that will only be suggestive, not definitive proof.
posted by biogeo at 6:05 PM on February 10, 2020 [2 favorites]
Can you load up your data in R or a similarly sophisticated stats package (ie, better than Excel)? This is a good case for a mixed-effects model. Assume the number of points per game per team is Poisson-distributed, and let team ID have random effects in the model, both on their own score (offense) and the opponent's (defense). Let hoop ID have a fixed effect on team score. See if the fixed effect is statistically significant. Or, better, run two versions of the model, one with and one without a fixed effect of hoop ID, and compare them using Akaike's Information Criterion to see if the one with hoop ID performs better.
If the above seems like more trouble than it's worth, it probably is. OTOH if the above seems like a fun way to learn some statistics that's slightly more sophisticated than intro-level stuff, it probably is. I'd say it's also probably the most principled way to answer your question given the data you have, but even if you find a positive result that will only be suggestive, not definitive proof.
posted by biogeo at 6:05 PM on February 10, 2020 [2 favorites]
Response by poster: To wrap things up - a lot of confirmation here with my gut feeling that we couldn't really solve this problem as described. But there were a lot of examples of really cool tools to analyze the data so that gave me some things to play with.
In the end we came into the playoffs as the last place seed and ended up winning all the way straight to the championship game, which we also won in a nailbiter of an ending. Don't ask me how we managed to do that!
Due to our low ranking the other teams always got to pick their side, and we still ended up winning, so I think the conclusion here is that the hoops are reasonably equal.
posted by Nonsteroidal Anti-Inflammatory Drug at 8:47 AM on March 19, 2020 [1 favorite]
In the end we came into the playoffs as the last place seed and ended up winning all the way straight to the championship game, which we also won in a nailbiter of an ending. Don't ask me how we managed to do that!
Due to our low ranking the other teams always got to pick their side, and we still ended up winning, so I think the conclusion here is that the hoops are reasonably equal.
posted by Nonsteroidal Anti-Inflammatory Drug at 8:47 AM on March 19, 2020 [1 favorite]
This thread is closed to new comments.
You will need a column where you differentiate say Red or Blue basket. Got that? Good.
Now, Descriptive Statistics will be in the data tab under Data Analysis. Filter your list into Red with all the goals that went in there. And then make a separate list that's all the blue ones.
Run Descriptive Statistics by highlighting the Red Goals column, check that your data uses labels in the top row, select summary statistics checkmark and you can output onto a new sheet (or wherever). Look at what the MEAN (average) is for both. Are they significantly different?
Your output will look like this btw I just put in a few numbers and ran it.
Goals
Mean 4.25
Standard Error 0.853912564
Median 4.5
Mode #N/A
Standard Deviation 1.707825128
Sample Variance 2.916666667
Kurtosis 0.342857143
Skewness -0.752837199
Range 4
Minimum 2
Maximum 6
Sum 17
Count 4
With enough games, this should give you a reasonable idea if the Red or Blue goal is jacked up. I did Descriptive Statistics because this gives you a lot of other numbers that might help you pick out irregularities too and it's cool.
posted by OnTheLastCastle at 1:45 PM on February 10, 2020