Help me understand a statement about statistics, please.
October 11, 2018 6:35 AM   Subscribe

I read an article yesterday about some study results being released, and the final sentence of the piece says "Instead of a margin of error, the study has an overall confidence interval of 95%." The last statistics thing I did in life was have a panic attack and only narrowly pass a stats-for-poli-sci final in grad school 12 years ago. Can someone explain this sentence to me in a way that I would understand?
posted by mccxxiii to Grab Bag (7 answers total) 2 users marked this as a favorite
I’m a statistician and I am not sure what they’re after, to be honest. Sounds like a mangled way of describing correcting for multiple comparisons? I am assuming this is a summary for the popular press - if you want/need to rely on the results perhaps you can get your hands on the original article.
posted by eirias at 6:40 AM on October 11, 2018 [1 favorite]

That's an odd sentence. A confidence interval is the range in which you expect a value to fall with 95% confidence (making some assumptions). So that sentence doesn't make a lot of sense because it doesn't tell you what the confidence interval IS. For example, if the study measured mean age in a university class it might have a 95% confidence interval of 22.1-23.4 . The confidence interval is distributed symettrically around the mean, so the mean age in that study would be 22.75 *and* it would be equally valid to state that "the mean was 22.75 with a margin of error +- 0.65, 19 times out of 20" which is how one normally sees these things reported.

It sounds like the article was a summary of research written by a person who doesn't understand statistics. There wouldn't be an *overall* confidence interval. Each parameter estimated would have a confidence interval which could be set at any level of confidence. So they set their desired confidence at 95%, but the sentence doesn't tell you what the confidence intervals they found at that level of confidence actually *were*.

Could you provide a link to the article?
posted by If only I had a penguin... at 6:44 AM on October 11, 2018 [7 favorites]

Usually when someone talks about a confidence interval and / or a margin of error, then they're measuring / estimating some number. A common example is a poll; for instance the poll might conclude:
Candidate A has 45% support with a Margin of Error of +/- 3%
Typically there is an implicit confidence interval behind that MOE: what they mean is, they are 95% confident that the true number is between 42% and 48%. So the margin of error makes a claim about how the estimate might be wrong, in addition to the question of whether it is.

So, usually a margin of error and a confidence interval go hand in hand. The only way I can think of to have one without the other is if you're measuring something that's discrete -- rather than a continuum of possibilities, they are in distinct categories and can be listed, in order. For instance:
We are 95% confident that your child will be born healthy.
5% of the time it might be wrong, but it could be wrong in any way and the statement makes no claim about which alternative would be true in that case.
posted by dbx at 7:27 AM on October 11, 2018

We are 95% confident that your child will be born healthy.

I don't think it even makes sense to say "there's a 95% confidence interval" in that case. The 95% estimate there is still a parameter estimate. In this case it would probably be made with a logit/logistic regression with the logged odds then transformed into a probability.

So the point estimate would be 95%, but then there would still be a confidence interval around that -- we're 95% sure, that the probability of a healthy birth is between 93.96 and 95.86% (note, in this case the margin of error is symmetric around the mean *when the mean is in the form of logged odds*, which makes it not-symmetric in this probability form...I just did these up in wolfram alpha with a .2 margin of error on the logged odds, I make no guarantees I did the math exactly right, but the idea should be right).
posted by If only I had a penguin... at 8:08 AM on October 11, 2018 [1 favorite]

Here is the piece that I am talking about. (Sorry I can’t link on my phone...) The sentence in question is the very last one.

I feel a little better about myself knowing that maybe this is an imprecise writing thing and not a stats concept that I don’t grok.
posted by mccxxiii at 10:44 AM on October 11, 2018

Thanks for following up mccxxiii. As near as I can tell, the "Hill" article is referring to this pdf. I did a Ctrl+F for "confidence" and there's only one reference to confidence intervals in the article, on page 145. It appears in the title of Appendix 1.3, "Sample sizes and Margins of Error (95% Confidence Interval)". There are explicit margins of error given in the table. So the "Hill" reporter simply made a mistake or doesn't know what they're talking about.
posted by dbx at 11:23 AM on October 11, 2018 [1 favorite]

There are often polls reported in the media that end with "a margin of error of 3 pts 19 times out of 20" and those articles confuse me because they will report a bunch of different findings (percent of likely voters who intend to vote for candidate A, percent who intend to vote for candidate B, percent of women likely to vote, percent of male evangelicals in south who eat peanut butter, etc. etc.) and there's no way the margin of error on each of those things is identical. Every point estimate should have it's own margin of error. But they always just report one number.

I wonder if the reporter is used to thinking of margins of error that way (there's one for the whole study) and when faced with a different margin of error for every number, figured that 95% was somehow the number "overall" and they could just report that instead.
posted by If only I had a penguin... at 11:46 AM on October 11, 2018

« Older A lamp that dims on its own at bedtime   |   Best quality way to connect a Macbook Air to... Newer »

You are not logged in, either login or create an account to post comments