Not my stats homework, I swear
February 15, 2014 10:10 AM Subscribe
An NPR blog cites an NSF study which claims that 26 percent of Americans asked answered that the Sun goes around the Earth, rather than vice versa. Believing that 1 in 4 of my fellow citizens doesn't know that the Earth circles the Sun is hard enough. But thinking about that number, it seems worse than that: if 26% got a 50/50 question wrong, wouldn't another 26% have answered correctly just based on chance rather than knowledge? That would mean that roughly half of Americans didn't know (and then split evenly on their guess). The idea that half of Americans don't know seems intuitively ludicrous to me. Am I missing something in how I think about this? Please help my statistically challenged brain...
The study itself may be nonsense, of course, but my first thought, that people were confused about "revolve," can't be it. The question asked was, ""Does the Earth go around the Sun, or does the Sun go around the Earth?"
So, is it an untrustworthy study, or my assumption that "for every person who gets it wrong, there's a person who got it right by sheer luck" that's to blame, or what?
The study itself may be nonsense, of course, but my first thought, that people were confused about "revolve," can't be it. The question asked was, ""Does the Earth go around the Sun, or does the Sun go around the Earth?"
So, is it an untrustworthy study, or my assumption that "for every person who gets it wrong, there's a person who got it right by sheer luck" that's to blame, or what?
Best answer: It is only 50/50 if you are guessing. Most people probably didn't choose randomly--they thought they knew. So the majority of the people who answered correctly probably knew the right answer You are correct there may have been some guessers in the right answer group but probably not half
posted by limagringo at 10:21 AM on February 15, 2014
posted by limagringo at 10:21 AM on February 15, 2014
Best answer: You have no information about how many of the respondents were guessing. All you know is that half of those who were guessing randomly are included in the 26% who got it wrong.
Also, be wary of making the same incorrect generalization that the title of the article does. It's not 26% of Americans, it's just 572 people in this 2200 person study that answered incorrectly.
posted by ceribus peribus at 10:22 AM on February 15, 2014 [2 favorites]
Also, be wary of making the same incorrect generalization that the title of the article does. It's not 26% of Americans, it's just 572 people in this 2200 person study that answered incorrectly.
posted by ceribus peribus at 10:22 AM on February 15, 2014 [2 favorites]
Response by poster: Ah! Ah! Thanks. What a total brain malfunction. It just didn't occur to me that nobody, or next to nobody, had to be guessing: everyone could have been sure of their answer, whether right or wrong.
posted by tyllwin at 10:31 AM on February 15, 2014 [1 favorite]
posted by tyllwin at 10:31 AM on February 15, 2014 [1 favorite]
ceribus peribus: " It's not 26% of Americans, it's just 572 people in this 2200 person study that answered incorrectly."
Presumably the study is meant to provide a representative sample of Americans, just like an opinion poll. If proper methodologies were followed, then this generalization would in fact be correct. Otherwise there would be little point in conducting such a survey.
posted by Conrad Cornelius o'Donald o'Dell at 10:34 AM on February 15, 2014 [5 favorites]
Presumably the study is meant to provide a representative sample of Americans, just like an opinion poll. If proper methodologies were followed, then this generalization would in fact be correct. Otherwise there would be little point in conducting such a survey.
posted by Conrad Cornelius o'Donald o'Dell at 10:34 AM on February 15, 2014 [5 favorites]
Also, be wary of making the same incorrect generalization that the title of the article does. It's not 26% of Americans, it's just 572 people in this 2200 person study that got it wrong.
Sampling 2200 respondents is plenty to represent the US population assuming the NSF applied proper sampling techniques. This isn't like an online Fox News poll. From the report itself these are telephone interviews and they list the confidence intervals. The NSF paper used the term "Americans" which gives me confidence they sampled and weighted the data correctly to be able to make these claims.
posted by birdherder at 10:35 AM on February 15, 2014 [2 favorites]
Sampling 2200 respondents is plenty to represent the US population assuming the NSF applied proper sampling techniques. This isn't like an online Fox News poll. From the report itself these are telephone interviews and they list the confidence intervals. The NSF paper used the term "Americans" which gives me confidence they sampled and weighted the data correctly to be able to make these claims.
posted by birdherder at 10:35 AM on February 15, 2014 [2 favorites]
I think it's also important to point out that sometimes people lie when surveyed, out of annoyance or for other reasons.
Those 572 may not actually think this.
A different manifestation of this principle (People don't necessarily tell the truth) is known as the "Bradley Effect". That particular manifestation is caused by "social proof", but there are other kinds of motivation, more pernicious, that can cause it.
The point is that the result of this survey tells us what people said, not necessarily what they actually believe.
posted by Chocolate Pickle at 10:53 AM on February 15, 2014 [4 favorites]
Those 572 may not actually think this.
A different manifestation of this principle (People don't necessarily tell the truth) is known as the "Bradley Effect". That particular manifestation is caused by "social proof", but there are other kinds of motivation, more pernicious, that can cause it.
The point is that the result of this survey tells us what people said, not necessarily what they actually believe.
posted by Chocolate Pickle at 10:53 AM on February 15, 2014 [4 favorites]
It just didn't occur to me that nobody, or next to nobody, had to be guessing: everyone could have been sure of their answer, whether right or wrong.
Exactly. The data are consistent with the possibility that 52% of the respondents were just guessing at random. The data are also consistent with the possibility that nobody was guessing, but 26% were confidently dead wrong.
For that matter, the data are also consistent with the possibility that 100% of the respondents were guessing — so long as you assume they were all guessing in a biased way that made them more likely to pick the heliocentric answer rather than the geocentric one. After all, randomly choosing between two answers does not necessarily mean you have a 50/50 chance of chosing each one. (For instance, suppose everyone had said "Okay, if I flip this coin twice in a row and it comes up heads both times, I'll say I'm a geocentrist." That scenario is implausible for other reasons, and we shouldn't take it too seriously — but strictly speaking, it's consistent with the survey results, since it would lead to roughly a quarter of the respondents claiming geocentrism.)
Here's what the data do tell us: "at least 26% of the population either (a) believe in geocentrism, or (b) aren't sure what they believe, or (c) misspoke, lied, or misunderstood the question." There's no scenario consistent with the data in which more than 74% of the respondents were confident, competent, honest heliocentrists.
posted by this is a thing at 11:42 AM on February 15, 2014 [2 favorites]
Exactly. The data are consistent with the possibility that 52% of the respondents were just guessing at random. The data are also consistent with the possibility that nobody was guessing, but 26% were confidently dead wrong.
For that matter, the data are also consistent with the possibility that 100% of the respondents were guessing — so long as you assume they were all guessing in a biased way that made them more likely to pick the heliocentric answer rather than the geocentric one. After all, randomly choosing between two answers does not necessarily mean you have a 50/50 chance of chosing each one. (For instance, suppose everyone had said "Okay, if I flip this coin twice in a row and it comes up heads both times, I'll say I'm a geocentrist." That scenario is implausible for other reasons, and we shouldn't take it too seriously — but strictly speaking, it's consistent with the survey results, since it would lead to roughly a quarter of the respondents claiming geocentrism.)
Here's what the data do tell us: "at least 26% of the population either (a) believe in geocentrism, or (b) aren't sure what they believe, or (c) misspoke, lied, or misunderstood the question." There's no scenario consistent with the data in which more than 74% of the respondents were confident, competent, honest heliocentrists.
posted by this is a thing at 11:42 AM on February 15, 2014 [2 favorites]
A bit of a side-track from the original question, but another interpretation of the NSF study is that a large portion of that 26% think "astrology" means "astronomy".
posted by straw at 11:54 AM on February 15, 2014 [1 favorite]
posted by straw at 11:54 AM on February 15, 2014 [1 favorite]
I'd have answered that the Sun goes around the Earth, too. It explains more phenomena of my daily experience, and allows me to accurately predict the future as related to this question (the Sun will raise tomorrow, roughly over there).
What I'm saying is, it's not a failure of rationality to think this. What changes in your life if you "know" that the Earth rotates around the Sun? Does it improve your life and daily decisions? (I've studied physics with the goal of becoming an astronomer, so this did matter to me at some point.)
posted by dhoe at 1:12 PM on February 15, 2014 [1 favorite]
What I'm saying is, it's not a failure of rationality to think this. What changes in your life if you "know" that the Earth rotates around the Sun? Does it improve your life and daily decisions? (I've studied physics with the goal of becoming an astronomer, so this did matter to me at some point.)
posted by dhoe at 1:12 PM on February 15, 2014 [1 favorite]
To follow on with dhoe, I had a rather embarrassing moment explaining the concept of sun-rise and set with my 3-year-old. I totally starting off explaining how the sun goes around the earth. All of a sudden, I was all, "wait a minute, the hell am I saying?" As a fan of science, and astronomy in particular, I was aghast at my error. However, I forgive myself in that, nearing age 40, it's been a long time since I've been in school. Heh. So, I can totally see myself, if someone came up to me randomly on the street and asked me this question, answering, quite quickly, the wrong answer. And then walking away and going, "D'oh!"
That's what I think about most of these Ammuricans-are-so-dumb surveys.
posted by amanda at 1:25 PM on February 15, 2014 [3 favorites]
That's what I think about most of these Ammuricans-are-so-dumb surveys.
posted by amanda at 1:25 PM on February 15, 2014 [3 favorites]
It seems that wrong answers to this particular question make you uneasy about the intelligence of the general population, but remember that we are all familiar with generally-held beliefs that we dismiss as quaint, rather than disturbing. Among the scientifically-wrong beliefs that we regularly encounter are:
That oxygen burns, or is somehow explosive.
That exposure to cold air, or going outside with wet hair, can cause disease.
That going twice as fast means it will take you twice as long to stop.
That turbulence is caused by "air pockets".
That a heavier body will fall faster.
That Venus is the "morning" (alternatively, "evening") star, or, for that matter, that it is a star at all.
That the new millennium began on January 1st, 2000.
We could all add many items to the list, but you get the idea. The notion that a quarter of the population believes in geocentrism (or just hasn't thought that much about it), while appalling on one level, probably doesn't mean much in the long run.
posted by dinger at 1:54 PM on February 15, 2014
That oxygen burns, or is somehow explosive.
That exposure to cold air, or going outside with wet hair, can cause disease.
That going twice as fast means it will take you twice as long to stop.
That turbulence is caused by "air pockets".
That a heavier body will fall faster.
That Venus is the "morning" (alternatively, "evening") star, or, for that matter, that it is a star at all.
That the new millennium began on January 1st, 2000.
We could all add many items to the list, but you get the idea. The notion that a quarter of the population believes in geocentrism (or just hasn't thought that much about it), while appalling on one level, probably doesn't mean much in the long run.
posted by dinger at 1:54 PM on February 15, 2014
Take this into consideration,
NSF, an organization and much of their fundraising is devoted to increasing advancing science education found that 25% of Americans believe the Sun orbits the Earth. This result suggest that an increase in science education is needed.
Wow! How lucky for the NSF that the study found the results they bank on.
posted by 2manyusernames at 2:51 PM on February 15, 2014 [4 favorites]
NSF, an organization and much of their fundraising is devoted to increasing advancing science education found that 25% of Americans believe the Sun orbits the Earth. This result suggest that an increase in science education is needed.
Wow! How lucky for the NSF that the study found the results they bank on.
posted by 2manyusernames at 2:51 PM on February 15, 2014 [4 favorites]
(Fwiw, NSF is a federal gov agency created to distribute funding to -- among other research -- external organizations like universities that study l evaluate or create models for formal and informal science education. Not saying that exempts anyone from bias, just giving context that they can't do fundraising :))
posted by NikitaNikita at 6:03 PM on February 15, 2014 [4 favorites]
posted by NikitaNikita at 6:03 PM on February 15, 2014 [4 favorites]
> NSF, an organization and much of their fundraising is devoted to increasing advancing science education
This is just plain wrong, as NikitaNikita said.
The National Science Foundation is an independent federal agency created by Congress in 1950 "to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…" With an annual budget of $7.2 billion (FY 2014), we are the funding source for approximately 21 percent of all federally supported basic research conducted by America's colleges and universities. In many fields such as mathematics, computer science and the social sciences, NSF is the major source of federal backing.
On your other point,
> [The NSF] found that 25% of Americans believe the Sun orbits the Earth. This result suggest that an increase in science education is needed. Wow! How lucky for the NSF that the study found the results they bank on.
On the contrary, if these numbers are getting worse over time, that would suggest that the NSF mission is failing, and if it wasn't already such an obvious state of affairs (see, e.g., the fact that we're now having debates between evolution and creationism) it would actually be an embarrassment for the NSF. As it stands, working scientists feel lucky if their NSF division budget stays flat in real dollars instead of declining, in spite of all the happy talk about doubling the science budget in the early aughts.
posted by RedOrGreen at 9:53 PM on February 15, 2014 [2 favorites]
This is just plain wrong, as NikitaNikita said.
The National Science Foundation is an independent federal agency created by Congress in 1950 "to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…" With an annual budget of $7.2 billion (FY 2014), we are the funding source for approximately 21 percent of all federally supported basic research conducted by America's colleges and universities. In many fields such as mathematics, computer science and the social sciences, NSF is the major source of federal backing.
On your other point,
> [The NSF] found that 25% of Americans believe the Sun orbits the Earth. This result suggest that an increase in science education is needed. Wow! How lucky for the NSF that the study found the results they bank on.
On the contrary, if these numbers are getting worse over time, that would suggest that the NSF mission is failing, and if it wasn't already such an obvious state of affairs (see, e.g., the fact that we're now having debates between evolution and creationism) it would actually be an embarrassment for the NSF. As it stands, working scientists feel lucky if their NSF division budget stays flat in real dollars instead of declining, in spite of all the happy talk about doubling the science budget in the early aughts.
posted by RedOrGreen at 9:53 PM on February 15, 2014 [2 favorites]
Substitute "fund raising" for "budget request"
and RedOrGreen I disagree. The government's usual method to fix a problem is to throw more money at it. It isn't that the mission is failing, it is simply that there isn't enough funds to do the mission properly.
and you can't equate a belief in religion with scientific illiteracy
posted by 2manyusernames at 7:25 AM on February 16, 2014 [1 favorite]
and RedOrGreen I disagree. The government's usual method to fix a problem is to throw more money at it. It isn't that the mission is failing, it is simply that there isn't enough funds to do the mission properly.
and you can't equate a belief in religion with scientific illiteracy
posted by 2manyusernames at 7:25 AM on February 16, 2014 [1 favorite]
> The government's usual method to fix a problem is to throw more money at it. It isn't that the mission is failing, it is simply that there isn't enough funds to do the mission properly.
Fair enough, but I wish that applied to science education. I can tell you from first-hand experience that the NSF budget situation has not been pleasant. I'll leave it at that.
> You can't equate a belief in religion with scientific illiteracy.
Obviously, and I wasn't doing that. Many of my co-workers find no conflict between their science and their faith, for example. But I have no problem equating belief in Young Earth Creationism with scientific illiteracy.
Seriously, our best current science favors the Big Bang theory - an ancient universe called into existence in a mysterious way that seems tailor-made for "Let there be light". It seems perverse to insist on a malicious Creator who creates fake evidence of fossils and radioisotopes and distant quasars to - to what, exactly? mess with creation? I don't think that is even good religion, let alone compatible with science.
posted by RedOrGreen at 11:45 AM on February 17, 2014 [1 favorite]
Fair enough, but I wish that applied to science education. I can tell you from first-hand experience that the NSF budget situation has not been pleasant. I'll leave it at that.
> You can't equate a belief in religion with scientific illiteracy.
Obviously, and I wasn't doing that. Many of my co-workers find no conflict between their science and their faith, for example. But I have no problem equating belief in Young Earth Creationism with scientific illiteracy.
Seriously, our best current science favors the Big Bang theory - an ancient universe called into existence in a mysterious way that seems tailor-made for "Let there be light". It seems perverse to insist on a malicious Creator who creates fake evidence of fossils and radioisotopes and distant quasars to - to what, exactly? mess with creation? I don't think that is even good religion, let alone compatible with science.
posted by RedOrGreen at 11:45 AM on February 17, 2014 [1 favorite]
This thread is closed to new comments.
This doesn't follow. You're assuming that roughly 50% of the people asked were guessing randomly, but I don't know why you'd make that particular assumption.
(My assumption on seeing this headline earlier was that some of the 26% had a brain fart. I can totally see myself managing to get this question wrong, at least if I was asked out loud--I'd be too busy being baffled that I was being asked and then throw in that "earth round the sun" isn't something I say very often and god knows what would come out of my mouth. If it was a written 'quiz', then I would assume 'false' incorrect answers were less likely.)
posted by hoyland at 10:18 AM on February 15, 2014 [2 favorites]