Framed by Science.

November 19, 2011 5:02 PM

What are the most reasonable, well-informed, and totally incorrect scientific papers?

Which papers look really good, yet have been proven wrong? Which papers are hard to kill even though they have been debunked? Which papers would trick the general population?

posted by Knigel to science & nature (26 answers total)

A well-known example is the Wakefield study that "linked" autism to childhood vaccines in the late 1990s and sparked the debate we've all heard about (oh, Jenny McCarthy), and can pretty reasonably be blamed for recent increases in the incidence of measles. His study (really a case study of 12 cases) was originally printed in the Lancet, a well-respected British medical journal, in 1998 but has since been retracted (in 2010 i think) and Wakefield lost his medical license.
posted by moshimosh at 5:11 PM on November 19, 2011 [9 favorites]

Rejecta Mathematica "is a real open access online journal publishing only papers that have been rejected from peer-reviewed journals in the mathematical sciences."
posted by caek at 5:11 PM on November 19, 2011

The Sokal hoax.
posted by Blazecock Pileon at 5:18 PM on November 19, 2011 [1 favorite]

Phlogiston theory
posted by Cuspidx at 5:21 PM on November 19, 2011 [1 favorite]

Ancel Keys lipid hypothesis paper in which he discusses the dangers of saturated fat, using data from his cherry-picked seven countries. We're seeing that saturated fat is not the cause of heart disease any longer with all of these low carbohydrate diet studies that have been coming out for the past 10 years or so.
posted by sunnychef88 at 5:58 PM on November 19, 2011 [9 favorites]

George Gaylord Simpson's 1943 Mammals and the Nature of Continents is a detailed rebuttal of the then-controversial theory of plate tectonics:

"The evidence definitely opposes drifting or transoceanic continents and favors stable continents. Statements of intercontinental faunal resemblances are often misleading and their interpretations have usually been subjective, unreliable, and unscientific."

"The fact that almost all paleontologists say that paleontological data oppose the various theories of continental drift should, perhaps, obviate further discussion of this point and would do so were it not that the adherents of these theories all agree that paleontological data do support them. It must be almost unique in scientific history for a group of students admittedly without special competence in a given field thus to reject the all but unanimous verdict of those who do have such competence."
posted by Paragon at 5:58 PM on November 19, 2011 [1 favorite]

Lamarckian evolution.
posted by axiom at 6:05 PM on November 19, 2011

The Schoen scandal was a Big Deal in the field, although the fact that the papers looked really good was actually part of their downfall (people who knew the limits of particular equipment knew that you simply couldn't get error bars that looked like that.)

On the intentionally silly end, the Journal of Irreproducible Results (sample article).
posted by tchemgrrl at 6:28 PM on November 19, 2011 [1 favorite]

The Lamarckism example is notable for a second reason that may interest you: it had been much derided for centuries following Darwin's theories, but recent advances in the field of epigenetics indicate that some traits acquired during an organism's life can be passed to offspring.

In other words, sometimes debunked theories later get un-debunked!

Oh yeah, my contribution is a paper by George Ricaurte showing that a single dose of MDMA could cause Parkinson's Disease. It was widely publicized, and seems to still be widely believed, even though he later retracted the paper (his lab didn't use MDMA, but methamphetamines). (Bogus data is less interesting than bogus reasoning, so perhaps not the best example).
posted by Talisman at 7:16 PM on November 19, 2011 [1 favorite]

It may also be worthwhile investigating some well-accepted neuroscience findings, for example:

* That the corpus callosum thickness differs appreciably between sexes (this one seems to toggle between accepted and not).

* That brain size is correlated with intelligence (debunked, and potentially un-debunked).

These fascinate me because their acceptance seems more closely linked to the politics of equality, unlike what happens in, say, mathematics.
posted by Talisman at 7:22 PM on November 19, 2011 [2 favorites]

Try Retraction Watch.
posted by demiurge at 7:46 PM on November 19, 2011 [2 favorites]

There was a huge study of identical twins who had been put up for adoption and were separated. The idea was that they were genetically identical but had been raised in different environments, and that seemed a way to get a peek into the nature-versus-nurture argument.

As it turned out, the primary researcher (Cyril Burt) was strongly in the "nature" camp, and he falsified a lot of the data to prove his side was right.
posted by Chocolate Pickle at 7:53 PM on November 19, 2011

Eric Poehlman was one of the leading scientists in his field (aging), and was jailed after falsifying a good bit of data (the jail time was because he'd used the falsified data to apply for federal grants). I've been told that his work was widely cited and respected, and that its loss as a trustworthy source pretty well screwed a lot of other scientist up.
posted by Dr.Enormous at 8:19 PM on November 19, 2011 [1 favorite]

Which papers are hard to kill even though they have been debunked?

You should look at the work of John Ioannides. He has written a number of articles detailing how frequently findings published in medical journals continue to be cited long after they have been disproven. Here's the abstract to his recent article on the topic in the Journal of the American Medical Association.
posted by googly at 8:27 PM on November 19, 2011

A paper wrongly concluding a link between coffee consumption and pancreatic cancer is now used in many epidemiology classes as a perfect example of how a seemingly obvious mistake doesn't always reveal itself until after a study has been published. This study's problem was that the comparison group included people with gastrointestinal disorders, many of whom had stopped drinking coffee because of their health condition, whereas the pancreatic cancer group wouldn't have had a physiological reason to stop drinking coffee. Therefore when the study asked about exposure in the present, they were comparing people whose present coffee consumption properly represented their past exposure with people whose present coffee consumption was way less than what they had consumed in the past. There is a wonderful summary that includes the original paper, an explanation, and some subsequent studies.
posted by gubenuj at 9:57 PM on November 19, 2011 [10 favorites]

The story of polywater is a good example of a phenomenon that was "discovered", published, and then later debunked.
posted by Multicellular Exothermic at 11:34 PM on November 19, 2011

This may not be precisely what you are looking for, but the Joachim Boldt investigation of an anesthesiologist who faked his research results.
posted by treehorn+bunny at 11:58 PM on November 19, 2011

Somebody already mentioned the Sokal Hoax. I'm not sure it's quite what you're looking for, but the story behind it is fascinating, particularly if you have any interest at all in the interaction/hostilities between the Humanities subjects and the 'hard' sciences. It's definitely worth a look. If you find yourself as interested by it as I was, the authors have also published a book called Intellectual Impostures that goes into much more detail about the kinds of problems the hoax was originally trying to address.

(Fair warning about the book, though: it quotes heavily from some pretty dense critical writings by a lot of French theorists, many of whom can be difficult to read even if you have a background in the subjects they're writing in. Take a look at an excerpt first; if you find your eyes glazing over as soon as you hit the excerpts, you might not want to bother with the whole book.)
posted by anaximander at 4:59 AM on November 20, 2011

Beginning In 1861, the French physician Paul Broca, published research indicating that many patients who had difficulty speaking after a stroke or other brain damage had lesions in the same area of the left frontal lobe. However his research was seen as supporting phrenology, and the idea that specific areas of the cortex had specialized functions had been debunked by Pierre Flourens. And Broca's research was contradicted by the findings of other neurologists who reported that other patients with speech impairments had damage to other parts of the brain and damage to the left frontal lobe wasn't always associated with speech impairment.

Broca's research has since been rehabilitated, and the area of the cortex he identified is now named after him. (It's now understood that Broca's area isn't the only area of the brain involved in speech production, and some people are right rather than left dominant for language, the same way people can be left or right handed, so the contradictory findings aren't actually contradictory.)

Florens' research is now seen as somewhat suspect. His experiments selectively ablating parts of animal brains, if done carefully, should have produced some evidence of localization (and later research using the same technique has). But somehow Florens managed not to.
posted by nangar at 5:51 AM on November 20, 2011

Now, Crick worked out the combinatorial math... and found that with triplets of 4 possible bases, one has to eliminate 44 of the 64 possiblilities as nonsense codons, to make a comma-free code. Voila! That leaves 20 valid codes for the 20 amino acids, saving parsimonious Nature from any sinful profligacy!

(Personally, as a scientist who works on RNA splicing, I also think the notion that the overarching notion that Nature is not wasteful is bunk and a holdover from early work on (admittedly amazing) systems such as the lac operon. Nature is willing to throw away an amazing amount of chemical energy to explore the possibility space of nucleotide sequence.)
posted by Jorus at 5:58 AM on November 20, 2011

http://en.wikipedia.org/wiki/Johann_Beringer
posted by yoyo_nyc at 8:44 AM on November 20, 2011

The American psychologist Henry Goddard, Research Director at the Vineland Training School for Feeble-Minded Girls and Boys in New Jersey in the early 20th century and later a professor at Ohio State University, was responsible for the publication of the first English translation of Binet and Simon's intelligence test and coined the term "moron" as designation for mild mental retardation. He also developed a theory that "moronism" was a single gene recessive trait, and advocated a method of sight recognition of moronism.

In 1912 Goddard published a case study of a New Jersey family (The Kallikak Family: A Study in the Heredity of Feeble-Mindedness), which he described as having a high rate of moronism and exhibiting a recessive pattern of inheritance. This was followed by the publication of additional case studies of families of inmates of the Vineland Training School in 1914 (Feeble-mindedness: Its causes and consequences), and further research indicating high rates of moronism among East European immigrants.

There were some fairly glaring methodological problems with Goddard's research. Most of the diagnoses in his case studies relied on his sight recognition technique rather than testing, and frequently were based on second or third hand descriptions of deceased family members (where vague descriptions like 'used to drink a lot', 'was once arrested for delinquency' or 'was odd' were counted as evidence of moronism). Goddard described a procedure of having people re-diagnosed by a second assistant if the data initially collected fail to fit the expected pattern. His diagnostic interviews with recent immigrants were conducted through translators of unknown reliability and sometimes resulted in people with advanced degrees from European universities, according to his notes, being classified as having the 'mental age' of pre-adolescents.

Goddard's research was nevertheless widely accepted at the time, and was influential in the passage of legislation restricting immigration, and laws in many states permitting the forced sterilization of the feeble-minded (in practice often juvenile delinquents classified as 'moral imbeciles' in the terminology of the time). "Kallikak", the anonymized name of the family in his best known case study entered slang vocabulary as a synonym of "redneck". ("Moron" is still with us of course, though it's no longer used as a technical term.)

Goddard's theory of moronism and his research supporting it seemed to confirm ethnic, regional and class prejudices prevalent at the time, and his research seems to have only come under critical scrutiny when those attitudes began to change - and when supposedly mentally inferior East European immigrants started entering American universities in large numbers and sometimes majoring in psychology.

The still-current idea that members of the lower classes and residents of rural areas are genetically inferior due to inbreeding probably owes a lot to Goddard's research.
posted by nangar at 9:21 AM on November 20, 2011

There's a recent one in nanotechology that caused an uproar in the scientific community a few years back. In short, Chinese researchers fed some metal compound to lab rats and the rats all died from the toxicity. Turns out they fed a human equivalent of an one-off dose of twenty pounds of an already well-known toxic metal to the poor rats, which boggles the mind as to why, and published the results as revolutionary. The media lapped it up.
I don't have the references under-hand but I'll check back into this thread to add them, hopefully in a few days.
posted by ruelle at 11:05 AM on November 20, 2011

I guess I should add (and I hope I'm not going on too much of a rant):

Some of the morons Goddard described who lived at the Vineland Training School and were the basis of the case studies in Feeble-mindedness: Its causes and consequences were pretty high-functioning. One them designed a working human-powered Merry-Go-Round which was built by the retarded kids and actually used by the institution. He redesigned it after the first one failed under heavy use. The second version was still in use when Goddard published Feeble-mindedness: Its causes and consequences. (Goddard reproduced the kid's drawings for the MerryGoRounds in his book, plus photos.)

Another kid took a special interest in math, complained bitterly when the staff wouldn't let him study it any more (because he'd reached the limits of his ability), and hatched an elaborate plot to escape from the institution, which involved reconstructing the entire NJ-NYC-Philly train system, including arrival and departure times and connections, based on seemingly casual conversations with maintenance workers, and taking sewing classes at the institution seriously enough to try to pass himself off as a tailor's apprentice. The staff only found his stash of notes until after he escaped. They never recaptured him. (Go kid!)

A reasonable person in the 1910's or 20's, like people now, might have concluded that some of these kids weren't actually retarded. Goddard acknowledged this, but highlighted these cases as proof of how difficult it was to diagnose moronism and how easily well-meaning people without proper training might mistake morons they met in real life for intelligent people who simply lacked education.

(OK. I'll try to shut up now.)
posted by nangar at 12:38 PM on November 20, 2011

In 1905, Einstein published his special theory of relativity. The very next year, German physicist Walter Kaufmann announced the results of experiments which represented, in his analysis, a clear refutation of the relativity principle. For about ten years, Kaufmann's experiments represented a weighty objection against relativity. By 1915, other experiments showed a preference for relativity, but in 1938 it was shown that the assumptions and setup of all the experiments were incorrect and that all their conclusions were erroneous. —Wikipedia, History of Special Relativity, Kaufmann–Bucherer–Neumann Experiments.
posted by exphysicist345 at 7:09 PM on November 20, 2011

Wikipedia has a category Hoaxes in Science and one for Experimental Errors and Frauds in Physics.
posted by exphysicist345 at 7:09 PM on November 20, 2011