Looking for the TLDR of the replication crisis in psychology
May 17, 2024 8:58 AM   Subscribe

I'm fascinated by the replication crisis in psychology. What I want is a two column summary: Experiment and did it replicate? Maybe with some commentary. I cannot find this. I can find links to hundreds of individual studies. That's too much reading for me. I can find summaries like "60% didn't replicate." That's too little info for me. I want to know which ones replicated and which didn't. Help?
posted by papergirl to Science & Nature (4 answers total) 10 users marked this as a favorite
 
Best answer: Looks like it's in this file. Hide columns as needed. (Sorry, on mobile.)
posted by avocet at 9:31 AM on May 17, 2024 [4 favorites]


Response by poster: Thank you!!!
posted by papergirl at 9:53 AM on May 17, 2024


Best answer: Avocet has identified the closest thing you're likely to find! A similar larger project (disclaimer, am one of several hundred authors) is working to generate a table for a random sampling of papers across the social sciences. Preprint describing the method here.

A skim of that paper may help give some intuitions why this kind of table can be hard to find - any such table (including the one above) will be heavily disputed line by line, so systematic replicators will tend to be extremely specific in their claims and methods. A grab bag of reasons include -

* The original paper uses (hopefully) a random sample, but the replication does too. Thus, either the original OR the replication could be either a false positive or false negative.
* Who decides what counts as successful replication? Most papers use more than one statistical test, and most statistical tests have some choices the analyzer needs to make. Most statistical tests require some interpretation to get to the claim the author actually wants to make.
* How did you pick what to replicate? We were about as systematic as possible (a random sample from a set of years in a set of journals), but even so were limited in what we *wanted* to try to replicate vs what we *could* produce an answer for either positive or negative.
posted by itsatextfile at 9:59 AM on May 17, 2024 [6 favorites]


Best answer: You may be interested in the work of Clearer Thinking, where they are doing replications.
posted by chiefthe at 10:02 AM on May 17, 2024


« Older Crtl+Z in life after making it weird   |   Help me build a list of neologisms-by-truncation!... Newer »

You are not logged in, either login or create an account to post comments