How do we establish causation?
January 8, 2008 6:09 PM Subscribe
We've all heard correlation does not imply causation. What does? Are there any statistical methods for showing a causal relationship? If there are, when/by whom were they developed?
I'd like to improve my ability to evaluate published research. Additionally, I am looking into how social scientists in 1920-1940 thought about causality and correlation. Understanding what they did and didn't know back then would help me put their statistical methodologies in the appropriate context.
I'd like to improve my ability to evaluate published research. Additionally, I am looking into how social scientists in 1920-1940 thought about causality and correlation. Understanding what they did and didn't know back then would help me put their statistical methodologies in the appropriate context.
Causation cannot be demonstrated solely by statistical methods.
posted by Steven C. Den Beste at 6:19 PM on January 8, 2008 [3 favorites]
posted by Steven C. Den Beste at 6:19 PM on January 8, 2008 [3 favorites]
It's amazing how rarely total causation is proven in social research. Since correlation is often confused for causation, scientists or organizations with agendas will only go as far as correlation before releasing their findings to an impressionable public.
One helpful exercise is to try and poke holes in the suspected causal relationship. Try to think of other variables that could be underlying both phenomena. For example, does breastfeeding cause increased IQ, or are breastfeeding and increased IQ both byproducts of simply having more parental involvement?
Actually, that's better advice in general than it is for your project. However, it's something to think about when examining their work. It's a step that many scientists either forget or fiendishly omit.
posted by Doctor Suarez at 6:22 PM on January 8, 2008
One helpful exercise is to try and poke holes in the suspected causal relationship. Try to think of other variables that could be underlying both phenomena. For example, does breastfeeding cause increased IQ, or are breastfeeding and increased IQ both byproducts of simply having more parental involvement?
Actually, that's better advice in general than it is for your project. However, it's something to think about when examining their work. It's a step that many scientists either forget or fiendishly omit.
posted by Doctor Suarez at 6:22 PM on January 8, 2008
Going on what Doctor Suarez said, I often try to flip the conclusions and see if it works; in his example, I'd think, "What if they said high IQ causes breastfeeding. Does that make sense? Yes, because high IQ probably correlates with reasonably well informed and slightly liberal, which tend to be the women who breastfeed, because they've read that it's better for their babies, and they're probably more likely to be into 'natural' things like breastfeeding; plus, women with higher IQs would be more likely to be wealthy, hence more likely to be able to take time off from work to breastfeed their children, which would not be true of women in more minimum-wage jobs that don't provide for maternity leave."
I figure if you can turn an argument on its head and have it make just as much sense, then you've shown there's no proof of causation.
posted by occhiblu at 6:28 PM on January 8, 2008 [2 favorites]
I figure if you can turn an argument on its head and have it make just as much sense, then you've shown there's no proof of causation.
posted by occhiblu at 6:28 PM on January 8, 2008 [2 favorites]
If A and B are correlated, and assuming it's not a coincidence, then there are three possibilities:
1. A causes B
2. B causes A
3. C causes both A and B
Like occhiblu said, sometimes you can reason it out. You may recall from Freakonomics that there is a correlation showing that cities which legalized abortion in the 1970s had reduced crime rates in the 1990s. Could the reduced crime rates have caused the legalized abortions? No, because that would require time-travel. Is there a third factor causing both? Maybe, but you'd have to tell a pretty complicated story to explain that. So the most likely explanation is that legalized abortion caused reduced crime rates.
posted by Dec One at 6:44 PM on January 8, 2008
1. A causes B
2. B causes A
3. C causes both A and B
Like occhiblu said, sometimes you can reason it out. You may recall from Freakonomics that there is a correlation showing that cities which legalized abortion in the 1970s had reduced crime rates in the 1990s. Could the reduced crime rates have caused the legalized abortions? No, because that would require time-travel. Is there a third factor causing both? Maybe, but you'd have to tell a pretty complicated story to explain that. So the most likely explanation is that legalized abortion caused reduced crime rates.
posted by Dec One at 6:44 PM on January 8, 2008
I would be surprised if there were any, apart from narrow highly technical things like Granger-causation (which is not exactly causation).
Here is what I tell grad students.
You need to start with a theory, which is a story about causation. Your theory will have several empirical implications. Ideally, it will have many empirical implications. Ideally, it will have empirical implications that are seemingly unrelated* to each other.
You should also have a sense of competing theories -- causal stories that compete with yours in some way. One might be the established or consensus causal story.
Then you go out and look at as many empirical implications of your causal story and the competing causal stories. This is disappointing. Many of things you'd like to look at are more or less unobservable. Many other things are observable if you have a half-million dollars to spend on a survey or on experiments, and you don't and never will. But if you're luckier than any social-science researcher in history, you'll be able to come back and say
"Look at the evidence. My causal story predicts A, B, C, and Not-D. We actually observe A, B, C, and Not-D. The main competitor causal story predicts Not-A, Not-B, Not-C and D. So the world as we see it is highly consistent with my causal story, and inconsistent with the competing causal story. So you should believe mine until something better comes along."
That is, you demonstrate causality by showing a web of empirical relationships that are consistent with your causal story. This is obviously a very imperfect way to do it.
*In the normal sense, not in the "seemingly-unrelated regression" sense.
posted by ROU_Xenophobe at 6:48 PM on January 8, 2008 [4 favorites]
Here is what I tell grad students.
You need to start with a theory, which is a story about causation. Your theory will have several empirical implications. Ideally, it will have many empirical implications. Ideally, it will have empirical implications that are seemingly unrelated* to each other.
You should also have a sense of competing theories -- causal stories that compete with yours in some way. One might be the established or consensus causal story.
Then you go out and look at as many empirical implications of your causal story and the competing causal stories. This is disappointing. Many of things you'd like to look at are more or less unobservable. Many other things are observable if you have a half-million dollars to spend on a survey or on experiments, and you don't and never will. But if you're luckier than any social-science researcher in history, you'll be able to come back and say
"Look at the evidence. My causal story predicts A, B, C, and Not-D. We actually observe A, B, C, and Not-D. The main competitor causal story predicts Not-A, Not-B, Not-C and D. So the world as we see it is highly consistent with my causal story, and inconsistent with the competing causal story. So you should believe mine until something better comes along."
That is, you demonstrate causality by showing a web of empirical relationships that are consistent with your causal story. This is obviously a very imperfect way to do it.
*In the normal sense, not in the "seemingly-unrelated regression" sense.
posted by ROU_Xenophobe at 6:48 PM on January 8, 2008 [4 favorites]
Experimentation. If you want to prove A causes B, select some people (if you're researching people) as randomly as possible. Then randomly choose half of those people to do A. Let the other half do whatever they feel like. If the incidence of B in the "intervention group" (the folks doing A) is greater than in the "control group" (the folks you leave to their own devices) then A causes B (to some extent or another).
No matter how much reasoning you do, you can not imply causation from correlation, even time-delay correlation, though some people try. You can gain pretty good confirming evidence, but the fact is that unless you know that you are the reason why A happens when it does, you have no reason to believe that A causes B as opposed to something that covaries with A.
For example, say you want to know whether coughing leads to lung cancer. You can do a field study--ask people if they cough, then wait and see if they get lung cancer. You can even screen to make sure the coughing people don't have lung cancer yet, to make sure there's no reverse causation. And maybe you even have a causal theory of some sort. Point is, I bet you find a correlation. But if you were to do an experiment--randomly select some people and induce half of them to cough--you probably wouldn't find that the intervention group was more likely to develop lung cancer. The only way to rule out other reasons for a correlation is to make sure that you control the independent variable.
posted by goingonit at 6:57 PM on January 8, 2008
No matter how much reasoning you do, you can not imply causation from correlation, even time-delay correlation, though some people try. You can gain pretty good confirming evidence, but the fact is that unless you know that you are the reason why A happens when it does, you have no reason to believe that A causes B as opposed to something that covaries with A.
For example, say you want to know whether coughing leads to lung cancer. You can do a field study--ask people if they cough, then wait and see if they get lung cancer. You can even screen to make sure the coughing people don't have lung cancer yet, to make sure there's no reverse causation. And maybe you even have a causal theory of some sort. Point is, I bet you find a correlation. But if you were to do an experiment--randomly select some people and induce half of them to cough--you probably wouldn't find that the intervention group was more likely to develop lung cancer. The only way to rule out other reasons for a correlation is to make sure that you control the independent variable.
posted by goingonit at 6:57 PM on January 8, 2008
Correlation does not imply causation. However, it is on no other grounds that we infer it.
Imagine that you've gone to the store for eggs, and on your return you've proceeded to make yourself an omelet. Curiously, you break three eggs and each of them turns out to have two yolks. Hey! you call your roommate to come see this. These eggs are all double-yolkers!
But she breaks a few, and they have single yolks.
You break a few more, and they have double yolks.
She breaks a few more, and they have single yolks.
You go to the store for more eggs.
And the same thing continues to happen: Whenever you break an egg, it has two yolks; whenever anyone else breaks one, it has one.
At what point in this exercise do you conclude that your being the breaker of the egg is causing the egg to have two yolks? You would be irrational not to conclude that at some point, given that you discover no exceptions to the "rule," but at no point will you have observed anything other than correlation.
Thank you David Hume.
posted by bricoleur at 7:03 PM on January 8, 2008 [5 favorites]
Imagine that you've gone to the store for eggs, and on your return you've proceeded to make yourself an omelet. Curiously, you break three eggs and each of them turns out to have two yolks. Hey! you call your roommate to come see this. These eggs are all double-yolkers!
But she breaks a few, and they have single yolks.
You break a few more, and they have double yolks.
She breaks a few more, and they have single yolks.
You go to the store for more eggs.
And the same thing continues to happen: Whenever you break an egg, it has two yolks; whenever anyone else breaks one, it has one.
At what point in this exercise do you conclude that your being the breaker of the egg is causing the egg to have two yolks? You would be irrational not to conclude that at some point, given that you discover no exceptions to the "rule," but at no point will you have observed anything other than correlation.
Thank you David Hume.
posted by bricoleur at 7:03 PM on January 8, 2008 [5 favorites]
The Hill criteria.
These criteria were proposed in the early days of epidemiology to help make sense of the data about emphysema, lung cancer, and smoking. While they've been endlessly debated and criticized - they are not perfect, nor idiot-proof - they're still taught to every epidemiologist in training, and they form the core of the modern understanding of how to infer causation from observed correlations in data.
posted by ikkyu2 at 7:05 PM on January 8, 2008
These criteria were proposed in the early days of epidemiology to help make sense of the data about emphysema, lung cancer, and smoking. While they've been endlessly debated and criticized - they are not perfect, nor idiot-proof - they're still taught to every epidemiologist in training, and they form the core of the modern understanding of how to infer causation from observed correlations in data.
posted by ikkyu2 at 7:05 PM on January 8, 2008
Here is a pretty good pdf on the matter. Go a few pages in and you'll see discussion of the Hill Criteria. There are other sets of causation rules-of-thumb in different fields, like Koch's postulates in Microbiology. As most would point out, the best you can do is an experiment. If you can independently play with A and B, the idea that A causes B can be tested rigorously. However, the expanded Hill criteria have some good ideas which can make less plausable that B causes A.
For example, Temporality: if A happens before B in time, then it seems pretty damn unlikely that B causes A. There could be joint cause C for A and B. A good dose-response relationship is nice to see.
posted by a robot made out of meat at 7:06 PM on January 8, 2008
For example, Temporality: if A happens before B in time, then it seems pretty damn unlikely that B causes A. There could be joint cause C for A and B. A good dose-response relationship is nice to see.
posted by a robot made out of meat at 7:06 PM on January 8, 2008
Longitudinal or time series designs can be pretty strong evidence of causation. When the independent variable is applied then removed and applied again and the presence of the dependent variable appears, diminishes and appears again (i.e., seems to mirror the IV), that can be compelling evidence of causation. Cook and Campbell's book Quasi Experimentation has a very good treatment of the subject of causation.
posted by jasper411 at 7:13 PM on January 8, 2008
posted by jasper411 at 7:13 PM on January 8, 2008
Time series analysis is about the best you are going to do without randomized experimentation.
posted by jtfowl0 at 7:15 PM on January 8, 2008
posted by jtfowl0 at 7:15 PM on January 8, 2008
If A and B are correlated, and assuming it's not a coincidence, then there are three possibilities:
1. A causes B
2. B causes A
3. C causes both A and B
There are many many more possibilities than that (in fact, I suspect it's a small percentage of correlations that fall into one of those three categories). They don't need to have a causal relationship nor a common cause e.g. If people who do A are likely to have condition B then maybe people who do A are also likely to do something else which causes B -- this is far more likely than any of the above three options (as there are maybe hundreds of things that people who do A are also likely to do).
posted by winston at 7:18 PM on January 8, 2008
1. A causes B
2. B causes A
3. C causes both A and B
There are many many more possibilities than that (in fact, I suspect it's a small percentage of correlations that fall into one of those three categories). They don't need to have a causal relationship nor a common cause e.g. If people who do A are likely to have condition B then maybe people who do A are also likely to do something else which causes B -- this is far more likely than any of the above three options (as there are maybe hundreds of things that people who do A are also likely to do).
posted by winston at 7:18 PM on January 8, 2008
David Hume is a philosopher who is famous for saying that we don't ever have evidence of a "necessary connection" between two distinct events - so if that's what causation is, one event necessitating another, then Hume thought we never had evidence that causation took place. (This is, of course, an oversimple summary but enough to let you decide if this is the issue you're interested in, or if you're looking more for what scientists regard as sufficient evidence to accept a causal claim.)
posted by LobsterMitten at 7:35 PM on January 8, 2008
posted by LobsterMitten at 7:35 PM on January 8, 2008
then maybe people who do A are also likely to do something else which causes B
Isn't that causation?
posted by jpdoane at 7:35 PM on January 8, 2008
Isn't that causation?
posted by jpdoane at 7:35 PM on January 8, 2008
Although, it really is no different than Dec One's option C
posted by jpdoane at 7:38 PM on January 8, 2008
posted by jpdoane at 7:38 PM on January 8, 2008
Also, Steven Jay Gould's The Mismeasure of Man has quite a bit of interesting stuff on statistical methods used by psychologists trying to measure intelligence in the time period you're talking about. Might have useful context or pointers for your search.
posted by LobsterMitten at 7:39 PM on January 8, 2008
posted by LobsterMitten at 7:39 PM on January 8, 2008
If A and B are correlated, and assuming it's not a coincidence. . .
That's a big assumption to make. Humans are very susceptible to observational bias
posted by chrisamiller at 7:55 PM on January 8, 2008
That's a big assumption to make. Humans are very susceptible to observational bias
posted by chrisamiller at 7:55 PM on January 8, 2008
respected social science research organizations use a method called "random assignment" to test the effects of planned interventions. an evaluator is testing the effects of a program (say, tuition assistance to college students, for example). they market the program, and enroll eligible students, only half of whom the program can serve. students enrolled into the study are then randomly assigned to the program group or the control group.
baseline data are collected prior to random assignment. other data are collected as the program unfolds - surveys, transcripts, enrollment data, et cetera. whatever is relevant to the study.
if the eligibility criteria are sound, and if the assignment was indeed random, then observed differences in certain outcomes between the program group and the control group can be reasonably assumed to be caused by the program/intervention. additional statistical tests are run to determine "statistical significance" - the statistical probability that the observed difference in outcomes was due to chance.
am i being clear? let me know if you'd like me to provide a more concrete example.
posted by entropone at 8:00 PM on January 8, 2008
baseline data are collected prior to random assignment. other data are collected as the program unfolds - surveys, transcripts, enrollment data, et cetera. whatever is relevant to the study.
if the eligibility criteria are sound, and if the assignment was indeed random, then observed differences in certain outcomes between the program group and the control group can be reasonably assumed to be caused by the program/intervention. additional statistical tests are run to determine "statistical significance" - the statistical probability that the observed difference in outcomes was due to chance.
am i being clear? let me know if you'd like me to provide a more concrete example.
posted by entropone at 8:00 PM on January 8, 2008
Of course the problem with that, entropone, is the placebo effect. A study like that isn't double blind (or even just plain blind, depending on study design).
In medicine, you can make a true double-blind study--- what do they do for social science variables where you can't for example trick the students and observers into thinking they were in the control group vs. the program group?
In other words, how do you ensure that the effect isn't just because the students thought you were doing *something*--- whatever that something might be?
posted by nat at 8:35 PM on January 8, 2008
In medicine, you can make a true double-blind study--- what do they do for social science variables where you can't for example trick the students and observers into thinking they were in the control group vs. the program group?
In other words, how do you ensure that the effect isn't just because the students thought you were doing *something*--- whatever that something might be?
posted by nat at 8:35 PM on January 8, 2008
Regarding Logicpunk's post: Structural equation modeling most certainly cannot prove causation. The conclusions you draw are only as good as the assumptions you put into the structural model, and those assumptions can fall prey to all of the same fallacies as normal old correlation. It is a more complex and often more powerful tool, but it has no more magical power to uncover causative relationships than garden variety correlation.
I agree with experimentation with random assignment as the best method, with time-series and carefully designed longitudinal studies as the next best thing. I second Quasi Experimentation as informative read.
(I have taught a class on SEM, and have published papers using it.)
posted by Chanther at 8:58 PM on January 8, 2008
I agree with experimentation with random assignment as the best method, with time-series and carefully designed longitudinal studies as the next best thing. I second Quasi Experimentation as informative read.
(I have taught a class on SEM, and have published papers using it.)
posted by Chanther at 8:58 PM on January 8, 2008
Response by poster: Thank you everyone for all the thoughtful answers. You've been a huge help. (And please keep 'em coming if you've got more.)
posted by whimwit at 9:29 PM on January 8, 2008
posted by whimwit at 9:29 PM on January 8, 2008
If the incidence of B in the "intervention group" (the folks doing A) is greater than in the "control group" (the folks you leave to their own devices) then A causes B (to some extent or another).
Not so. You might have just drawn unlucky samples, or split the people in an unlucky way, so that the people in the treatment group happen to have more of whatever the true causative factor is, if there is one.
Experimentation doesn't get you causality. All it gets you is, hopefully, good random samples and independent variables whose values are chosen by the researcher instead of by nature. After that, you're still just correlating data.
posted by ROU_Xenophobe at 9:33 PM on January 8, 2008
Not so. You might have just drawn unlucky samples, or split the people in an unlucky way, so that the people in the treatment group happen to have more of whatever the true causative factor is, if there is one.
Experimentation doesn't get you causality. All it gets you is, hopefully, good random samples and independent variables whose values are chosen by the researcher instead of by nature. After that, you're still just correlating data.
posted by ROU_Xenophobe at 9:33 PM on January 8, 2008
I'm far from a statistics guru, but this does ring a bell from my science days..
In my developmental biology classes, your question and answer was a huge crux of the class. Actually, for our class, it was almost the entire basis of determining "why stuff happened?" Correlation can be useful in determining causation- kind of a starting point-, but is the weakest method. The strongest way to determine causation is "gain of function (GoF)" and "loss of function (LoF)."
For instance, the classical developmental experiment is to observe active genes and enzymes in a certain limb which are absent in another- this would be simply correlation. Activite this gene in a different limb where it is not active, and if a new kind of limb grows, you have gain of function which is pretty powerful in showing that an active gene causing certain development. The opposite is true- knock out the genes associated with a certain limb and if the limb stops growing, you have loss of function.
What we were always looking for is both GoF and LoF in developmental studied. One observation on its own was decent evidence- get both going and you have a pretty strong case (of course, with all things science, that's not to say observing both LoF and GoF means your observations are 100% correct as there are always other players).
I can't think of specific examples off the top of my head, but we did observe a fair number of experiments where correlation alone did not represent the underlying development of life. It was through the LoF & GoF experiments where correlation was shown to be false on numerous occasions.
I think the same could be true of the social science. "Why does poverty happen and what can we do to solve it?" We can observe what's happening (correlation)- the cycle of poverty, bad luck, natural disasters, etc. However, unlike science where you can isolate all other variables to say with confidence, it's much harder to do that in "real life."
posted by jmd82 at 6:12 AM on January 9, 2008
In my developmental biology classes, your question and answer was a huge crux of the class. Actually, for our class, it was almost the entire basis of determining "why stuff happened?" Correlation can be useful in determining causation- kind of a starting point-, but is the weakest method. The strongest way to determine causation is "gain of function (GoF)" and "loss of function (LoF)."
For instance, the classical developmental experiment is to observe active genes and enzymes in a certain limb which are absent in another- this would be simply correlation. Activite this gene in a different limb where it is not active, and if a new kind of limb grows, you have gain of function which is pretty powerful in showing that an active gene causing certain development. The opposite is true- knock out the genes associated with a certain limb and if the limb stops growing, you have loss of function.
What we were always looking for is both GoF and LoF in developmental studied. One observation on its own was decent evidence- get both going and you have a pretty strong case (of course, with all things science, that's not to say observing both LoF and GoF means your observations are 100% correct as there are always other players).
I can't think of specific examples off the top of my head, but we did observe a fair number of experiments where correlation alone did not represent the underlying development of life. It was through the LoF & GoF experiments where correlation was shown to be false on numerous occasions.
I think the same could be true of the social science. "Why does poverty happen and what can we do to solve it?" We can observe what's happening (correlation)- the cycle of poverty, bad luck, natural disasters, etc. However, unlike science where you can isolate all other variables to say with confidence, it's much harder to do that in "real life."
posted by jmd82 at 6:12 AM on January 9, 2008
Occhiblu's answer is especially interesting because the situation she addresses, where A causes B and B causes A, amounts to a positive feedback loop in any circumstance where there are cycles and the output of one cycle is the input of the next, and this is the circumstance of all life on Earth in all of its particulars, including the example of breastfeeding that she uses.
Positive feedback of even a very weak sort can result in very high levels of correlation, as is demonstrated by the establishment throughout a population of characteristics conferring mere percentage points of advantage in simulations of evolution, and so it would seem that anytime you see a very high level of correlation when observing living things (or anything else to which iterative cycles may reasonably be imputed), that is a cardinal sign not just of causation, but causation in both directions.
posted by jamjam at 8:58 AM on January 9, 2008
Positive feedback of even a very weak sort can result in very high levels of correlation, as is demonstrated by the establishment throughout a population of characteristics conferring mere percentage points of advantage in simulations of evolution, and so it would seem that anytime you see a very high level of correlation when observing living things (or anything else to which iterative cycles may reasonably be imputed), that is a cardinal sign not just of causation, but causation in both directions.
posted by jamjam at 8:58 AM on January 9, 2008
winston said: If people who do A are likely to have condition B then maybe people who do A are also likely to do something else which causes B
Now you've introduced another variable. What causes this "something else" in your example? Does A cause it, or does B cause it, or does it cause A or B? Or is it caused by C? Or is it C?
jpdoane said: it really is no different than Dec One's option C
Right, if you assume that "something else" is C and that it causes A and B.
posted by Dec One at 9:46 AM on January 9, 2008
Now you've introduced another variable. What causes this "something else" in your example? Does A cause it, or does B cause it, or does it cause A or B? Or is it caused by C? Or is it C?
jpdoane said: it really is no different than Dec One's option C
Right, if you assume that "something else" is C and that it causes A and B.
posted by Dec One at 9:46 AM on January 9, 2008
Dec One, jpdoane -- You can observe covariance without necessarily making any attribution of causality. Take, for instance, this informative graph that demonstrates an inverse relationship between global average temperature and number of pirates. Is there a causal relationship there?
There are many "accidental" relationships with no causal connection. Covariance, in and of itself, cannot license any causal conclusions, even the very basic "there is a causal connection between these phenomena" (except insofar as the big bang implies all phenomena are causally linked).
posted by goingonit at 10:22 AM on January 9, 2008
There are many "accidental" relationships with no causal connection. Covariance, in and of itself, cannot license any causal conclusions, even the very basic "there is a causal connection between these phenomena" (except insofar as the big bang implies all phenomena are causally linked).
posted by goingonit at 10:22 AM on January 9, 2008
goingonit: That's why I prefaced my original post with "assuming it's not a coincidence". Let me rephrase. If two variables are correlated, there are four possibilities:
1. A causes B
2. B causes A
3. C causes A and B
4. It's a coincidence
This is what I remember learning in college, and I'm not convinced by this thread that it's wrong. But I'm still open to being persuaded.
posted by Dec One at 2:35 PM on January 9, 2008
1. A causes B
2. B causes A
3. C causes A and B
4. It's a coincidence
This is what I remember learning in college, and I'm not convinced by this thread that it's wrong. But I'm still open to being persuaded.
posted by Dec One at 2:35 PM on January 9, 2008
You can observe covariance without necessarily making any attribution of causality.
Not with a statistically significant sample size (which is really just saying there actually is correlation). If there is correlation, then there is a cause for the correlation, even if the two things that are correlated are not directly related. It still boils down to option 3: C causes both A and B
For the pirate example, we can label C something like "modernization". Modernization causes both fewer pirates and higher temperature.
posted by jpdoane at 9:23 PM on January 9, 2008
Not with a statistically significant sample size (which is really just saying there actually is correlation). If there is correlation, then there is a cause for the correlation, even if the two things that are correlated are not directly related. It still boils down to option 3: C causes both A and B
For the pirate example, we can label C something like "modernization". Modernization causes both fewer pirates and higher temperature.
posted by jpdoane at 9:23 PM on January 9, 2008
Judea Pearl, Clark Glymour and others have done a lot of work recently on this question. Here's a brief explanation. It's related to structural equation modeling which others have already mentioned.
posted by euphotic at 1:00 AM on January 17, 2008
posted by euphotic at 1:00 AM on January 17, 2008
As a social scientist (6 credits shy of Anthropology/sociology in a combined BA), I have to suggest something that maybe you already know, but other readers here in future might find useful.
The best way to evaluate social research is to read the entire peer reviewed journal article. (This is especially true if you are reading materials from the 20's through 40's, it will help you keep some of your current "knowledge" about a topic at bay.) Start at the abstract, where they'll be kind enough to tell you pretty much up front what they discovered. This isn't science fiction, the authors have nothing to gain from keeping you in suspense. But, the revelation hasn't spoiled the story. You have to keep going. After you've read the whole abstract, skim the lit review and methodology. Look carefully at all the tables, charts and graphs. Read the Conclusion. Since you had so much fun, read it all again, no skimming on round 2.
What you'll find is that this article uses some methods that are similar to and some that are different from previous researchers. Perhaps they are trying to replicate an earlier study. They may have decided that previous researchers used samples that were biased or used inadequate definitions.
It's very important to see where a study fits in with the body of work that's already out there. Check out the references pages. See who is considered to have produced work seminal in the area and whether this author agrees or disagrees with those findings. What further avenues of research have been suggested?
I haven't read any of the above mentioned studies on breastfeeding and IQ, but in the work I'm checking out, researchers are quite happy to admit what they don't know, what puzzles them, and where their methodology is flawed. Here's the thing. Every methodology in social science must be assumed to be flawed. There is no perfect way to ask questions about what people do or how they feel, especially about sensitive topics like discrimination, sexuality, and even boobies. What happens is that 30 pages of peer reviewed research gets "distilled" into two paragraphs for the New York Times. Gregory M. Herek doesn't have to worry about selling copies of the journals that he's in (although he does publish books about homosexuality in the military, AIDS stigma, and other topics). The New York Times, however is very interested in getting people to talk and getting people to buy the paper. So who is more likely to say, "This is the way it is, we've solved the problem!?" I think you'll guess correctly.
posted by bilabial at 7:00 AM on January 19, 2008
The best way to evaluate social research is to read the entire peer reviewed journal article. (This is especially true if you are reading materials from the 20's through 40's, it will help you keep some of your current "knowledge" about a topic at bay.) Start at the abstract, where they'll be kind enough to tell you pretty much up front what they discovered. This isn't science fiction, the authors have nothing to gain from keeping you in suspense. But, the revelation hasn't spoiled the story. You have to keep going. After you've read the whole abstract, skim the lit review and methodology. Look carefully at all the tables, charts and graphs. Read the Conclusion. Since you had so much fun, read it all again, no skimming on round 2.
What you'll find is that this article uses some methods that are similar to and some that are different from previous researchers. Perhaps they are trying to replicate an earlier study. They may have decided that previous researchers used samples that were biased or used inadequate definitions.
It's very important to see where a study fits in with the body of work that's already out there. Check out the references pages. See who is considered to have produced work seminal in the area and whether this author agrees or disagrees with those findings. What further avenues of research have been suggested?
I haven't read any of the above mentioned studies on breastfeeding and IQ, but in the work I'm checking out, researchers are quite happy to admit what they don't know, what puzzles them, and where their methodology is flawed. Here's the thing. Every methodology in social science must be assumed to be flawed. There is no perfect way to ask questions about what people do or how they feel, especially about sensitive topics like discrimination, sexuality, and even boobies. What happens is that 30 pages of peer reviewed research gets "distilled" into two paragraphs for the New York Times. Gregory M. Herek doesn't have to worry about selling copies of the journals that he's in (although he does publish books about homosexuality in the military, AIDS stigma, and other topics). The New York Times, however is very interested in getting people to talk and getting people to buy the paper. So who is more likely to say, "This is the way it is, we've solved the problem!?" I think you'll guess correctly.
posted by bilabial at 7:00 AM on January 19, 2008
This thread is closed to new comments.
posted by jourman2 at 6:13 PM on January 8, 2008