How to know if two proportions are statistically significant?
March 16, 2007 7:43 AM Subscribe
How do I determine whether the difference between two or more proportions is significant?
I took introductory statistics last year, but I only barely passed and I can't find my notes anywhere!
Anyway, I'm doing a project that involves studying the impact of a policy on four different subsets of the population (by race: whites, blacks, Hispanics, and Asians). It does seem to affect these different groups differently, but I'm trying to remember how to test if this difference is significant. I've googled it, but the results I've found look vaguely familiar, yet incomprehensible.
Does anyone know of a simple explanation online, or can explain it to me simply? Specifically, I have information on whether or not a policy affects these groups, so it's expressed as proportions.
I'm looking for help at school, too, but I might not get it till after I need to submit my draft!
Much appreciated!
I took introductory statistics last year, but I only barely passed and I can't find my notes anywhere!
Anyway, I'm doing a project that involves studying the impact of a policy on four different subsets of the population (by race: whites, blacks, Hispanics, and Asians). It does seem to affect these different groups differently, but I'm trying to remember how to test if this difference is significant. I've googled it, but the results I've found look vaguely familiar, yet incomprehensible.
Does anyone know of a simple explanation online, or can explain it to me simply? Specifically, I have information on whether or not a policy affects these groups, so it's expressed as proportions.
I'm looking for help at school, too, but I might not get it till after I need to submit my draft!
Much appreciated!
Convert proportions to percentages? Just divide the proportion and multiply by 100. I'm not quite sure if that is what you are seeking...
posted by JJ86 at 8:58 AM on March 16, 2007
posted by JJ86 at 8:58 AM on March 16, 2007
Sounds like it could be a Chi-square problem, but you would be better off getting the full numbers rather than the proportions.
The Chi-square is used to test whether observed numbers in a group are significantly different from expected numbers in a group. Your observed numbers would be the number of people of each category participating in the program. The expected numbers should be derived from the best available local census of the community.
posted by KirkJobSluder at 9:07 AM on March 16, 2007
The Chi-square is used to test whether observed numbers in a group are significantly different from expected numbers in a group. Your observed numbers would be the number of people of each category participating in the program. The expected numbers should be derived from the best available local census of the community.
posted by KirkJobSluder at 9:07 AM on March 16, 2007
I'm not quite sure if that is what you are seeking...
That is always a fantastic reason not to answer.
The relevant test statistic is a z-statistic; that is, a standard normal. You do need to know the two sample sizes that were taken. Say n1 and n2 were the sample size (e.g., 100 and 150) and p1 and p2 were the observed proportions (e.g. 1/2 and 2/3).
First, compute φ=(n1p1+n2p2)/(n1+n2). (This is the so-called "pooled-sample proportion")
Then your test statistic is
zobs = (p1-p2) / Sqrt (φ(1-φ)(1/n1 + 1/n2)).
If you remember how to do hypothesis testing at all, that should tell you what you need: if the zobs is too far away from zero on either side, you may consider the two proporitions significantly different.
posted by Wolfdog at 9:42 AM on March 16, 2007
That is always a fantastic reason not to answer.
The relevant test statistic is a z-statistic; that is, a standard normal. You do need to know the two sample sizes that were taken. Say n1 and n2 were the sample size (e.g., 100 and 150) and p1 and p2 were the observed proportions (e.g. 1/2 and 2/3).
First, compute φ=(n1p1+n2p2)/(n1+n2). (This is the so-called "pooled-sample proportion")
Then your test statistic is
zobs = (p1-p2) / Sqrt (φ(1-φ)(1/n1 + 1/n2)).
If you remember how to do hypothesis testing at all, that should tell you what you need: if the zobs is too far away from zero on either side, you may consider the two proporitions significantly different.
posted by Wolfdog at 9:42 AM on March 16, 2007
Sorry... if you want to do more than two, there's really more than one way to approach it. You can simply test all possible pairs using the previous method, and in a way this is the most revealing. That will tell you exactly which pairs of populations differ significantly from each other.
You can approach it in an "all-at-one-go" manner by using a chi-square test. If you have r different populations to test then it's a chi-square with (r-1) degrees of freedom; I can provide details. However, be clear about what you'd be testing: the null hypothesis would be that ALL r of the populations have the same proportion, and if you reject that on the basis of a large chi-square result, then the conclusion would only be that "not all the proportions are the same". You'd also have to use an pooled estimate of what the alleged common proportion is, which complicates things a bit. I don't think that would be as helpful in your case as comparing them pairwise.
posted by Wolfdog at 10:49 AM on March 16, 2007
You can approach it in an "all-at-one-go" manner by using a chi-square test. If you have r different populations to test then it's a chi-square with (r-1) degrees of freedom; I can provide details. However, be clear about what you'd be testing: the null hypothesis would be that ALL r of the populations have the same proportion, and if you reject that on the basis of a large chi-square result, then the conclusion would only be that "not all the proportions are the same". You'd also have to use an pooled estimate of what the alleged common proportion is, which complicates things a bit. I don't think that would be as helpful in your case as comparing them pairwise.
posted by Wolfdog at 10:49 AM on March 16, 2007
Wolfdog, how is multiple test correction performed here?
posted by NucleophilicAttack at 11:31 AM on March 16, 2007
posted by NucleophilicAttack at 11:31 AM on March 16, 2007
Response by poster: Thanks, everyone.
Wolfdog, yes, that is exactly what I need to do. Thanks so much! Someone also just showed me how to import my spreadsheets into STATA, so I think I'll do the tests that way...
posted by lunasol at 1:19 PM on March 16, 2007
Wolfdog, yes, that is exactly what I need to do. Thanks so much! Someone also just showed me how to import my spreadsheets into STATA, so I think I'll do the tests that way...
posted by lunasol at 1:19 PM on March 16, 2007
This thread is closed to new comments.
posted by DevilsAdvocate at 8:34 AM on March 16, 2007