Stats test to compare 2 groups
October 23, 2008 4:40 PM Subscribe
I’m trying to statistically compare the body dimensions (height, weight, waist circumference, etc) of two groups of 200 people. I need help determining the correct statistical tests to use.
I’m trying to statistically compare the body dimensions (height, weight, waist circumference, etc) of two groups of 200 people. While some of the dimensions are normally distributed and have equal variances, other dimensions are either non-normal or have unequal variances (violating two of the assumptions of the t-test). One report I’m reading that compared the dimensions of two groups used a non-parametric test for all of the dimensions if at least one of the dimensions was non-normal or had heteroscedastic variance. Another paper only used the non-parametric tests on the dimensions that were non-normal (using a t-test on the dimensions that were normally distributed). Is it a generally accepted practice in statistics to use a non-parametric test for all the dimensions if at least one of the dimensions violates the assumptions of the t-test?
I’m trying to statistically compare the body dimensions (height, weight, waist circumference, etc) of two groups of 200 people. While some of the dimensions are normally distributed and have equal variances, other dimensions are either non-normal or have unequal variances (violating two of the assumptions of the t-test). One report I’m reading that compared the dimensions of two groups used a non-parametric test for all of the dimensions if at least one of the dimensions was non-normal or had heteroscedastic variance. Another paper only used the non-parametric tests on the dimensions that were non-normal (using a t-test on the dimensions that were normally distributed). Is it a generally accepted practice in statistics to use a non-parametric test for all the dimensions if at least one of the dimensions violates the assumptions of the t-test?
BTW, you didn't say anything about the sample design. Are these two groups chosen by simple random samples from two different populations?
posted by mikeand1 at 6:21 PM on October 23, 2008
posted by mikeand1 at 6:21 PM on October 23, 2008
Response by poster: Thanks for your response. Yes, I'm trying to compare the means.
"For an N of 200, that's going to be normally distributed pretty much regardless of the distribution of the dimension."
I don' follow what you mean, can you please clarify your point. Some dimensions/distributions are definitely not normal...waist circumference is skewed to the right.
Sample design - yes, samples are random.
posted by hithere at 6:34 PM on October 23, 2008
"For an N of 200, that's going to be normally distributed pretty much regardless of the distribution of the dimension."
I don' follow what you mean, can you please clarify your point. Some dimensions/distributions are definitely not normal...waist circumference is skewed to the right.
Sample design - yes, samples are random.
posted by hithere at 6:34 PM on October 23, 2008
Check out this demonstration to see how the sampling distribution of the mean is pretty close to normal even when the underlying variable has a gnarly distribution.
But when in doubt, it's a good idea to follow the analytical conventions of your field of study. There might be good reason to use a non-parametric test for one or more of your variables (e.g., if the variable is essentially dichotomous in the population). That said, I wouldn't run a non-parametric test on a variable that has a nice normal distribution just because one of my other variables was categorical.
posted by nixxon at 6:47 PM on October 23, 2008
But when in doubt, it's a good idea to follow the analytical conventions of your field of study. There might be good reason to use a non-parametric test for one or more of your variables (e.g., if the variable is essentially dichotomous in the population). That said, I wouldn't run a non-parametric test on a variable that has a nice normal distribution just because one of my other variables was categorical.
posted by nixxon at 6:47 PM on October 23, 2008
The t-test is robust against violations of these two assumptions, actually, esp. given your sample size and the random sampling.
posted by messylissa at 6:48 PM on October 23, 2008
posted by messylissa at 6:48 PM on October 23, 2008
"I don' follow what you mean, can you please clarify your point. Some dimensions/distributions are definitely not normal...waist circumference is skewed to the right."
OK, let's back up a bit. Usually, when something like a t-test comes into play, you have a situation like this:
There are two populations: A and B. You'd like to know if the mean value of a variable (say, waist circumference) in the two populations is different.
So, you take a simple random sample from population A and measure the mean waist circumference for that sample. Then, you take a simple random sample from population B, and measure the mean waist circumference for that sample.
You then calculate the t-test for the two sample means, and that tells you if there is a statistically significant difference in them -- that is, the test tells you if the difference between the sample means is likely to be due to random sampling error even assuming the mean waist circumference is the same for both populations.
If you have a sample size of N=200 for each sample, it doesn't really matter if a histogram of the waist circumferences skews to the right, say. That's because the distribution of the sample mean will still be approximately normal.
Are these two groups of people drawn by simple random samples from two different populations? Is there a sampling scheme at all? Or are you just comparing two groups of people? If it's the latter, you don't need to do inferential statistics.
posted by mikeand1 at 6:57 PM on October 23, 2008
OK, let's back up a bit. Usually, when something like a t-test comes into play, you have a situation like this:
There are two populations: A and B. You'd like to know if the mean value of a variable (say, waist circumference) in the two populations is different.
So, you take a simple random sample from population A and measure the mean waist circumference for that sample. Then, you take a simple random sample from population B, and measure the mean waist circumference for that sample.
You then calculate the t-test for the two sample means, and that tells you if there is a statistically significant difference in them -- that is, the test tells you if the difference between the sample means is likely to be due to random sampling error even assuming the mean waist circumference is the same for both populations.
If you have a sample size of N=200 for each sample, it doesn't really matter if a histogram of the waist circumferences skews to the right, say. That's because the distribution of the sample mean will still be approximately normal.
Are these two groups of people drawn by simple random samples from two different populations? Is there a sampling scheme at all? Or are you just comparing two groups of people? If it's the latter, you don't need to do inferential statistics.
posted by mikeand1 at 6:57 PM on October 23, 2008
"There might be good reason to use a non-parametric test for one or more of your variables (e.g., if the variable is essentially dichotomous in the population). That said, I wouldn't run a non-parametric test on a variable that has a nice normal distribution just because one of my other variables was categorical."
No. If you are comparing two means, and you have a sample N of 200, it generally doesn't matter if the underlying variable is dichotomous, categorical, or whatever.
For a sample N of 200, you'd have to have a really really really really skewed underlying variable for it to matter to the sample mean.
posted by mikeand1 at 7:02 PM on October 23, 2008
No. If you are comparing two means, and you have a sample N of 200, it generally doesn't matter if the underlying variable is dichotomous, categorical, or whatever.
For a sample N of 200, you'd have to have a really really really really skewed underlying variable for it to matter to the sample mean.
posted by mikeand1 at 7:02 PM on October 23, 2008
"No. If you are comparing two means, and you have a sample N of 200, it generally doesn't matter if the underlying variable is dichotomous, categorical, or whatever."
That's not true -- if the underlying variable is really dichotomous you need to use a nonparametric test. For example, think about the temperature of a freezer when it's either on or off. The specific measured temperature might vary a little, but it's basically either cold (on) or warm (off). In that case the mean is a lousy measure of central tendency, and inferential statistics about the mean don't really make sense either. It's conceptually a non-parametric situation, even though the distribution might not look perfectly dichotomous because of the variability around the "on" and "off" temperatures. I agree, though, that it's very unlikely that the OPs data has a distribution like that, but it's worth thinking about especially if it's common practice in the OPs research area to treat one or more of these variables as categorical.
posted by nixxon at 7:20 PM on October 23, 2008
That's not true -- if the underlying variable is really dichotomous you need to use a nonparametric test. For example, think about the temperature of a freezer when it's either on or off. The specific measured temperature might vary a little, but it's basically either cold (on) or warm (off). In that case the mean is a lousy measure of central tendency, and inferential statistics about the mean don't really make sense either. It's conceptually a non-parametric situation, even though the distribution might not look perfectly dichotomous because of the variability around the "on" and "off" temperatures. I agree, though, that it's very unlikely that the OPs data has a distribution like that, but it's worth thinking about especially if it's common practice in the OPs research area to treat one or more of these variables as categorical.
posted by nixxon at 7:20 PM on October 23, 2008
Nthing everyone else. The shape of your sample and population distributions are not relevant at all. The only distribution that needs to be normal is the sampling distribution. Sampling distributions always approach normal as your N goes up, so hooray. Use the standard t-test.
posted by shadow vector at 7:27 PM on October 23, 2008
posted by shadow vector at 7:27 PM on October 23, 2008
"That's not true -- if the underlying variable is really dichotomous you need to use a nonparametric test."
Please explain, then, when the Central Limit Theorem is so easily demonstrated with coin flips (a classic dichotomous variable).
Just look at the language of the CLT:
"Let X1, X2, X3, ... Xn be a sequence of n independent and identically distributed random variables having each finite values of expectation µ and variance σ2 > 0. The central limit theorem states that as the sample size n increases, the distribution of the sample average of these random variables approaches the normal distribution with a mean µ and variance σ2 / n irrespective of the shape of the original distribution."
How can you possibly interpret "irrespective of the shape of the original distribution" to mean "except for dichotomous distributions"?
posted by mikeand1 at 7:56 PM on October 23, 2008
Please explain, then, when the Central Limit Theorem is so easily demonstrated with coin flips (a classic dichotomous variable).
Just look at the language of the CLT:
"Let X1, X2, X3, ... Xn be a sequence of n independent and identically distributed random variables having each finite values of expectation µ and variance σ2 > 0. The central limit theorem states that as the sample size n increases, the distribution of the sample average of these random variables approaches the normal distribution with a mean µ and variance σ2 / n irrespective of the shape of the original distribution."
How can you possibly interpret "irrespective of the shape of the original distribution" to mean "except for dichotomous distributions"?
posted by mikeand1 at 7:56 PM on October 23, 2008
"The specific measured temperature might vary a little, but it's basically either cold (on) or warm (off). In that case the mean is a lousy measure of central tendency, and inferential statistics about the mean don't really make sense either."
Well, first of all the original poster already specified that he/she is using the mean.
But more importantly: Even if the freezer is either "on" or "off", why would you say it's a "lousy measure of central tendency" to say, for example, that the freezer is on 63% of the time? Doesn't that tell you everything you need to know?
posted by mikeand1 at 8:00 PM on October 23, 2008
Well, first of all the original poster already specified that he/she is using the mean.
But more importantly: Even if the freezer is either "on" or "off", why would you say it's a "lousy measure of central tendency" to say, for example, that the freezer is on 63% of the time? Doesn't that tell you everything you need to know?
posted by mikeand1 at 8:00 PM on October 23, 2008
Some dimensions/distributions are definitely not normal...waist circumference is skewed to the right.
The distribution of waist circumferences is skewed to the right.
The distribution of sample means of waist circumferences, however, is not skewed, at least not for large samples. For samples of 200 drawn randomly from the parent population(s) of waist circumferences, their means are distributed normally with a mean of the population mean and a standard deviation of the population standard deviation divided by 14.1.
The way to think about it is this: you have two populations with means mu_1 and mu_2. You're testing against a null that mu_1=mu_2, or rather that mu_1-mu_2=0. But you don't know mu_1 and mu_2, you only know xbar_1 and xbar_2. So what you're asking from the t-test is, if mu_1-mu_2 were actually 0, what would be the probability of getting an xbar_1-xbar_2 like I actually did get given the sample variances I observe? Here, it doesn't matter what the distributions of x_1 and x_2 are, what matters is how xbar_1 and xbar_2 are distributed. Which we know thanks to St. Gauss, peace be upon him.
Practically, I have no idea what you should do here. I come from a very strongly regression-oriented world, and the "Let's find another absurdly complex way to do a difference of means test!" world you bio/sometimes psych/etc types inhabit is strange and confusing to me. In general, do what published work in your subfield does, with the caveat that if this is for a class, do whatever the class gives you incentives to do.
posted by ROU_Xenophobe at 10:46 PM on October 23, 2008
The distribution of waist circumferences is skewed to the right.
The distribution of sample means of waist circumferences, however, is not skewed, at least not for large samples. For samples of 200 drawn randomly from the parent population(s) of waist circumferences, their means are distributed normally with a mean of the population mean and a standard deviation of the population standard deviation divided by 14.1.
The way to think about it is this: you have two populations with means mu_1 and mu_2. You're testing against a null that mu_1=mu_2, or rather that mu_1-mu_2=0. But you don't know mu_1 and mu_2, you only know xbar_1 and xbar_2. So what you're asking from the t-test is, if mu_1-mu_2 were actually 0, what would be the probability of getting an xbar_1-xbar_2 like I actually did get given the sample variances I observe? Here, it doesn't matter what the distributions of x_1 and x_2 are, what matters is how xbar_1 and xbar_2 are distributed. Which we know thanks to St. Gauss, peace be upon him.
Practically, I have no idea what you should do here. I come from a very strongly regression-oriented world, and the "Let's find another absurdly complex way to do a difference of means test!" world you bio/sometimes psych/etc types inhabit is strange and confusing to me. In general, do what published work in your subfield does, with the caveat that if this is for a class, do whatever the class gives you incentives to do.
posted by ROU_Xenophobe at 10:46 PM on October 23, 2008
If you are really concerned about the underlying distribution of the population, then the Kolmogorov-Smirnov Test may be the thing:
"The Kolmogorov-Smirnov test (KS-test) tries to determine if two datasets differ significantly. The KS-test has the advantage of making no assumption about the distribution of data. (Technically speaking it is non-parametric and distribution free.) Note however, that this generality comes at some cost: other tests (for example Student's t-test) may be more sensitive if the data meet the requirements of the test. In addition to calculating the D statistic, this page will report if the data seem normal or lognormal. (If it is silent, assume normal data at your own risk!) It will enable you to view the data graphically which can help you understand how the data is distributed."
The folks at St. Johns also have an online K-S test, so you don't even have to code.
posted by GarageWine at 9:01 AM on October 24, 2008
"The Kolmogorov-Smirnov test (KS-test) tries to determine if two datasets differ significantly. The KS-test has the advantage of making no assumption about the distribution of data. (Technically speaking it is non-parametric and distribution free.) Note however, that this generality comes at some cost: other tests (for example Student's t-test) may be more sensitive if the data meet the requirements of the test. In addition to calculating the D statistic, this page will report if the data seem normal or lognormal. (If it is silent, assume normal data at your own risk!) It will enable you to view the data graphically which can help you understand how the data is distributed."
The folks at St. Johns also have an online K-S test, so you don't even have to code.
posted by GarageWine at 9:01 AM on October 24, 2008
That uses terrible example data. In the second example, it says that the t-test can't see the difference, but it returns a two-tailed p of .07, which is to say it sees the difference.
With an N of 200 instead of the 20 the examples use, this seems like something hithere would not need to worry about in a just and proper world. But (s)he should do whatever the field does anyway.
If I were really concerned about the distributions, I wouldn't bother with trying to find some extant test named after someone. I'd just bootstrap up a unified population out of the data I have, draw two samples of 200 from that population, note the difference in means, and repeat 100,000 times. This will return the density of differences of means if the null hypothesis of no difference were true.
posted by ROU_Xenophobe at 9:49 AM on October 24, 2008
With an N of 200 instead of the 20 the examples use, this seems like something hithere would not need to worry about in a just and proper world. But (s)he should do whatever the field does anyway.
If I were really concerned about the distributions, I wouldn't bother with trying to find some extant test named after someone. I'd just bootstrap up a unified population out of the data I have, draw two samples of 200 from that population, note the difference in means, and repeat 100,000 times. This will return the density of differences of means if the null hypothesis of no difference were true.
posted by ROU_Xenophobe at 9:49 AM on October 24, 2008
This thread is closed to new comments.
What statistic are you trying to compare for each dimension? The mean? For an N of 200, that's going to be normally distributed pretty much regardless of the distribution of the dimension.
posted by mikeand1 at 6:16 PM on October 23, 2008