Comments on: Statistical analysis for a prospective cohort study with quantifiable outcomes?
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes/
Comments on Ask MetaFilter post Statistical analysis for a prospective cohort study with quantifiable outcomes?Thu, 15 Nov 2012 12:57:32 -0800Thu, 15 Nov 2012 13:00:52 -0800en-ushttp://blogs.law.harvard.edu/tech/rss60Question: Statistical analysis for a prospective cohort study with quantifiable outcomes?
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes
What kind of statistical analysis would I use to compare the outcomes of a prospective cohort study, one with an intervention and one as the control? <br /><br /> Sorry, more specifically, my friend and I are performing a prospective cohort study where there are two groups of 15 athletes, and the goal is to provide one (randomized) cohort with an intervention that we think will prevent injuries; we are then going to compare outcomes by giving everyone in the intervention and the control group a validated test that is a good predictor for injuries. The test output is quantifiable, it will be either a single number for each subject, or a set of 3-5 numbers. Once we have test results for both groups, what kind of analysis would make the most sense to use? <br>
<br>
Just to be clear, we are not tracking injuries, so there is no 'survival' time as in clinical trials. We are simply using the validated test as a proxy to extrapolate likelihood for injury. The test numbers are the outcome. Is there some specific justification we can provide for our (quite low) n, aside from the fact that this is just what we consider doable at this point?<br>
<br>
Normally for something like this I would sit down and try to read more on the internet, but we're short on time as we unexpectedly have to submit this for IRB approval with an organization by tomorrow.post:ask.metafilter.com,2012:site.228974Thu, 15 Nov 2012 12:57:32 -0800legospacemanstatisticsstudyprospectivecohortBy: brainmouse
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3313832
Isn't this something that you'd want to use just a straight-up, super normal <a href="http://en.wikipedia.org/wiki/T-test">t-test</a>? You can also do a <a href="http://en.wikipedia.org/wiki/Statistical_power">Power analysis</a> to see if your sample size is reasonable.comment:ask.metafilter.com,2012:site.228974-3313832Thu, 15 Nov 2012 13:00:52 -0800brainmouseBy: theodolite
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3313842
If assignment is random, and you're not worried about noncompliance or anything, a simple comparison of means (t-test) should work. However, you should probably give everyone the test at the beginning as well, to verify that the indicator is not significantly different between groups at the baseline.comment:ask.metafilter.com,2012:site.228974-3313842Thu, 15 Nov 2012 13:10:02 -0800theodoliteBy: un petit cadeau
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3313850
You might want to consider using the <a href="http://en.wikipedia.org/wiki/Mann–Whitney_U">Mann-Whitney U-test</a> (also known as the Wilcoxon rank-sum test) too. Can you tell us more about the properties of your outcome variable?comment:ask.metafilter.com,2012:site.228974-3313850Thu, 15 Nov 2012 13:15:29 -0800un petit cadeauBy: floweredfish
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3313866
Well, what is your outcome exactly? Are you looking to prove that the intervention is non-inferior (e.g. just as good as) to no intervention? Or are you looking to prove that the intervention is superior to doing nothing? I'd recommend a non-inferiority analysis using propensity scores.comment:ask.metafilter.com,2012:site.228974-3313866Thu, 15 Nov 2012 13:26:01 -0800floweredfishBy: legospaceman
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3313897
I forgot to mention, we will be establishing a baseline by testing in the beginning as well.<br>
<br>
The proposed outcome is improved performance on a balance test which has a measurable, linear output; the higher the number, the better you have performed.comment:ask.metafilter.com,2012:site.228974-3313897Thu, 15 Nov 2012 13:44:03 -0800legospacemanBy: a robot made out of meat
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3313931
<em>Once we have test results for both groups, what kind of analysis would make the most sense to use?</em> <br>
<br>
Ask important determinant of the outcome at baseline (with the pre-test) and followup that you think matter and can plausibly change over the course of the study. After the fact, 1) demonstrate that important determinants have about the same frequency in both groups (this just shows that randomization worked) 2) just do a t-test (or a rank-based relative like MWU) 3) do a regression with changes in measured determinants and treatment as predictors and the change in your test as the outcome. I would not use propensity scores or other complex causal inference tools since you have randomized controls and need to submit tomorrow.<br>
<br>
<em>Is there some specific justification we can provide for our (quite low) n, aside from the fact that this is just what we consider doable at this point?</em><br>
<br>
If you have a reasonable idea of how variable the short-term change in your test is likely to be, you can say how large a treatment effect you can exclude with a null result. This is the flip side of a sample size calculation, you can use minimum detectable difference calculators like <a href="http://hedwig.mgh.harvard.edu/sample_size/size.html">these</a>.comment:ask.metafilter.com,2012:site.228974-3313931Thu, 15 Nov 2012 14:00:19 -0800a robot made out of meatBy: Keter
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3314071
If you are establishing baselines as well, wouldn't a repeated-measures AN(C)OVA with two time points and simple between-groups factor (possibly with baseline <outcome> as covariate) be appropriate?</outcome>comment:ask.metafilter.com,2012:site.228974-3314071Thu, 15 Nov 2012 17:00:57 -0800KeterBy: srboisvert
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3314154
I'd suggest using whatever tests are commonly used in the literature you are citing.<br>
<br>
Different fields have different preferred statistical methods and quirks and using something unfamiliar means your peer reviewers will be scratching their heads both when reviewing your grant and when you try to publish what you have done. Always make it as easy as possible for the consumers of your work to understand what you have done.comment:ask.metafilter.com,2012:site.228974-3314154Thu, 15 Nov 2012 19:01:49 -0800srboisvertBy: lulu68
http://ask.metafilter.com/228974/Statistical-analysis-for-a-prospective-cohort-study-with-quantifiable-outcomes#3314247
If your outcome is a continuous variable and you have two groups (control and intervention) you would use a t-test (mentioned previously) if the outcome variable is distrubuted normally, and a Mann-Whitney U (also mentioned previously) if the outcome variable is skewed. A t-test compares means, a Mann-Whitney compares medians. If the outcome does not approximate the normal distribution, it is violating a major assumption of a t-test, which is that the data are normally distributed. That is why you would use a Mann-Whitney.<br>
<br>
Caveat: with such a small sample size you are unlikely to find a statistically significant difference between groups.comment:ask.metafilter.com,2012:site.228974-3314247Thu, 15 Nov 2012 22:08:24 -0800lulu68