Comments on: Too much noise in my data
http://ask.metafilter.com/223581/Too-much-noise-in-my-data/
Comments on Ask MetaFilter post Too much noise in my dataSat, 01 Sep 2012 11:33:39 -0800Sat, 01 Sep 2012 11:34:45 -0800en-ushttp://blogs.law.harvard.edu/tech/rss60Question: Too much noise in my data
http://ask.metafilter.com/223581/Too-much-noise-in-my-data
Putting on the math signal: calling the statistics-literate. Trying to change my weight/body composition and track progress in a useful way, but I'm having trouble separating normal daily variation from actual real change. <br /><br /> After more than 30 years of being the skinny kid, I would like to look more like a normal adult male person. While I don't need to be a ripped men's magazine model, looking good in my clothes for once would be nice. So I'm doing Starting Strength, which I am enjoying, and also concentrating on my diet. I definitely feel better, and think I'm starting to see some improvement in how I look. And I guess that's the important part.<br>
<br>
But here's the annoying part. It's not super-important because, like I said, how I feel is the main thing. But, as a curiosity/motivation thing, I'd like to track my progress with data of some sort.<br>
<br>
So I'm tracking the weight I'm lifting, obviously, but also my body weight. Problem is, my weight seems to vary so much anyway from day to day, I can't really tell if there is a trend in my weight, or if I just randomly got a few high measurements in a row.<br>
<br>
What I've got now is a spreadsheet with my daily measured weight. I'm doing my best to eliminate sources of error, such as using the same scale every time, same time of day, etc., but I'm still looking at a pretty noisy set of data. I'm using a double exponentially-smoothed curve to kind of eyeball it, with the smoothing factors set pretty low, and that looks kind of promisingly up-slope, but when I look at the raw data points, I could also easily convince myself that there has been no change at all, with standard deviation of my points over two pounds. Plus, standard deviation from a simple mean is less than RMS error of my smoothed curve. Is that bad?<br>
<br>
So my question is for actual scientific-type people, in two three parts: 1) what would you measure in the first place to track progress, if not body weight, and how would you track it using a simple tool like excel 2) how can I tell (like, use nerdy statistical tests) if my eyeball impression of "looks like it's going up" is actually a real trend or not, and if so, how much is it going up? Or is it just one of those things where the change is going to be too small to see until a long time passes? (I'm only 3 weeks in)<br>
<br>
(Yes, I know. You're envious/would love to have my problem/hate me. I'm kind of weary of those jokes, thanks.)post:ask.metafilter.com,2012:site.223581Sat, 01 Sep 2012 11:33:39 -0800ctmfstatisticsmathhealthexerciseweightlosstrackingBy: unixrat
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3232975
Physicsdiet.com?comment:ask.metafilter.com,2012:site.223581-3232975Sat, 01 Sep 2012 11:34:45 -0800unixratBy: facetious
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3232993
it sounds like you are overthinking it. if you weigh more every week than you did the week before, then you're gaining. daily measurements are just not going to make sense.comment:ask.metafilter.com,2012:site.223581-3232993Sat, 01 Sep 2012 12:00:10 -0800facetiousBy: feral_goldfish
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3232996
beeminder.com does a lovely job of turning daily measurements into pretty graphs that include time-lagging (to eliminate 'noise'), optional rose-colored spectacles, etc.comment:ask.metafilter.com,2012:site.223581-3232996Sat, 01 Sep 2012 12:01:56 -0800feral_goldfishBy: muddgirl
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233000
When I was in college I tried the Hacker's Diet method of using an <a href="http://www.fourmilab.ch/hackdiet/www/subsubsection1_2_4_0_4_3.html#SECTION0240430000000000000">exponentially smoothed weighted moving average</a>. It takes a couple weeks of input to really start working well, but if you've been tracking your daily weight already you should be able to apply the moving average to your past data.comment:ask.metafilter.com,2012:site.223581-3233000Sat, 01 Sep 2012 12:05:34 -0800muddgirlBy: muddgirl
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233001
(I think if you dig around that site a little bit you can find the actual implementation, but it's pretty easy to figure out what's going on from the name itself.comment:ask.metafilter.com,2012:site.223581-3233001Sat, 01 Sep 2012 12:07:06 -0800muddgirlBy: ctmf
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233004
facetious: yeah, I am, but I'm doing it on purpose. It's more of a math curiosity thing to do while I wait. :)<br>
<br>
I do weigh more this week than last... but do I? My measurements vary by many pounds day to day, so can I really say I gained 1.5 pounds last week? I was hoping there was some kind of mathmatical test of "statistical significance" of the change, like, I'm 90% sure I did gain weight and didn't just get a random outlier measurement.comment:ask.metafilter.com,2012:site.223581-3233004Sat, 01 Sep 2012 12:08:52 -0800ctmfBy: muddgirl
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233005
(And yes, if you're only three weeks in, your daily noise may be higher than any real effect on your weight).comment:ask.metafilter.com,2012:site.223581-3233005Sat, 01 Sep 2012 12:09:17 -0800muddgirlBy: ctmf
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233009
I'm using <a href="http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc433.htm">this</a> smoothing method which looks like <a href="http://mlkshk.com/p/J077">this</a>.comment:ask.metafilter.com,2012:site.223581-3233009Sat, 01 Sep 2012 12:12:35 -0800ctmfBy: atrazine
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233014
If you're gaining muscle mainly for aesthetic reasons, then the best things to measure are muscle sizes: bicep circumference etc. Also good to measure is your one-rep maximum of various exercises which you can measure every other week or so. There are guides online that you can use to learn the correct way to consistently measure muscle size and 1-RM.<br>
<br>
These are the best measurements in your case because they're the closest to what you actually want to measure.<br>
<br>
Weight is obviously also something you can measure, I measure mine using an exponentially smoothed moving average which, I got from the Hacker's Diet.<br>
<br>
If you really wanted to do statistical tests to determine whether you were actually gaining weight, you'd want to do the following:<br>
1) Weigh yourself a number of times over the course of a day to get an idea of what the distribution of measures looks like due to measurement error and gastric fullness.<br>
2) Use that model to predict how likely it is that two measured weights X lbs apart actually represent different "real" weights.comment:ask.metafilter.com,2012:site.223581-3233014Sat, 01 Sep 2012 12:16:26 -0800atrazineBy: Brian Puccio
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233049
People tracking securities often want to do the same thing you're trying to do, ignore the day to day fluctuations in highly volatile securities and instead see the bigger picture. They often use either the <a href="http://en.wikipedia.org/wiki/Moving_average#Simple_moving_average">simple moving average</a> (which lags the underlying data) or <a href="http://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average">exponential moving average</a> (which is an attempt to do what the SMA does, but without lag). To give you an idea how they work, here's the Intel stock price over the past 6 months with SMA-13 (green) and EMA-13 (blue):<br>
<br>
<a href="http://i.imgur.com/oBerH.png">Click Clicky</a><br>
<br>
Excel should be able to figure out both for you rather simply.comment:ask.metafilter.com,2012:site.223581-3233049Sat, 01 Sep 2012 13:03:23 -0800Brian PuccioBy: eotvos
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233052
As far as #2 goes, there are plenty of reasonable ways to investigate whether the trends you're seeing are real or not. However, they may not tell you much beyond what you've already discovered by smoothing: that you've got pretty low signal to noise on the data you're trying to measure. Human eyes are pretty good at finding slopes in data, and you're probably more likely to see a slope where there isn't one than to not see one that exists.<br>
<br>
Chances are the variations on a timescale of many hours are real, and there's not much more you can do to measure an underlying long-timescale trend except to take more data. Even if you can tease out a systematic trend now, until it's at least comparable the magnitude of daily variations, you'll have a tough time insuring that you haven't accidentally done something to correlate power on the two different timescales. (eg. by waking or eating lunch a little bit later in the day as the season changes.)<br>
<br>
Whenever applying statistical tests to anything, good advice is to choose a test you understand well and won't screw up. The simplest option might be to do a least squares fit to the data with two simple functions, for example, a line with a slope and offset (weight=a+b*time) vs. just a constant. You can then compare the chi-squared values and, hopefully, claim that the linear slope is a good fit and that the constant-weight result is statistically unlikely. There's a bit of sneakiness involved, since you'll have to estimate the noise on a given measurement to get a value for the probability to exceed a given chisq, and the only way to do that is from the data itself.<br>
<br>
A really easy and only slightly wrong approach would be to calculate the root-mean-square value of a small segment of the data and then use that as your estimate of the noise in a given measurement when calculating chisq in the two fits. This is cheating, since you're using the data twice - but, if the chunk you use for the rms calculation is small compared to the whole data set, it won't change the results by much. An alternative approach is to fit a line and slope, subtract it from the data, and then use the RMS of the residuals to estimate the error on a measurement. This is *really* cheating, as it insures that the chisq of the linear fit will come out exactly as you expect. But, if you then plug in the same error estimate into both fits and the probability to exceed chisq for the constant-weight fit is significantly lower, it's not unreasonably to claim that your slope is real. (In principle, you can even derive an estimate of the error on your derived slope: eg, a weight gain per day +/- some percent, and most canned least-square fitting routines will hand you that result. But, keep in mind that you're cheating both by re-using data and by assuming the noise on a measurement is Gaussian, both of which would make me hesitate to trust those values.)<br>
<br>
On longer timescales, you could change to a more complicated function, either by adding additional (eg. polynomial) terms or by stitching together piece-wise linear fits.<br>
<br>
To really do this rigorously is a lot harder - but, for these purposes it doesn't seem crazy to do something like this. The simple way to do it with less cheating is to go back in time and start weighing yourself regularly well before starting the exercise routine. (There are also many sophisticated statistical tests which could potentially do a little better than this at teasing out a result - but, the ratio of payoff to complicated math that one can screw up grows rather quickly.)comment:ask.metafilter.com,2012:site.223581-3233052Sat, 01 Sep 2012 13:04:19 -0800eotvosBy: gimletbiggles
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233248
There isn't much data but it looks like you have fluctuations on a roughly 7 day time scale. You have 5 high points, then 2 low points (though the last week is a little jagged). This makes sense and is similar in magnitude to my own weekly weight fluctuation. <br>
<br>
You just need a couple more weeks to establish a trend. I would just take a 7-day centered moving average. A smoothed exponential is fine, but should use at least a week's worth of measurements.<br>
<br>
Measuring weight is easy so it's nice to do. But you should measure muscle circumferences once per week: biceps (left and right), forearms, thighs, chest waist. I've always found that hard to keep up with, but you may find it motivating.comment:ask.metafilter.com,2012:site.223581-3233248Sat, 01 Sep 2012 17:49:51 -0800gimletbigglesBy: ctmf
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233267
atrazine: <em>If you really wanted to do statistical tests to determine whether you were actually gaining weight, you'd want to do the following:</em><br>
<br>
That's <em>exactly</em> what I want to do. Any pointers to an explanation how I would go about that? I think it would be funny to be able to say, "Data shows I'm gaining weight! (small print: 4% confidence)" At least then I could say (humorously, I'm not reading false precision into this) it was progress when that turned into 12% confidence.<br>
<br>
gimletbiggles: I noticed the 7-day periodicity, too. I hate eating, so I'm lazy about it on the weekends, but I hadn't ever realized it would make a difference that striking. And Tuesday I skipped lunch working. So, yay, data; now I know.comment:ask.metafilter.com,2012:site.223581-3233267Sat, 01 Sep 2012 18:26:21 -0800ctmfBy: flabdablet
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233281
The measuring and progress tracking thing is fun as far as it goes, but in my experience once you start fiddling about with your data to make it look prettier it will start telling you lies that <em>you will start believing.</em><br>
<br>
You already know what you need to do, your body will tell you from inside whether you're doing it right, and the fewer opportunities you give your rationalizing mind to screw that process up, the better off you'll be.<br>
<br>
<em>I definitely feel better, and think I'm starting to see some improvement in how I look. And I guess that's the important part.</em><br>
<br>
That's the <strong>only</strong> part. Don't do any datawankery at all until you truly believe that.comment:ask.metafilter.com,2012:site.223581-3233281Sat, 01 Sep 2012 19:05:25 -0800flabdabletBy: Lady Li
http://ask.metafilter.com/223581/Too-much-noise-in-my-data#3233370
You'll need to average it over a good part of time; so any day's data won't tell you much.<br>
<br>
There are exceptions, and in fact there's a whole field of manufacturing engineering that's built around this question: "How can I tell if this latest data point indicates something has changed?" The eye can see slopes pretty well but it can imagine them too, and it needs to be accustomed to the typical variation of the data in order not to over-react. So there are plenty of systems out there for statistically analyzing this - Statistical Process Control is the technical term.<br>
<br>
I'd probably do something like this: <br>
1. Figure out the typical / current distribution of your weight. <br>
2. figure out what the standard deviation is, and multiply that by 3 to get the three-sigma variation.<br>
3. If your weight goes over that point, it's fairly unambiguous that you've gained weight!<br>
4. In addition, if your weight is over the mean-plus-one-standard-deviation for several days it is similarly very likely you've gained weight. You have a 16% chance of being over 1-std-dev if there's been no true change, drops to <3% after even just a second day. So that's a 97% confidence interval that you weigh more.<br>
<br>
There are two problems here. The statistical one is that you need to have enough initial data / a normal enough distribution of weights to define your initial distribution. The practical one is that, even if you've gained a few pounds of weight it may not be due to your workouts. Maybe it's just because it's now Tuesday and you've been eating enough again. So you'll want to look for a stronger signal than that - something like several weeks of consistent trend up in the average.comment:ask.metafilter.com,2012:site.223581-3233370Sat, 01 Sep 2012 21:25:50 -0800Lady Li