Skip
# Too much noise in my data

(adsbygoogle = window.adsbygoogle || []).push({});

(adsbygoogle = window.adsbygoogle || []).push({});

Post

# Too much noise in my data

September 1, 2012 11:33 AM Subscribe

Putting on the math signal: calling the statistics-literate. Trying to change my weight/body composition and track progress in a useful way, but I'm having trouble separating normal daily variation from actual real change.

After more than 30 years of being the skinny kid, I would like to look more like a normal adult male person. While I don't need to be a ripped men's magazine model, looking good in my clothes for once would be nice. So I'm doing Starting Strength, which I am enjoying, and also concentrating on my diet. I definitely feel better, and think I'm starting to see some improvement in how I look. And I guess that's the important part.

But here's the annoying part. It's not super-important because, like I said, how I feel is the main thing. But, as a curiosity/motivation thing, I'd like to track my progress with data of some sort.

So I'm tracking the weight I'm lifting, obviously, but also my body weight. Problem is, my weight seems to vary so much anyway from day to day, I can't really tell if there is a trend in my weight, or if I just randomly got a few high measurements in a row.

What I've got now is a spreadsheet with my daily measured weight. I'm doing my best to eliminate sources of error, such as using the same scale every time, same time of day, etc., but I'm still looking at a pretty noisy set of data. I'm using a double exponentially-smoothed curve to kind of eyeball it, with the smoothing factors set pretty low, and that looks kind of promisingly up-slope, but when I look at the raw data points, I could also easily convince myself that there has been no change at all, with standard deviation of my points over two pounds. Plus, standard deviation from a simple mean is less than RMS error of my smoothed curve. Is that bad?

So my question is for actual scientific-type people, in two three parts: 1) what would you measure in the first place to track progress, if not body weight, and how would you track it using a simple tool like excel 2) how can I tell (like, use nerdy statistical tests) if my eyeball impression of "looks like it's going up" is actually a real trend or not, and if so, how much is it going up? Or is it just one of those things where the change is going to be too small to see until a long time passes? (I'm only 3 weeks in)

(Yes, I know. You're envious/would love to have my problem/hate me. I'm kind of weary of those jokes, thanks.)

After more than 30 years of being the skinny kid, I would like to look more like a normal adult male person. While I don't need to be a ripped men's magazine model, looking good in my clothes for once would be nice. So I'm doing Starting Strength, which I am enjoying, and also concentrating on my diet. I definitely feel better, and think I'm starting to see some improvement in how I look. And I guess that's the important part.

But here's the annoying part. It's not super-important because, like I said, how I feel is the main thing. But, as a curiosity/motivation thing, I'd like to track my progress with data of some sort.

So I'm tracking the weight I'm lifting, obviously, but also my body weight. Problem is, my weight seems to vary so much anyway from day to day, I can't really tell if there is a trend in my weight, or if I just randomly got a few high measurements in a row.

What I've got now is a spreadsheet with my daily measured weight. I'm doing my best to eliminate sources of error, such as using the same scale every time, same time of day, etc., but I'm still looking at a pretty noisy set of data. I'm using a double exponentially-smoothed curve to kind of eyeball it, with the smoothing factors set pretty low, and that looks kind of promisingly up-slope, but when I look at the raw data points, I could also easily convince myself that there has been no change at all, with standard deviation of my points over two pounds. Plus, standard deviation from a simple mean is less than RMS error of my smoothed curve. Is that bad?

So my question is for actual scientific-type people, in two three parts: 1) what would you measure in the first place to track progress, if not body weight, and how would you track it using a simple tool like excel 2) how can I tell (like, use nerdy statistical tests) if my eyeball impression of "looks like it's going up" is actually a real trend or not, and if so, how much is it going up? Or is it just one of those things where the change is going to be too small to see until a long time passes? (I'm only 3 weeks in)

(Yes, I know. You're envious/would love to have my problem/hate me. I'm kind of weary of those jokes, thanks.)

it sounds like you are overthinking it. if you weigh more every week than you did the week before, then you're gaining. daily measurements are just not going to make sense.

posted by facetious at 12:00 PM on September 1, 2012

posted by facetious at 12:00 PM on September 1, 2012

beeminder.com does a lovely job of turning daily measurements into pretty graphs that include time-lagging (to eliminate 'noise'), optional rose-colored spectacles, etc.

posted by feral_goldfish at 12:01 PM on September 1, 2012

posted by feral_goldfish at 12:01 PM on September 1, 2012

When I was in college I tried the Hacker's Diet method of using an exponentially smoothed weighted moving average. It takes a couple weeks of input to really start working well, but if you've been tracking your daily weight already you should be able to apply the moving average to your past data.

posted by muddgirl at 12:05 PM on September 1, 2012 [2 favorites]

posted by muddgirl at 12:05 PM on September 1, 2012 [2 favorites]

(I think if you dig around that site a little bit you can find the actual implementation, but it's pretty easy to figure out what's going on from the name itself.

posted by muddgirl at 12:07 PM on September 1, 2012

posted by muddgirl at 12:07 PM on September 1, 2012

facetious: yeah, I am, but I'm doing it on purpose. It's more of a math curiosity thing to do while I wait. :)

I do weigh more this week than last... but do I? My measurements vary by many pounds day to day, so can I really say I gained 1.5 pounds last week? I was hoping there was some kind of mathmatical test of "statistical significance" of the change, like, I'm 90% sure I did gain weight and didn't just get a random outlier measurement.

posted by ctmf at 12:08 PM on September 1, 2012

I do weigh more this week than last... but do I? My measurements vary by many pounds day to day, so can I really say I gained 1.5 pounds last week? I was hoping there was some kind of mathmatical test of "statistical significance" of the change, like, I'm 90% sure I did gain weight and didn't just get a random outlier measurement.

posted by ctmf at 12:08 PM on September 1, 2012

(And yes, if you're only three weeks in, your daily noise may be higher than any real effect on your weight).

posted by muddgirl at 12:09 PM on September 1, 2012

posted by muddgirl at 12:09 PM on September 1, 2012

I'm using this smoothing method which looks like this.

posted by ctmf at 12:12 PM on September 1, 2012

posted by ctmf at 12:12 PM on September 1, 2012

If you're gaining muscle mainly for aesthetic reasons, then the best things to measure are muscle sizes: bicep circumference etc. Also good to measure is your one-rep maximum of various exercises which you can measure every other week or so. There are guides online that you can use to learn the correct way to consistently measure muscle size and 1-RM.

These are the best measurements in your case because they're the closest to what you actually want to measure.

Weight is obviously also something you can measure, I measure mine using an exponentially smoothed moving average which, I got from the Hacker's Diet.

If you really wanted to do statistical tests to determine whether you were actually gaining weight, you'd want to do the following:

1) Weigh yourself a number of times over the course of a day to get an idea of what the distribution of measures looks like due to measurement error and gastric fullness.

2) Use that model to predict how likely it is that two measured weights X lbs apart actually represent different "real" weights.

posted by atrazine at 12:16 PM on September 1, 2012

These are the best measurements in your case because they're the closest to what you actually want to measure.

Weight is obviously also something you can measure, I measure mine using an exponentially smoothed moving average which, I got from the Hacker's Diet.

If you really wanted to do statistical tests to determine whether you were actually gaining weight, you'd want to do the following:

1) Weigh yourself a number of times over the course of a day to get an idea of what the distribution of measures looks like due to measurement error and gastric fullness.

2) Use that model to predict how likely it is that two measured weights X lbs apart actually represent different "real" weights.

posted by atrazine at 12:16 PM on September 1, 2012

People tracking securities often want to do the same thing you're trying to do, ignore the day to day fluctuations in highly volatile securities and instead see the bigger picture. They often use either the simple moving average (which lags the underlying data) or exponential moving average (which is an attempt to do what the SMA does, but without lag). To give you an idea how they work, here's the Intel stock price over the past 6 months with SMA-13 (green) and EMA-13 (blue):

Click Clicky

Excel should be able to figure out both for you rather simply.

posted by Brian Puccio at 1:03 PM on September 1, 2012

Click Clicky

Excel should be able to figure out both for you rather simply.

posted by Brian Puccio at 1:03 PM on September 1, 2012

As far as #2 goes, there are plenty of reasonable ways to investigate whether the trends you're seeing are real or not. However, they may not tell you much beyond what you've already discovered by smoothing: that you've got pretty low signal to noise on the data you're trying to measure. Human eyes are pretty good at finding slopes in data, and you're probably more likely to see a slope where there isn't one than to not see one that exists.

Chances are the variations on a timescale of many hours are real, and there's not much more you can do to measure an underlying long-timescale trend except to take more data. Even if you can tease out a systematic trend now, until it's at least comparable the magnitude of daily variations, you'll have a tough time insuring that you haven't accidentally done something to correlate power on the two different timescales. (eg. by waking or eating lunch a little bit later in the day as the season changes.)

Whenever applying statistical tests to anything, good advice is to choose a test you understand well and won't screw up. The simplest option might be to do a least squares fit to the data with two simple functions, for example, a line with a slope and offset (weight=a+b*time) vs. just a constant. You can then compare the chi-squared values and, hopefully, claim that the linear slope is a good fit and that the constant-weight result is statistically unlikely. There's a bit of sneakiness involved, since you'll have to estimate the noise on a given measurement to get a value for the probability to exceed a given chisq, and the only way to do that is from the data itself.

A really easy and only slightly wrong approach would be to calculate the root-mean-square value of a small segment of the data and then use that as your estimate of the noise in a given measurement when calculating chisq in the two fits. This is cheating, since you're using the data twice - but, if the chunk you use for the rms calculation is small compared to the whole data set, it won't change the results by much. An alternative approach is to fit a line and slope, subtract it from the data, and then use the RMS of the residuals to estimate the error on a measurement. This is *really* cheating, as it insures that the chisq of the linear fit will come out exactly as you expect. But, if you then plug in the same error estimate into both fits and the probability to exceed chisq for the constant-weight fit is significantly lower, it's not unreasonably to claim that your slope is real. (In principle, you can even derive an estimate of the error on your derived slope: eg, a weight gain per day +/- some percent, and most canned least-square fitting routines will hand you that result. But, keep in mind that you're cheating both by re-using data and by assuming the noise on a measurement is Gaussian, both of which would make me hesitate to trust those values.)

On longer timescales, you could change to a more complicated function, either by adding additional (eg. polynomial) terms or by stitching together piece-wise linear fits.

To really do this rigorously is a lot harder - but, for these purposes it doesn't seem crazy to do something like this. The simple way to do it with less cheating is to go back in time and start weighing yourself regularly well before starting the exercise routine. (There are also many sophisticated statistical tests which could potentially do a little better than this at teasing out a result - but, the ratio of payoff to complicated math that one can screw up grows rather quickly.)

posted by eotvos at 1:04 PM on September 1, 2012 [1 favorite]

Chances are the variations on a timescale of many hours are real, and there's not much more you can do to measure an underlying long-timescale trend except to take more data. Even if you can tease out a systematic trend now, until it's at least comparable the magnitude of daily variations, you'll have a tough time insuring that you haven't accidentally done something to correlate power on the two different timescales. (eg. by waking or eating lunch a little bit later in the day as the season changes.)

Whenever applying statistical tests to anything, good advice is to choose a test you understand well and won't screw up. The simplest option might be to do a least squares fit to the data with two simple functions, for example, a line with a slope and offset (weight=a+b*time) vs. just a constant. You can then compare the chi-squared values and, hopefully, claim that the linear slope is a good fit and that the constant-weight result is statistically unlikely. There's a bit of sneakiness involved, since you'll have to estimate the noise on a given measurement to get a value for the probability to exceed a given chisq, and the only way to do that is from the data itself.

A really easy and only slightly wrong approach would be to calculate the root-mean-square value of a small segment of the data and then use that as your estimate of the noise in a given measurement when calculating chisq in the two fits. This is cheating, since you're using the data twice - but, if the chunk you use for the rms calculation is small compared to the whole data set, it won't change the results by much. An alternative approach is to fit a line and slope, subtract it from the data, and then use the RMS of the residuals to estimate the error on a measurement. This is *really* cheating, as it insures that the chisq of the linear fit will come out exactly as you expect. But, if you then plug in the same error estimate into both fits and the probability to exceed chisq for the constant-weight fit is significantly lower, it's not unreasonably to claim that your slope is real. (In principle, you can even derive an estimate of the error on your derived slope: eg, a weight gain per day +/- some percent, and most canned least-square fitting routines will hand you that result. But, keep in mind that you're cheating both by re-using data and by assuming the noise on a measurement is Gaussian, both of which would make me hesitate to trust those values.)

On longer timescales, you could change to a more complicated function, either by adding additional (eg. polynomial) terms or by stitching together piece-wise linear fits.

To really do this rigorously is a lot harder - but, for these purposes it doesn't seem crazy to do something like this. The simple way to do it with less cheating is to go back in time and start weighing yourself regularly well before starting the exercise routine. (There are also many sophisticated statistical tests which could potentially do a little better than this at teasing out a result - but, the ratio of payoff to complicated math that one can screw up grows rather quickly.)

posted by eotvos at 1:04 PM on September 1, 2012 [1 favorite]

There isn't much data but it looks like you have fluctuations on a roughly 7 day time scale. You have 5 high points, then 2 low points (though the last week is a little jagged). This makes sense and is similar in magnitude to my own weekly weight fluctuation.

You just need a couple more weeks to establish a trend. I would just take a 7-day centered moving average. A smoothed exponential is fine, but should use at least a week's worth of measurements.

Measuring weight is easy so it's nice to do. But you should measure muscle circumferences once per week: biceps (left and right), forearms, thighs, chest waist. I've always found that hard to keep up with, but you may find it motivating.

posted by gimletbiggles at 5:49 PM on September 1, 2012

You just need a couple more weeks to establish a trend. I would just take a 7-day centered moving average. A smoothed exponential is fine, but should use at least a week's worth of measurements.

Measuring weight is easy so it's nice to do. But you should measure muscle circumferences once per week: biceps (left and right), forearms, thighs, chest waist. I've always found that hard to keep up with, but you may find it motivating.

posted by gimletbiggles at 5:49 PM on September 1, 2012

atrazine:

That's

gimletbiggles: I noticed the 7-day periodicity, too. I hate eating, so I'm lazy about it on the weekends, but I hadn't ever realized it would make a difference that striking. And Tuesday I skipped lunch working. So, yay, data; now I know.

posted by ctmf at 6:26 PM on September 1, 2012

*If you really wanted to do statistical tests to determine whether you were actually gaining weight, you'd want to do the following:*That's

*exactly*what I want to do. Any pointers to an explanation how I would go about that? I think it would be funny to be able to say, "Data shows I'm gaining weight! (small print: 4% confidence)" At least then I could say (humorously, I'm not reading false precision into this) it was progress when that turned into 12% confidence.gimletbiggles: I noticed the 7-day periodicity, too. I hate eating, so I'm lazy about it on the weekends, but I hadn't ever realized it would make a difference that striking. And Tuesday I skipped lunch working. So, yay, data; now I know.

posted by ctmf at 6:26 PM on September 1, 2012

The measuring and progress tracking thing is fun as far as it goes, but in my experience once you start fiddling about with your data to make it look prettier it will start telling you lies that

You already know what you need to do, your body will tell you from inside whether you're doing it right, and the fewer opportunities you give your rationalizing mind to screw that process up, the better off you'll be.

That's the

posted by flabdablet at 7:05 PM on September 1, 2012

*you will start believing.*You already know what you need to do, your body will tell you from inside whether you're doing it right, and the fewer opportunities you give your rationalizing mind to screw that process up, the better off you'll be.

*I definitely feel better, and think I'm starting to see some improvement in how I look. And I guess that's the important part.*That's the

**only**part. Don't do any datawankery at all until you truly believe that.posted by flabdablet at 7:05 PM on September 1, 2012

You'll need to average it over a good part of time; so any day's data won't tell you much.

There are exceptions, and in fact there's a whole field of manufacturing engineering that's built around this question: "How can I tell if this latest data point indicates something has changed?" The eye can see slopes pretty well but it can imagine them too, and it needs to be accustomed to the typical variation of the data in order not to over-react. So there are plenty of systems out there for statistically analyzing this - Statistical Process Control is the technical term.

I'd probably do something like this:

1. Figure out the typical / current distribution of your weight.

2. figure out what the standard deviation is, and multiply that by 3 to get the three-sigma variation.

3. If your weight goes over that point, it's fairly unambiguous that you've gained weight!

4. In addition, if your weight is over the mean-plus-one-standard-deviation for several days it is similarly very likely you've gained weight. You have a 16% chance of being over 1-std-dev if there's been no true change, drops to <3% after even just a second day. So that's a 97% confidence interval that you weigh more.

There are two problems here. The statistical one is that you need to have enough initial data / a normal enough distribution of weights to define your initial distribution. The practical one is that, even if you've gained a few pounds of weight it may not be due to your workouts. Maybe it's just because it's now Tuesday and you've been eating enough again. So you'll want to look for a stronger signal than that - something like several weeks of consistent trend up in the average.

posted by Lady Li at 9:25 PM on September 1, 2012 [1 favorite]

There are exceptions, and in fact there's a whole field of manufacturing engineering that's built around this question: "How can I tell if this latest data point indicates something has changed?" The eye can see slopes pretty well but it can imagine them too, and it needs to be accustomed to the typical variation of the data in order not to over-react. So there are plenty of systems out there for statistically analyzing this - Statistical Process Control is the technical term.

I'd probably do something like this:

1. Figure out the typical / current distribution of your weight.

2. figure out what the standard deviation is, and multiply that by 3 to get the three-sigma variation.

3. If your weight goes over that point, it's fairly unambiguous that you've gained weight!

4. In addition, if your weight is over the mean-plus-one-standard-deviation for several days it is similarly very likely you've gained weight. You have a 16% chance of being over 1-std-dev if there's been no true change, drops to <3% after even just a second day. So that's a 97% confidence interval that you weigh more.

There are two problems here. The statistical one is that you need to have enough initial data / a normal enough distribution of weights to define your initial distribution. The practical one is that, even if you've gained a few pounds of weight it may not be due to your workouts. Maybe it's just because it's now Tuesday and you've been eating enough again. So you'll want to look for a stronger signal than that - something like several weeks of consistent trend up in the average.

posted by Lady Li at 9:25 PM on September 1, 2012 [1 favorite]

This thread is closed to new comments.

posted by unixrat at 11:34 AM on September 1, 2012