Oompa Loompa doompadee doo I've got another puzzle for you
January 18, 2010 7:09 AM Subscribe
I am thinking about doing a study in my school that correlates diet with behaviour, what advice would you give me to ensure watertight data and conclusions? Any other interesting variables I could look at?
There are roughly 800 kids at the school I currently work in. There is an electronic detention logging system which registers the time of the offense etc, so it will be easy to correlate this with many factors such as lessons/lunch/weather.
This is just for personal interest although the data would be useful for the kids to analyse in lessons.
Assume basic knowledge of statistics in your answers!
There are roughly 800 kids at the school I currently work in. There is an electronic detention logging system which registers the time of the offense etc, so it will be easy to correlate this with many factors such as lessons/lunch/weather.
This is just for personal interest although the data would be useful for the kids to analyse in lessons.
Assume basic knowledge of statistics in your answers!
How will you account for the fact that inattentive parents are more likely to send their kids off with a crappy breakfast and money for lunch at the local McDonalds, while attentive parents are more likely to look after their kid's diet as well as their education and behavior?
posted by MuffinMan at 7:44 AM on January 18, 2010
posted by MuffinMan at 7:44 AM on January 18, 2010
This is not the answer that you want to hear, but because you may use this study for the students to analyze I'll give a more elaborate answer.
I would point out to the students that causation does not imply correlation and this will not reach a point of “watertight data and conclusions”.
I would also give the students examples of stronger studies that they always use as a reference when making a conclusion (for example, a prospective, randomized, placebo-controlled trial trumps a retrospective nonrandomized study). Or you could demonstrate what a well designed study should include (ie, a large study population, you do have that). You can use examples from the news to make this relevant, and then they could look at this data set. If possible, I would try to teach students these skills – in the future, they would be able to look at a study and see what it is missing or why study A may have more information than study B.
To make this study study stronger (something beyond correlation), you could try to put a small pilot study that includes randomization and various treatment groups. I think it would be great to include students in this activity and involve students in study design. Why not have an advanced class read a few journal articles and determine what questions have previously been investigated? Propose a hypothesis and alternative hypothesis. Design a study that includes different treatments (ie, high carb group, low carb group, follow current nutrition guidelines -I don’t know, you guys decide). See if you can then get another group of students to volunteer as participants and randomize students to the treatment groups. How will you measure adherence to the plan? Students could collect data (blood pressure at different time points, etc.), etc.
Other variables that have shown to be far more robust in terms of impactiing behavior/cognitive impairment (versus diet) has been sleep and sleep deprivation. You could probably easily have students record when the go to sleep, wake up, consumption of stimulants, etc. You can also find tests in journal articles that measure attention, etc., and you could measure for a difference.
posted by Wolfster at 7:52 AM on January 18, 2010
I would point out to the students that causation does not imply correlation and this will not reach a point of “watertight data and conclusions”.
I would also give the students examples of stronger studies that they always use as a reference when making a conclusion (for example, a prospective, randomized, placebo-controlled trial trumps a retrospective nonrandomized study). Or you could demonstrate what a well designed study should include (ie, a large study population, you do have that). You can use examples from the news to make this relevant, and then they could look at this data set. If possible, I would try to teach students these skills – in the future, they would be able to look at a study and see what it is missing or why study A may have more information than study B.
To make this study study stronger (something beyond correlation), you could try to put a small pilot study that includes randomization and various treatment groups. I think it would be great to include students in this activity and involve students in study design. Why not have an advanced class read a few journal articles and determine what questions have previously been investigated? Propose a hypothesis and alternative hypothesis. Design a study that includes different treatments (ie, high carb group, low carb group, follow current nutrition guidelines -I don’t know, you guys decide). See if you can then get another group of students to volunteer as participants and randomize students to the treatment groups. How will you measure adherence to the plan? Students could collect data (blood pressure at different time points, etc.), etc.
Other variables that have shown to be far more robust in terms of impactiing behavior/cognitive impairment (versus diet) has been sleep and sleep deprivation. You could probably easily have students record when the go to sleep, wake up, consumption of stimulants, etc. You can also find tests in journal articles that measure attention, etc., and you could measure for a difference.
posted by Wolfster at 7:52 AM on January 18, 2010
Yeah, I'd be most concerned with the quality of the data about what they eat. Even if you know what the school served for lunch that day, you have no data about who actually ate what, what else they ate, and what they're eating at other, non-school times. This is a notoriously hard problem that has plagued nutrition and health researchers forever. Self-reports are wildly inaccurate, so unless you plan to lock the kids in a room and monitor their diets for the length of the study, you're not going to be able to get good data on this.
posted by decathecting at 7:52 AM on January 18, 2010
posted by decathecting at 7:52 AM on January 18, 2010
Because there is quite a bit of selection bias and confounding by baseline factors, you should only make within-kid comparisons. Preferably there would be an intervention that you control, and you would do pre-post. Alternatively, $factor on the 10 days before an incident compared to ten other days for the same kid (matched case-control).
There's much less information within-kid than between kid, but you have more validity. You still can't exclude there there is a third factor which varies across time with whatever your supposed risk factor is. For example, kids eat worse in reaction to trouble at home and also pick fights.
I'd also advise you to get a concrete notion about what data you are allowed to access and use for what purposes. Children and their parents have rights to not participate.
posted by a robot made out of meat at 8:09 AM on January 18, 2010
There's much less information within-kid than between kid, but you have more validity. You still can't exclude there there is a third factor which varies across time with whatever your supposed risk factor is. For example, kids eat worse in reaction to trouble at home and also pick fights.
I'd also advise you to get a concrete notion about what data you are allowed to access and use for what purposes. Children and their parents have rights to not participate.
posted by a robot made out of meat at 8:09 AM on January 18, 2010
You might want to read Good Calories, Bad Calories by Gary Taubes; it talks about studies that show especially when it comes to what people eat, correlation does not equal causation. In other words, you can't know if someone is getting fatter or skinnier because they're eating more or less, or if they're getting fatter and eating more or getting skinnier or eating less because of an underlying metabolic issue that causes both. The same goes for any physical activity you may wish to monitor.
In other words, even if you get past the problems others have mentioned of monitoring these things to begin with, the correlation won't be meaningful. Watertight isn't going to happen, unfortunately.
posted by Nattie at 8:14 AM on January 18, 2010
In other words, even if you get past the problems others have mentioned of monitoring these things to begin with, the correlation won't be meaningful. Watertight isn't going to happen, unfortunately.
posted by Nattie at 8:14 AM on January 18, 2010
Are the "kids" minors? If so, you're going to need to get permission from the school district and the parents to collect this kind of data on their kids.
posted by k8t at 10:37 AM on January 18, 2010
posted by k8t at 10:37 AM on January 18, 2010
Response by poster: More information:
Every kid (and staff member) at the school eats the same meal as there is only one item daily. There is a good range throughout the week however, from fish to pizza, to curry etc. Sweets are also banned at all times, but of course there are smugglers.
The ability to log the time the detention was given allows us to consider behaviour after the consumption of the known foods rather than after whatever they had, or didnt have, for breakfast.
My early hypothesis is that there will be a relationship between high sugar/fat/carbohydrate and poor behaviour: I havent looked into much detail yet but will formulate more as I read around.
Control groups will be difficult: although there are vegetarian/halal groups which could be considered.
posted by Saddo at 11:05 AM on January 18, 2010
Every kid (and staff member) at the school eats the same meal as there is only one item daily. There is a good range throughout the week however, from fish to pizza, to curry etc. Sweets are also banned at all times, but of course there are smugglers.
The ability to log the time the detention was given allows us to consider behaviour after the consumption of the known foods rather than after whatever they had, or didnt have, for breakfast.
My early hypothesis is that there will be a relationship between high sugar/fat/carbohydrate and poor behaviour: I havent looked into much detail yet but will formulate more as I read around.
Control groups will be difficult: although there are vegetarian/halal groups which could be considered.
posted by Saddo at 11:05 AM on January 18, 2010
It is certainly reasonable to suggest that, in younger-people, sugar (and possibly some artificial colorings) could contribute to hyperactivity and bad behaviour. However, several studies have instead focused on vitamin deficiencies as a cause of misconduct. Bernard Gesch, at the University of Oxford, has done placebo-controlled experiments into the effectiveness of giving vitamin supplements to prisoners as a way of reducing their violence.
You say that there is a good range of foods throughout the week, but is the order the same every week? This could cause a problem, as diet would be confounded with day of the week. For example, if you give everyone fish on a Friday you might conclude that fish causes poor behavior, when in fact people are just getting restless as the weekend approaches.
Also, could people bring in additional snacks from home, introducing variation in what is being eaten?
Finally, have you looked at FAB research? It looks like it might be useful.
posted by James Scott-Brown at 11:23 AM on January 18, 2010
You say that there is a good range of foods throughout the week, but is the order the same every week? This could cause a problem, as diet would be confounded with day of the week. For example, if you give everyone fish on a Friday you might conclude that fish causes poor behavior, when in fact people are just getting restless as the weekend approaches.
Also, could people bring in additional snacks from home, introducing variation in what is being eaten?
Finally, have you looked at FAB research? It looks like it might be useful.
posted by James Scott-Brown at 11:23 AM on January 18, 2010
If you're in the US, I would worry about inadvertantly violating FERPA by disclosing information to students that could conceivably allow others to learn that a particular student was formally sanctioned.
posted by ROU_Xenophobe at 11:40 AM on January 18, 2010
posted by ROU_Xenophobe at 11:40 AM on January 18, 2010
Response by poster: I'm in the UK so not sure about that: I can always make it anonymous though.
posted by Saddo at 11:59 AM on January 18, 2010
posted by Saddo at 11:59 AM on January 18, 2010
It sounds like you can experimentally manipulate the food that everyone gets. What you could do then is distribute the days of high-bad-thing foods in a pattern (the optimal pattern is complicated) and look at aggregate violations by hour on different food-days. Since the kind of food you put out is uncorrelated to anything, you should be able to draw some conclusions. You also don't have to report any unique violations, just aggregates.
You could just make the order of the options random each week. That's simple. What you want to avoid is an embarrassing randomization where the random number generator puts curry on Monday every week. The exact pattern that you want will be complicated. You obviously want food by day of week to be balanced. You don't want the same order every week, since there could be a lag. If you have more than 5 foods you want them to be roughly the same frequency as time goes by. Are there other things about the calendar which influence bad behavior? Football games? You should think about that and make sure that a food doesn't get a systematically "bad" slot by any of them.
posted by a robot made out of meat at 11:59 AM on January 18, 2010 [1 favorite]
You could just make the order of the options random each week. That's simple. What you want to avoid is an embarrassing randomization where the random number generator puts curry on Monday every week. The exact pattern that you want will be complicated. You obviously want food by day of week to be balanced. You don't want the same order every week, since there could be a lag. If you have more than 5 foods you want them to be roughly the same frequency as time goes by. Are there other things about the calendar which influence bad behavior? Football games? You should think about that and make sure that a food doesn't get a systematically "bad" slot by any of them.
posted by a robot made out of meat at 11:59 AM on January 18, 2010 [1 favorite]
It is certainly reasonable to suggest that, in younger-people, sugar... could contribute to hyperactivity and bad behaviour.
Most recent research disagrees
Regardless of what parents might believe, however, sugar is not to blame for out of control little ones. At least 12 double blind randomised controlled trials have examined how children react to diets containing different levels of sugar.2 None of these studies, not even studies looking specifically at children with attention-deficit/hyperactivity disorder, could detect any differences in behaviour between the children who had sugar and those who did not.3
posted by drezdn at 1:35 PM on January 18, 2010
Most recent research disagrees
Regardless of what parents might believe, however, sugar is not to blame for out of control little ones. At least 12 double blind randomised controlled trials have examined how children react to diets containing different levels of sugar.2 None of these studies, not even studies looking specifically at children with attention-deficit/hyperactivity disorder, could detect any differences in behaviour between the children who had sugar and those who did not.3
posted by drezdn at 1:35 PM on January 18, 2010
Best answer: The ability to log the time the detention was given allows us to consider behaviour after the consumption of the known foods rather than after whatever they had, or didnt have, for breakfast.
Yeah but what about underlying nutritional status? That's going to have a huge effect on behaviour over time. And who says each kid ate the whole meal? Or all the other myriad of factors that go into behaviour over time? You're going to need to work out all the other confounding factors, home life, co-morbidities, overall nutritional status, known behavioural problems, etc, for any of your information to be valid.
Also your hypothesis is a bit suspect, sugar and fat is bad mkay? Nutrition isn't that simple, and behaviour even less so. It sounds suspicously 'trendy', like you haven't read a lot about all the many dietary factors that go into behaviour and instead have chosen the obvious bad guy. I may be wrong but if so you'll need to have a really solid understanding of the literature to convince people otherwise.
You will, at the very least, need have every meal fully characterised, full nutritional content not just the things you've pre-decided. Otherwise how can you tell if it's the change in sugar content having an effect rather than something else that's different between two meals? You also need to consider diet/environment effects, what else is going on with the child and how does that interact with what they've been eating? A child that had no breakfast might react differently to a high sugar lunch than one that had eaten in the morning, or one that is sick or under stress may also have different metabolic capacity for sugar than one that is healthy. Knowing something about their genetic background would be even better, does one group of children metabolise sugars differently than another? Probably less important for macro nutrients like sugar but definitely important if things like vitamin deficiencies start to come into play, and also less useful if you only ever do intra-child comparisons (which I would strongly advise).
Using a group of children on a different diet as a control group doesn't sound right to me. they'll be a different population (particularly if there are cultural factors in their reason for choosing a different diet). Instead you should read about cross over and washout studies. You also need to look at the children having the same diet with different levels of behaviour, what is different and the same between the two? There is literature around about the different types of experimetnal design used in this area and also looking at the methods sections of other papers can help (British Journal of Nutrition and Journal of Nutrition are worth a look), and I don't know if there is one best way of doing this kind of study. Oh yeah, you'll need to do a power analysis to work out minimum group size, although I'm often surprised at how small pilot diet studies are while still being valid.
Diet studies are hard. Read about diet questionnaires and how bad they are and how we don't really have anything better yet. Behavioural studies are also hard, confounding factors have such a huge impact. Trying to put the two together in a rigorous manner is a big task.
Also getting parents to agree to this kind of stuff is super hard. One of my colleagues is doing a nutritional study that involves children, much smaller, simpler and less controversial than what you're proposing, and getting enough kids enrolled was one of her biggest hurdles (getting ethics approval in the first place was the biggest, you need to look into how hard that's going to be for you). And what happens if a child not in the study is interacting with, and influencing, a child that is in the study? (and yeah, you really really can't do this without parent's permission.)
Right now you have, at best, a small, biased pilot study which might tell you if there's anything further worth studying. But even then as described you wouldn't get ethics approval in my country (to be fair, our system is really crazy strict and time consuming). Looking for more info is great but I think this is a much bigger task than one askme can help with. So my main suggestion is that you contact a nutrition department at a University and try to hook up with some researchers, you need someone with experience in these fields to help with experimental design, background reading, and sanity check material. I know you said it's personal interest but you won't get ethics approval without it being scientifically rigorous, and you most definitely 100% should not be doing anything like this without it.
(disclaimer: I'm a researcher in a nutrition based field but not with humans and am not a nutritionist)
posted by shelleycat at 1:45 PM on January 18, 2010 [1 favorite]
Yeah but what about underlying nutritional status? That's going to have a huge effect on behaviour over time. And who says each kid ate the whole meal? Or all the other myriad of factors that go into behaviour over time? You're going to need to work out all the other confounding factors, home life, co-morbidities, overall nutritional status, known behavioural problems, etc, for any of your information to be valid.
Also your hypothesis is a bit suspect, sugar and fat is bad mkay? Nutrition isn't that simple, and behaviour even less so. It sounds suspicously 'trendy', like you haven't read a lot about all the many dietary factors that go into behaviour and instead have chosen the obvious bad guy. I may be wrong but if so you'll need to have a really solid understanding of the literature to convince people otherwise.
You will, at the very least, need have every meal fully characterised, full nutritional content not just the things you've pre-decided. Otherwise how can you tell if it's the change in sugar content having an effect rather than something else that's different between two meals? You also need to consider diet/environment effects, what else is going on with the child and how does that interact with what they've been eating? A child that had no breakfast might react differently to a high sugar lunch than one that had eaten in the morning, or one that is sick or under stress may also have different metabolic capacity for sugar than one that is healthy. Knowing something about their genetic background would be even better, does one group of children metabolise sugars differently than another? Probably less important for macro nutrients like sugar but definitely important if things like vitamin deficiencies start to come into play, and also less useful if you only ever do intra-child comparisons (which I would strongly advise).
Using a group of children on a different diet as a control group doesn't sound right to me. they'll be a different population (particularly if there are cultural factors in their reason for choosing a different diet). Instead you should read about cross over and washout studies. You also need to look at the children having the same diet with different levels of behaviour, what is different and the same between the two? There is literature around about the different types of experimetnal design used in this area and also looking at the methods sections of other papers can help (British Journal of Nutrition and Journal of Nutrition are worth a look), and I don't know if there is one best way of doing this kind of study. Oh yeah, you'll need to do a power analysis to work out minimum group size, although I'm often surprised at how small pilot diet studies are while still being valid.
Diet studies are hard. Read about diet questionnaires and how bad they are and how we don't really have anything better yet. Behavioural studies are also hard, confounding factors have such a huge impact. Trying to put the two together in a rigorous manner is a big task.
Also getting parents to agree to this kind of stuff is super hard. One of my colleagues is doing a nutritional study that involves children, much smaller, simpler and less controversial than what you're proposing, and getting enough kids enrolled was one of her biggest hurdles (getting ethics approval in the first place was the biggest, you need to look into how hard that's going to be for you). And what happens if a child not in the study is interacting with, and influencing, a child that is in the study? (and yeah, you really really can't do this without parent's permission.)
Right now you have, at best, a small, biased pilot study which might tell you if there's anything further worth studying. But even then as described you wouldn't get ethics approval in my country (to be fair, our system is really crazy strict and time consuming). Looking for more info is great but I think this is a much bigger task than one askme can help with. So my main suggestion is that you contact a nutrition department at a University and try to hook up with some researchers, you need someone with experience in these fields to help with experimental design, background reading, and sanity check material. I know you said it's personal interest but you won't get ethics approval without it being scientifically rigorous, and you most definitely 100% should not be doing anything like this without it.
(disclaimer: I'm a researcher in a nutrition based field but not with humans and am not a nutritionist)
posted by shelleycat at 1:45 PM on January 18, 2010 [1 favorite]
This thread is closed to new comments.
posted by yesster at 7:18 AM on January 18, 2010