My good friend Ted Kyle passed along to me this morning a report from the Robert Wood Johnson Foundation, showing that childhood obesity is declining in many areas. With all due respect to RWJF, they have regularly used last year's report that obesity was declining in preschoolers as a demonstration that their policy efforts have been effective. They cite that again in this report, ignoring reports from the same data that contradict those findings. RWJF does incredibly important work, but these particular numbers do not show what we want to believe they do.
I see a bunch of numbers showing declines in obesity? Are you saying they are wrong?
No, I'm saying the numbers in the RWJF report do not say anything about childhood obesity. They say, for the most part, something about the population of underprivileged young children who are enrolled in WIC.
Well, isn't that good news?
Maybe. Maybe not. Children in WIC in 2008 may be different than those in 2011. In fact, we have every reason to believe they were. With the recession, more families became eligible for WIC, including those who were previously "better off" and thus had children at lower risk for obesity.
But we know these places were trying various policies? Even if it's only preschool children, aren't we showing they work?
No. When examining the effect of an intervention--like the policies referred to in the report--we must have a control group. Let me say that again. The decline in obesity rates means nothing without a control group. This report picks out the 19 states with reported declines in obesity in their WIC program. We can't determine if the effects are due to policy changes or regression to the mean. What about all the other states? Did they show an increase? Also, take a look at the magnitude of change. We are talking about changes typically under one percentage point.
Who cares if it's regression to the mean, obesity is still going down. Why are you so hung up on this control group?
Let's assume the data are all wonderful and we do indeed have a real decline in obesity, even if only a percentage point or so. That tells us absolutely nothing. We don't know which policies mattered. This is a critical point. We don't know where to spend our money in the future. We spend millions (billions?) trying to address childhood obesity, and we can't tell you what your money bought.
Maybe money isn't the biggest issue. Maybe we are just thrilled to see some improvement.
Or maybe we are causing harm and don't know it. The flipside of all interventions is that they can have unintended consequences. Maybe we are causing unhealthy eating behaviors. Maybe we're causing undue financial strain. Maybe we are generating psychological harm by stigmatizing obesity. We. Don't. Know.
Are you saying we should stop doing everything?
Of course not. There are many policies, particularly related to WIC that are objectively good, and should be studied for their effectiveness and cost-effectiveness. One example are those that provide WIC vouchers for fresh fruits and vegetables. These are unlikely to cause harm, and likely improve access to fresh fruits and vegetables. This is a great outcome on its own. It's a much bigger leap to assume they play a role in obesity rates.
But Asheley, you research childhood obesity. Aren't you a little biased towards keeping this a problem?
Indeed, I do a great deal of work in childhood obesity. However, I'm far more interested in the health consequences of obesity, measurement, and severe obesity. More importantly, if the childhood obesity problem just disappeared, I could do work in all the other areas that interest me, including child maltreatment and mental health and prescription drug abuse. Maybe I'd get funded in those areas!
So, what do we know for sure?
We know that childhood obesity has not significantly declined for the last 15 years or so.
We don't know that our policies related to childhood obesity have any effect, because we so rarely use control groups, or even adequate quasi-experimental designs in our study of them.
Why do you even care?
I care because I don't want the country to assume childhood obesity is all better. I don't want the need for healthy foods and activity to drop from our radar. I want to be sure that we are not spending enormous amounts of money on programs that are ineffective at best, harmful and stigmatizing at worst. I want our children to be healthier, and reports about reductions in obesity--a measure that itself is hugely problematic--should not be interpreted as improvements in children's health. I want people to understand this problem, what it is and what it isn't, and to understand that numbers do not always tell a real story.
No comments:
Post a Comment