Examine publishes rigorous, unbiased analysis of the latest and most important nutrition and supplementation studies each month, available to all Examine Members. Click here to learn more or log in.

Quick Navigation

Issue #47 (September 2018)

From the Editor

Volume 1

Becoming more “healthy” can actually mean a few different things. Sometimes it means improving performance, like being able to do a half marathon after conquering a 5k. Or it could mean getting rid of, or at least mitigating, an already-existing problem, like pain. It could also mean preventing those problems from occurring in the first place. These are all pretty different forms of health, but they have one theme in common: improving things people directly care about.

Yet a lot of outcomes that are measured in studies don’t look at the things people actually care about, at least not directly. They instead examine interventions’ effects on surrogate markers — things like blood pressure, blood sugar, or signs of muscle protein synthesis. People don’t care about these things for their own sake. Instead, we care about them because they are (hopefully!) strongly linked with outcomes we care about, like not having a heart attack, not getting nerve damage, and getting stronger.

In this volume of the NERD, we cover a novel study taking a look at a specific probiotic’s effect on bone mineral density (BMD), which is, at the end of the day, a surrogate for what people probably care about — not fracturing bones easily. However, BMD doesn’t tell the whole story, even though it’s hard to conceive of what osteoporosis would be without the concept. That may be in part due to the World Health Organization's 1994 definition of osteoporosis in terms of T-score, which is statistically based. The T-score essentially looks at the distribution of BMDs for healthy young people (at the peak of their lifetime BMD), which fall along a bell curve. If someone (regardless of their age) gets their BMD tested and it falls in the lower 0.6% of that curve made up of healthy young people’s BMDs, then they are defined to have osteoporosis.

The T-score does have some correlation to fracture risk, which is the thing people actually care about, but it doesn’t tell the whole story. Research since then has found that the same T-score can reflect different fracture risks depending on age. For instance, a woman aged 75 years with a T-score of -2.5 can have several times the fracture risk of a 50-year-old woman with the exact same T-score! That, coupled with the finding that the majority of fractures actually occur in people with non-osteoporotic T-scores (like those from the probiotic study we review in this volume) led WHO to create an solid tool to predict 10-year fracture risk. BMD can play a role in assessing risk in this tool, but it doesn’t have to. Fracture risk assessments have since become an important part of fracture prevention guidelines in the U.K., Canada, and the U.S.

This shift from focusing on single surrogate markers to overall risk has occurred for other things like managing cholesterol. I hope to see more shifts to risk assessment in the future, since risk is focusing on outcomes that actually matter to people.

Gregory Lopez, MA, PharmD
Editor-in-chief, Nutrition Examination Research Digest

Volume 2

There’s a lot to take into account when reading research. In order to get the full picture, you have to take a look at whether the motivation for the research is justified given the state of knowledge in the field, the study design and how it compares to preregistration, the choice of control groups, the statistical analysis, and much, much more. But even if a study is well-executed and analyzed, it can be hard to make any confident statements about the research’s implications if key facts aren’t reported. In this volume of the NERD, we cover one study which pits fruits and veggies against whole grains to see what kind of impact they make on subclinical inflammation and the gut microbiome compared to refined grains. While the research is interesting and the study was the first of its kind, the conclusions we can draw from this paper are unfortunately limited, in part due to the way it’s reported.

One of the strengths of randomized controlled trials (RCTs) lies in the comparison between the treatment group and the control group. This comparison is very important, since without a control arm, you can’t tell whether changes in the treatment arm are due to the treatment itself, or other factors. After all, some conditions change on their own over time, for better or for worse. If you randomize people to treatment or control, you’d expect these changes to be roughly equal — this takes random drift out of the equation. Also, even in trials that are as close to real life as possible, other aspects of the trial design could impact the outcomes. For instance, people could change their usual diets to be more healthy, even if instructed not to, since they’re under the watchful eye of clinicians. Since factors like this should be similar for both the treatment and control groups, they should affect both equally as long as participants are well-randomized. But if the control group wasn’t there, we couldn’t rule out such factors influencing the results.

The fruits and veggies versus whole grain study we report in this volume randomized people to whole grains, fruits and veggies, or refined grains, which served as the control group. Unfortunately, they didn’t report the statistics that were necessary to know whether or not there was a difference between groups, which is odd since they do report that they perform the tests necessary to compare groups. The information is simply not present. Instead, they report only the statistics regarding how each group changed from baseline. The reasons for this aren’t clear. Perhaps they thought that if they report changes from baseline, it’s obvious that there were a change between groups. Or, perhaps there were no differences between groups at all. We just can’t tell from what’s written.

We’ve reached out to the authors of the study concerning the issue, and I’ll let you know what we hear. The take-home lesson here is that reporting results matters just as much as study design and analysis. When you read research papers, it’s necessary to not just look at the results, but also the process that generates them.

Gregory Lopez, MA, PharmD
Editor-in-chief, Nutrition Examination Research Digest

See other articles in Issue #47 (September 2018) of Study Deep Dives.