Examine publishes rigorous, unbiased analysis of the latest and most important nutrition and supplementation studies each month, available to all Examine Members. Click here to learn more or log in.

Quick Navigation

Issue #44 (June 2018)

From the Editor

Volume 1

Big news this month — a landmark study examining the effects of a Mediterranean diet, the Prevención con Dieta Mediterránea (PREDIMED) trial, has been retracted, reanalyzed, and republished.

The story started last year, when a researcher looked at around 5,000 randomized controlled trials (RCTs) to see if the baseline characteristics of the populations in those trials would be what you’d expect if they were actually randomized. The researcher found that many didn’t look well-randomized. Eleven of these were published in the New England Journal of Medicine (NEJM), which prompted its editors to take a closer look at these trials. They found that study authors confusing standard deviation with standard error explained the anomaly for five of the articles, and that another five could be explained by limitations in the 2017 analysis, like not accounting for baseline variables being correlated. This left only one NEJM article unaccounted for: PREDIMED.

The NEJM prompted the PREDIMED researchers took a closer look at their data. When they did, they found a few problems with their randomization process. One of these problems was known to the authors already, but they didn’t report it in their original writeup — people from the same household were assigned to the same diet. This was done intentionally so that people living together didn’t have to go on different diets, but could lead to problems because people living together can share other aspects of their environment or genetics. This makes it harder to conclude that any effects seen are actually caused by their assigned diet, as opposed to some other shared environmental variable. The authors also discovered poor randomization at other sites that they weren’t aware of previously.

In short: the 2017 analysis was correct: PREDIMED had problems with randomization, affecting the data of over 10% of its participants.

This led the PREDIMED authors to retract the original study and reanalyze it, taking into account the randomization issues and also exploring the data more to see how robust it is to different assumptions and possible problems. The overall results stayed roughly the same.

What does all this mean for interpreting PREDIMED? My opinion is: not much. There are other, stronger reasons to take its results with a grain of salt that aren’t really affected by the randomization problem, like issues with control group diet adherence. On the other hand, if you think PREDIMED is strong evidence favoring the Mediterranean diet, the re-analysis probably won’t affect your take much, either.

If you have any thoughts about PREDIMED, I’d be interested to hear them over at the NERD Facebook forum!

Gregory Lopez, MA, PharmD
Editor-in-chief, Nutrition Examination Research Digest

Volume 2

If a study has a tiny sample size, measures surrogates instead of clinical outcomes that people actually care about, under conditions that don’t really match the real world, does that make it a crap study? Absolutely not — it depends on why the study was done in the first place, and what conclusions the authors are drawing from it.

This volume of the NERD covers such a study, which examines the effects of early time-restricted feeding. “Time-restricted feeding” is a concept that, if you’re not already familiar with, is pretty self-explanatory — it’s a diet where you eat only during a small part of the day. Most Americans eat over a 12 to 15 hour period each day; time-restricted feeding (TRF) shrinks this window. The “early” part refers to restricting that limited eating window to early in the day.

Why test this specific diet? Well, animal studies suggest that TRF can be beneficial even if the animals don’t lose weight. But the limited data looking at how this works in humans is mixed; some studies showed a benefit, and some didn’t. The authors of the study we cover here noticed that there seemed to be worse outcomes if eating was restricted to later in the day. Since metabolism is influenced by our biological clock, the authors suspected that moving the TRF window to early in the day would be beneficial, even in the absence of weight loss. That’s the specific hypothesis they set out to test.

The thing is, to properly test a novel hypothesis, a lot of confounding variables need to be controlled. Compliance should be high, and food intake needs to be carefully measured so that bodyweight remains pretty stable. This becomes very hard as sample size increase, and also becomes very expensive. Using gold-standard measurements instead of surrogate markers would also add to the cost. So, in this case, it makes a lot of sense to run a smaller proof-of-concept study. That way, if the hypothesis doesn’t pan out, you haven’t blown through your budget. And if it does pan out, then you have data justifying further research (and further expense).

So, given the authors’ intentions, the study design makes a lot of sense. I’ve read other studies (often funded by groups with a vested interest in positive outcomes) that have small sample sizes and test a lot of surrogate markers whose conclusions I don’t put much faith in. That’s because the research seemed designed to get a positive outcome, not to honestly test a novel hypothesis.

However, this early TRF paper seems to me to fall squarely in the latter camp; it sets out to see whether, under ideal conditions, early TRF has the effects they suspect, and justifies further research to see whether it can work practically. As such, it serves as an interesting first step in the scientific exploration of how much timing matters when it comes to time-restricted feeding.

Gregory Lopez, MA, PharmD
Editor-in-chief, Nutrition Examination Research Digest

See other articles in Issue #44 (June 2018) of Study Deep Dives.