Examine publishes rigorous, unbiased analysis of the latest and most important nutrition and supplementation studies each month, available to all Examine Members. Click here to learn more or log in.

Quick Navigation

Issue #68 (June 2020)

From the Editor

There are lots of good reasons for why we call the methods section the most important part of the study in our “How to read a scientific study” guide. In clinical trials, one of the important parts of the methods section is the inclusion and exclusion criteria. These are important because they determine the study population’s demographics, and the demographics are important because they determine who the study applies to. If the population in a study is too different from who you’re aiming to apply the study to, then it’s less likely that the study’s results will actually apply to that group. And if the population you care about is explicitly excluded when the study participants are being recruited, then it’s less likely the results described in the paper will apply to that excluded group.

Issue 65 covered research examining a multi-ingredient supplement’s effect on tinnitus. The researchers explicitly excluded people who had tinnitus resulting from a traumatic injury, sudden loud noises, or middle ear inflammation. People who were taking medications like certain antibiotics and chemotherapeutic agents that are known to induce tinnitus were also excluded. That means that a sizable portion of people who experience tinnitus weren’t included in the study, so the results of the study may not apply to them. However, an astute reader brought to our attention that we failed to mention this issue, which was an error on our part. We should have mentioned and emphasized these exclusion criteria in the original writeup. We’ve since gone back and edited the review to discuss some of the exclusion criteria, but we made a mistake in not mentioning them in the first place. We do make mistakes, but we also try to own up to them!

The methods section is just as important for meta-analyses as it is for clinical trials. While researchers conducting a trial make judgment calls about the demographics of the people to include in the study, a meta-analysis also requires judgment calls about what studies to include, in part through the search strategy used. Ideally, the search strategy is loose enough to capture all the articles the authors are interested in, but strict enough to avoid studies that have little relevance to the question at hand.

In this issue, we cover a meta-analysis in which paying close attention to the methods section was pretty important. The question the authors of the meta-analysis seemed to be interested in concerned the vascular effects of coffee intake. However, a look at the authors’ search strategy, along with the studies they included in their meta-analysis, shows that they didn’t just include coffee, but also components of coffee, like pure caffeine. Also, relatively few of the studies actually compared coffee intake to no coffee intake, raising the question of how relevant to studies were to the question the authors were trying to address. Including such a wide range of studies led to a highly heterogeneous set of studies that don’t really address the question at hand and whose results probably shouldn’t have been combined using a meta-analysis in the first place.

While the methods section can be a dry read, it pays dividends to dig into the details.

Gregory Lopez, MA, PharmD
Editor-in-chief, Nutrition Examination Research Digest

See other articles in Issue #68 (June 2020) of Study Deep Dives.