After months of online searches, poring over academic literature, and talking to experts to investigate industry ties to scientific research, The Examination food beat reporter Sasha Chavkin set out to uncover relationships between academics and companies a different way: by eyeballing it.

This was made possible by reporting partners at Columbia Journalism School’s Tow Center for Digital Journalism, who created a visualization of thousands of rows of data the team had collected on conflicts of interest.

The network graph resembled a mass of connected spheres, much like a giant molecule of atoms with many bonds. At the center of each ring was the name of a researcher who had declared a financial tie to the energy drink giant Red Bull. Chavkin already knew most of these names through his reporting — except one.

Jonathan Howland, a professor emeritus at Boston University, had written a commentary for a leading medical journal about the risks of mixing alcohol and energy drinks. In the conflict of interest statement, Howland disclosed a Red Bull-funded student survey that was not related to the preparation of the article. Chavkin had only been looking at studies that disclosed funding from Red Bull or other industry groups. What had become of the Red-Bull funded survey?

After a phone call, Chavkin learned that Howland never ended up publishing it. Though he was to be principal investigator on a contract between Red Bull and Boston Medical Center worth over a hundred thousand dollars, Howland withdrew from it because of his concerns that the methodology would produce biased results.

The Tow Center’s exploratory data tool had enabled The Examination to identify a researcher who would become a key source in our investigation into Red Bull’s financial ties to research on energy drinks.

It also laid the groundwork for an important finding: Studies financed by Red Bull or conducted by researchers who had disclosed financial ties with the company almost unanimously came to the same conclusion: Consuming energy drinks and alcohol together is no more harmful than drinking alcohol alone.

This stands in stark contrast to most independent research on the topic, which reached the opposite conclusion.

Here’s how we did the analysis, step by step.

Extracting conflicts of interest

The Examination partnered with the Tow Center to map connections between organizations, researchers and their studies, analyzing conflict of interest disclosures, funding statements and affiliations across research.

To explore these patterns, we used PubMed, a database of health-related scholarly literature that contains more than 40 million summaries of academic journal articles. The Tow Center used different keywords to download an initial exploratory set of abstracts from PubMed about energy drink research. But extracting each author's statements from conflict of interest sections was not easy. There was no uniform structure, and authors were often listed only by initials or surnames.

To turn this information into data that we could analyze, the Tow Center used a large language model (LLM) to identify the relationships between author names and organizations. To verify and improve the accuracy, we compared these results with those of a prior project by The Examination, in which we manually tagged conflicts of interest for authors across hundreds of articles in PubMed.

Once we were confident that this system was accurate, we looked for reporting leads. The Tow Center created a network graph, which enabled reporters to explore relationships.

Building our dataset

From there, The Examination focused on our particular reporting area: studies about the human health and behavioral risks of mixing alcohol and energy drinks. We wanted to see whether there was a difference in conclusions between studies that had ties to the energy drink industry and those that didn’t.

We decided to focus on the conclusions of studies because they provide a good window for examining potential industry influence.

To ensure we had a comprehensive look at all energy drink-related studies from PubMed, we downloaded a new version of the energy drink dataset. This time, we collected summaries of articles from PubMed that mentioned “energy drinks” or “energy drink” in the title and abstracts and that were categorized under the “energy drink” tag (a label used by medical libraries to group articles by their specific subject). 

When we noticed some summaries were missing conflict of interest statements, we wrote a script to check if those disclosures were included in full articles available on PubMed Central. We retrieved the missing information using a list of more than 30 phrases commonly used in conflict of interest or funding statements.

Our dataset had more than 2,300 publications on energy drinks.

From here, we narrowed our studies to those focused on alcohol mixed with energy drinks, based on the focus of Chavkin’s reporting. We searched for the phrase “alcohol mixed with energy drinks,” the abbreviation “AMED,” and combinations of words and phrases that researchers commonly used. 

We identified more than 150 publications about alcohol mixed with energy drinks. We focused our analysis on journal articles, so we removed publications such as editorials, letters and comments, as well as those that did not address health or behavioral risks.

The final corpus for the analysis had more than 100 journal articles. 

Identifying financial links and assessing conclusions favorable to companies

The next task was to identify which studies had ties to the energy drink industry.

Using market reports, The Examination made a list of energy drink companies, product brands and industry groups. Based on that list and the organizations named in the affiliation, funding, grants and conflict of interest statements, we sorted publications about alcohol mixed with energy drinks into two groups — those with industry ties and those without.

We classified a study as “industry-tied” if it mentioned grants or other funding from energy drink companies or related industry groups, or if any of the researchers declared financial ties with the industry, such as consultancies and research grants.

We randomly checked the categorization manually to make sure it was accurate.

All of the “industry-tied” studies from our dataset had financial links to Red Bull. 

We then defined a criteria to assess the articles. If an article’s conclusion stated that mixing alcohol and energy drinks did not increase health or behavioral risks, we labeled it as favorable. If the conclusion was the opposite, we labeled it as unfavorable. And if it was unclear or mixed, we labeled the article unknown.

Two people from The Examination's data team independently read each study’s abstract, conclusions, and, when available, the results and discussion sections. A third reviewer checked discrepancies between the two reviews. Experts reviewed our methodology, including Lisa Bero, a member of The Examination’s Scientific Advisory Committee, who also advised on assessments for a few of the most challenging cases. 

Of the studies that received funding by Red Bull or were conducted by researchers who disclosed financial conflicts of interest with the company, 95% concluded that mixing alcohol with energy drinks did not increase those risks. Roughly 80% of independent studies reached the opposite conclusion, we found.

Red Bull executives did not answer questions about our findings. Instead, Karlheinz Niederreiter, Global Head of Science, Regulatory and Government Affairs at Red Bull, provided a brief statement: “Health authorities across the world have concluded that Red Bull Energy Drink is safe to consume.” 

Limitations

Our conclusion analysis is based on positive disclosures, when funding, grants, conflict of interest statements and their variants were proactively reported in articles in our dataset. 

Not all publications include conflict of interest disclosures. Sometimes that field is missing or blank; other times authors simply fail to report a relationship.

Disclosure periods also can vary across journals. While many journals have modelled policies based on guidelines developed by the International Committee of Medical Journal Editors, these standards are difficult to enforce. Journals often treat disclosures as voluntary, and there isn’t a standard definition for what constitutes a conflict. Ultimately, there is no systematic way to determine when conflicts go undisclosed. 

Finally, the analysis is based on entries found in PubMed because it’s a leading biomedical research database, but articles on energy drink risks may exist in other platforms