Conflicts of interest in nutritional sciences: The forgotten bias in meta-analysis

Abstract

Awareness of conflicts of interest (COI) in medicine began in the 1980s. More recently, the problem has gained notoriety in nutritional sciences. COI with industry could bias study conclusions in the context of research activities and scientific publications on nutritional sciences. The issue of COI in nutritional sciences deserves more attention and requires careful analyses as biased information can negatively impact the development of dietary guidelines and, ultimately, population health. Decision-making is generally based on available, published evidence, but when the results are ambivalent, it is easier to opt for the status quo and ask for more studies. Readers might wonder if research is subsidized by industry as a counterbalancing strategy based on levels of evidence-only to slow down eminent positions and/or legislation on the food sector? How can this problem be overcome without producing paranoia and McCarthyism while trying to be as methodological as possible?

Key Words: Conflicts of interest, Nutrition, Nutritional sciences, Bias, Systematic reviews, Meta-analysis

Core tip: Decision-making in the field of nutrition is based on published evidence, but when results are ambivalent, it is easier to opt for the status quo and ask for more studies. Because conflicts of interest (COI) in nutritional sciences can bias conclusions and negatively impact dietary recommendations and population health, it deserves more attention and requires careful analyses. To regard evidence properly and in a rigorous manner, COI in systematic reviews and meta-analyses must be evaluated systematically to guarantee the trustworthiness of nutrition-related studies, and must therefore be obligatory sub-analyses to reduce the risk of bias in data interpretation.

INTRODUCTION

Thompson[1] defined conflicts of interest (COI) as “a set of conditions in which professional judgment concerning a primary interest (such as a patient’s welfare or the validity of research) tends to be unduly influenced by a secondary interest (such as financial gain)”. To researchers, financial incentive is not necessarily the only influent interest. The desire for recognition, academic advancement, success in publication and funding are other powerful stimulants[2]. Scientists delight in believing they are immune to and very mindful of COI. According to Young[3]: “We are not always aware of our own biases. The idea that scientists are objective seekers of truth is a pleasing fiction, but counterproductive in so far as it can lessen vigilance against bias.” The purpose of this editorial is to discuss justifiable concern about the COI problem in nutritional sciences and the need to seriously take it into account with critical reading of scientific journals and inclusion of its analysis in systematic reviews (SR) as well as meta-analyses (MA). The nature of COI and human behavior relevant to COI are beyond the scope of this editorial, as they have already been well-described by other authors[3,4].

Even if we cannot accurately ascertain the beginnings of COI investigation in medicine, its consideration intensified in the 1980’s[5,6] and it still continues[7]. In 2009, the Institute of Medicine dedicated a full report to COI, indicating that its concerns are justifiable[8]. Corruption of healthcare by Big Pharma is a long-standing debate, but the one on Big Food is much more recent[9]. Indeed, PloS Med and BMJ recently published a complete series on the food and beverage industries, their influence and COI[10,11]. To Loder[10], the industry-researcher relationship “is not evidence of research malpractice. It does, however, contribute to perceptions that nutrition science might be for sale.” However, nutritional sciences are not the exclusive domain of COI. Such biases are well-known in the tobacco[12] and pharmaceutical industries[2], and parallels also apply to the food industry[13].

Although most scientific journals instruct authors to report all COI, not all published studies declare them. Lesser et al[14] noted that 54% of scientific articles - relating to drinks (beverages, juice, and milk) and published between 1999 and 2003 - named their financial sponsorships. They assessed the influence of funding bias by determining the relationship between industry (sponsorship) funding (yes, no, mixed support) and the conclusions of scientific articles (favorable, unfavorable or neutral). They reported an odds ratio of 7.61 (95%CI: 1.27-45.73) for favorable vs unfavorable conclusions in all industry-funded articles compared to those without industry funding. They concluded that “industry funding of nutrition-related scientific articles may bias conclusions in favor of sponsors products, with potentially significant implications for public health”. Diels et al[15] scrutinized the relationship between COI and study outcomes (favorable/unfavorable) in the realm of genetically-modified food products, using similar methodology. They found that financial COI were not associated with the results, but discerned strong linkage between professional COI (author affiliation with industry) and study outcomes. Bes-Rastrollo et al[16] examined relationships between COI and food companies, conclusions on sugar-sweetened beverage (SSB) consumption, and weight gain in published SR. Among the 6 SR that identified “COI with food companies”, 83.3% (n = 5) reported no positive linkage between SSB intake and weight gain, whereas among the 12 SR that found “no COI with food companies”, 83.3% ascertained positive associations. They noted that studies with “COI with food companies” were 5 times (relative risk = 5.0; 95%CI: 1.3-19.3) more likely to present no positive association between SSB consumption and weight gain than those without COI. These contradictory findings do not, however, establish which SR is right, but they clearly indicate discrepancies, depending on whether or not COI exist.

Drug studies have advantages over those in nutrition-the results are first submitted to government agencies for scrutiny before drug approval. Therefore, data from unpublished drug research are available, but this is not the case in nutrition. Moreover, by accessing trials registered with the United States Food and Drug Administration[17], the European Medicines Agency[18] and other government bodies, it can be determined if they have been published or not, if more negative studies are unpublished, if published results agree or conflict with agency decisions, and if there is risk of publication bias. Although not perfect, different methods detect and correct for publication bias[19,20]. However, unless bias is severe, these tests have low power and high false-positive rates in perceiving significant asymmetry[19]. Therefore, no statistical methods are superior to any others in assessing publication bias and they should be viewed as exploratory analyses. Publication bias in SR and MA is related not only to published and unpublished studies, but also to factors which influence published studies (e.g., statistical significance, study size and quality, type of study design, etc.)[21]. COI are often forgotten factors.

To help in the critical analysis of published SR and MA, guidelines such as PRISMA[22] (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) should be followed. The publication of protocols for SR and MA in the Cochrane Library (http://community.cochrane.org/) or Prospero (http://www.crd.york.ac.uk/prospero/)[23] could ensure rigorous methodology. Although these tools are driven by methodological markers, they do not constitute an analysis of quality in themselves. Peer review committees and scientific panels do not always guarantee the quality of published studies. If included studies are of poor methodological quality, their results are likely to be biased. Therefore, the outcomes of SR and MA could be linked with high risk of bias, even if the methodology is implemented with great rigor. The Cochrane Statistical Methods Group and the Cochrane Bias Methods Group developed a risk of bias tool that focuses on 6 domains: Selection, performance, detection, attrition, reporting and other biases[24]. However, this risk of bias tool seems insufficient to capture biases related to study funding sources[25]. A Cochrane Review found that the risk of bias between drug industry and non-drug industry-funded studies was similar, but observed that drug industry-funded studies reported more “favorable” results than non-drug industry-funded studies[26].

As mentioned earlier, the problem of COI is complicated by the fact that not all studies sufficiently declare sponsorships and the financial affiliations of authors. Indeed, in the 2010 Cochrane Database of Systematic Reviews, a very low proportion of published drug studies reported funding sources (30%, 46/151 reviews), author-industry financial ties or employment (11%, 16/151 reviews)[27]. These findings are overwhelming, considering that Methodological Expectations of Cochrane Intervention Reviews Item No. C44 (Describing Studies) is mandatory[28]. Funding sources and COI declarations by primary researchers should be collected in this process and appear in the table on “Characteristics of included studies”.

As suggested by Bero[25]: “The impact of the bias can be assessed descriptively or by using subgroup analysis, comparing industry-funded to non-industry-funded studies, as is commonly done in Cochrane Reviews. A bias should not be ignored even if we do not fully understand its mechanism, just as we should not ignore harms of interventions if we don’t understand how they arose, or ignore the harm of smoking because we don’t know how smoking causes cancer. Therefore, a study’s funding source should be evaluated as an independent risk of bias.” As pointed out by Rothman[29], there is a risk of McCarthyism with labeling of scientists as having COI. He recommended “that a work should be judged solely on its merits” and “We can halt this new McCarthyism in science and get back to focusing on the work of a scientist rather than on his or her life story”.

To regard evidence properly and in a rigorous manner, COI in SR and MA must be evaluated systematically to guarantee the trustworthiness of nutrition-related studies, and must therefore be obligatory sub-analyses to reduce the risk of bias in data interpretation. COI may influence the results not only by showing statistically significant associations between exposure and disease, but also by demonstrating lack of associations, especially among groups that protect interests by inducing doubts and claiming unproven causation. Surprisingly and unfortunately, none of the MA in nutritional sciences assessed COI - to the best of the author’s knowledge while writing this editorial! Therefore, SR and MA must include sub-analyses that try to examine if studies with COI: (1) industry-sponsored; (2) authors-industry-affiliated; and (3) sponsorship or author affiliation - have more favorable outcomes (results, conclusions) than other investigations. The methodology employed by Diels et al[15] for SR and by Lundh et al[26] for MA could establish definitions, such as: (1) Sponsorships: industry-funded, non-industry-funded, and unknown/unclear sponsorship; (2) Authors’ affiliations: Industry-affiliated, non-industry-affiliated, and unknown/unclear; and (3) Classifications that combine sponsorship and author affiliations: COI (sponsorship or author affiliations), no COI, and unknown/unclear.

Nutrition is one of the most vital health determinants of society, not only in regard to the etiology of chronic diseases, but also because it is an important target for public health interventions. Investment in epidemiological approaches - allowing rigorous study into the roles of both individual and overall diets in disease risk - is undoubtedly a key to success. However, to continue to attract interest and trustworthiness, nutritional sciences must be faultless.