Fate of meta-analyses: The case of Helicobacter pylori

Abstract

AIM: To overview the current diversity of meta-analysis and the implementation of their results in international guidelines.

METHODS: Relevant meta-analysis were identified from PubMed/Medline. The topics of meta-analyses were determined. Some topics (genetics, extragastric tumors) were analysed separately. Core journals publishing meta-analyses on Helicobacter pylori were ranked. The rate of citation of meta-analysis in major guidelines was calculated.

RESULTS: Between 1992 and 2014, some 356 meta-analyses were published on PubMed. These mainly appeared in core journals, but were also found in 128 other journals. Eradicating of the infection was the most addressed topic with 134 articles. Meta-analyses were rarely used in formulating statements and recommendations in the international guidelines. In other topics - genetics, extraintestinal manifestations - meta-analyses were rather overused.

CONCLUSION: The implementation of meta-analysis in current guidelines is rather rare, while other topics benefit from many studies. A more extensive use of meta-analyses in evidence-based medicine is recommended in the future, otherwise their continuous proliferation will lose reason and scientific significance.

Key Words: Consensus guidelines, Helicobacter pylori, Meta-analysis, Randomised controlled trials, Systematic review

Core tip: The article provides a subjective overview of the meta-analysis published on the subject of Helicobacter pylori, profiling the topic, their distribution in literature, giving examples of over- and underuse, and revealing a discordance between the low implementation of meta-analysis in guidelines and their importance as top-level evidence.

INTRODUCTION

The discovery of Helicobacter pylori (H. pylori) had a tremendous impact on the clinical practice, public health and basic research, leading to an unsurpassed proliferation of written and electronic literature[1]. Besides 35472 articles published in peer-reviewed journals (http://www.pubmed.com, accessed on September 5 and 30, 2014), dozens of printed and some e-books have been published in the past 30 years. The plethora of literature created confusion, as readers were faced with many contradictory results and statements. The general purpose of this subjective overview is to present the development from a historical viewpoint and the current state of meta-analysis in the field of H. pylori; specifically, to analyse the use and implementation of meta-analytical results in current international guidelines for diagnosing and treating of the infection.

Historical background

The history of meta-analysis differs according to the source: according to the anonymous writer of Wikipedia’s entry, the first meta-analysis was performed by Chinese philosopher Chu Hsi[2] (1130-1200) by simply summarising data from literature of his time. Scholars date the roots of meta-analysis back to the 17^th century, when Blaise Pascal (1623-1662) approached games of chance mathematically[3]. The first medical meta-analysis was published in 1904: Karl Pearson[4] (1857-1936) summarised data on the effect of enteric fever bacteria inoculation in volunteer soldiers across the British Empire and studied the association of infection, mortality and inoculation. Considerable progress was subsequently made by the works of Ronald Fisher (1890-1962) and Frank Yates (1902-1994), although they were active in the agricultural field. William Gemmel Cochran (1909-1980) stressed the need for randomised controlled trials and studied the results of the then in vogue vagotomy for curing peptic ulcer. In the modern era, the first meta-analysis was performed by Gene V Glass[5], a psychologist at the University of Colorado, in 1976. He also coined the term “meta-analysis”, which later gained several entries and definitions in dictionaries (Merriam-Webster’s, Dorland’s Medical Dictionary, A Dictionary of Epidemiology, etc.).

Meta-analysis is a rapidly evolving field of statistics and over the past 3 decades increasingly sophisticated methods have been developed: these are available in books[6,7], online courses are also accessible and included in statistical packages and software programmes. It became clear that robust meta-analytical data could only be obtained by using (1) a selection of high-quality trials; and (2) a complex statistical workup of the data, including an assessment of heterogeneity, effect sizes, random or fixed effects, subgroup analysis, meta-regression, publication bias, etc. Specific statistical methods were introduced from 2002, when Higgins et al[8] from Cambridge University elaborated methods to identify heterogeneity between studies. The QUORUM and PRISMA statements were proposed in 2006 and 2010 respectively, as a uniform reporting mode for meta-analysis: unfortunately, only a small number of authors report their results according to these statements[9]. Weak data leads to uncertain results and doubtful conclusions: mixing of good and bad studies is an early mistake and is increasingly avoided in recent studies; on the other hand, weak data is perhaps better than no data at all. For reasons unknown to the author, there are no mega-trials on H. pylori including thousands of patients as in the case of hypertension, diabetes or hyperlipidaemia treatment. Most of the studies on H. pylori included a rather small number of cases and under these circumstances, assessing heterogeneity and selecting adequate statistical methods are of pivotal importance. This was not always the case. In the meantime, other more sophisticated methods emerged, like network- and combinatorial meta-analysis: both are only starting to be used in H. pylori research.

It must be emphasised that meta-analyses are (1) retrospective; and (2) they include studies on populations with different ethnic and genetic backgrounds, mostly geographically remote from each other, and probably infected with different strains of H. pylori, resulting in a “mixed bug”. Therefore, meta-analysis do not rule out the need for local, well-designed, prospective and adequately sized controlled trials[6,9].

Systematic reviews are structured studies of a focused subject-H. pylori, in our case-aiming to synthesize the evidence from the literature based on the most relevant publications. They may or may not use statistics to combine the results of the selected studies (both full-length articles or abstracts). The PRISMA statement standardised the requirements the complete reporting requirements for systematic reviews[9]. In practice, meta-analysis and systematic reviews are often performed and reported together.

The fate of meta-analysis in H. pylori research

The first meta-analysis on H. pylori was published 10 years after the discovery of the bacterium: Chiba et al[10] from the McMaster University, Canada calculated the pooled eradication rates of single, double and triple therapies against H. pylori from 27 studies. In 1996, Scandinavian authors assessed the efficiency of omeprazole-based and bismuth-based triple therapies in the same way[11]. Obviously, these studies are no longer valid today because of the simplified methodology, and many other regimens against the infection have been proposed in the meantime[12].

MATERIALS AND METHODS

Using the MESH terms “Helicobacter pylori” AND “meta-analysis” AND “systematic review”, 504 articles were found in Medline/PubMed (accessed on September 5 and 30, 2014). After reviewing the abstracts, 148 were found to be irrelevant to our subject and 356 eligible meta-analyses/systematic reviews were identified. This is a fairly low compared to other fields (Table 1) (PubMed, accessed on September 30, 2014), but comparable with other gastrointestinal diseases. The articles were classified according to their topic and method of study (meta-analysis, systematic review or combined) and total percentages were calculated (Table 2).

Table 1 Number of meta-analyses published on selected topics (from PubMed, accessed on September 30, 2014).

Topic/field	No. of meta-analyses
Diabetes mellitus	3245
Hypertension	2964
Coronary heart disease	3159
Gastrointestinal cancer	2787
Statins	957
Hepatitis C	550
Liver cirrhosis	435
Proton pump inhibitors	356
Helicobacter pylori	356
Peptic ulcer	395
Gastroesophageal reflux	231

Table 2 Profile and No. of published meta-analyses and systematic reviews on Helicobacter pylori.

Topic	Total No. of publications	Meta-analyses	Systematic reviews	Meta-analysis + systematic review
Epidemiology	6	5	1	0
Diagnosis	23	13	4	6
Antibiotic resistance	7	5	2	0
Genetics	32	30	0	2
Eradication regimens	134	98	17	19
Extragastric manifestations	49	35	2	12
Probiotics	11	11	0	0
Peptic ulcer	9	7	1	1
Gastric cancer	16	12	0	4
Pathogenesis	22	21	1	1
Other cancers (oesophagus, colon, pancreas, liver, biliary, lung)	26	24	0	2
Children	8	7	0	1
Methodological issues	12	0	12	0
Traditional Chinese medicine	1	0	0	1
Total	356	268 (75.2%)	40 (11.3%)	48 (13.4%)

The spectrum of journals publishing meta-analyses and systematic reviews on H. pylori was also studied and a group of core journals was selected, defined arbitrarily as those publishing > 10 meta-analyses and/or systematic reviews (Table 3).

Table 3 Core journals publishing meta-analyses and systematic reviews on Helicobacter pylori.

Title	No. of meta-analyses	Impact factor (2013)
Alim Pharmacol Ther	45	5.478
Helicobacter	27	2.993
World J Gastroenterol	25	2.433
Am J Gastroenterol	21	9.131
Plos ONE	14	3.534
BMJ	11	16.378
Eur J Gastroenterol Hepatol	10	2.152
Total	153	Not applicable

The reference list of the main consensus meetings between 2007 and 2013 (Table 4) was searched for citations of meta-analysis and a similarity analysis was performed[13-19].

Table 4 Implementation of meta-analyses in international consensus guidelines.

Year	Consensus meeting	No. of ref.	No. of meta-analyses/systematic reviews cited
2007	Maastricht III consensus	99	10 (9.9%)
2007	Cervia II Working Group guideline	72	5 (6.5%)
2007	American College of Gastroenterology guideline	175	23 (13.1%)
2009	Second Asia-Pacific Consensus Guidelines	118	12 (10.1%)
2012	Maastricht-Florence 4 guideline	325	36 (11.0%)
2013	3rd Brazilian Consensus	216	25 (11.3%)
2013	Revised Korean consensus	208	19 (9.3%)
	Total	1223	130 (10.6%)

To assess the average citation rates, five meta-analyses published in core journals between 2006 and 2010 were randomly selected and their citation was searched on the Web of Science (accessed on September 4, 2014)[19-23]. The reference list of 5 randomly selected expert review articles from special issues on H. pylori, published with the 20^th anniversary of the World Journal of Gastroenterology was also analysed[24-28].

RESULTS

Our search identified 356 studies. Most of the authors (75%) preferred to use meta-analysis, the rest of the studies were either systematic reviews (11.3%) or a combination of the two methods (13.4%). This preference for meta-analysis was maintained in almost every one of the 14 topics (Table 2).

The topic addressed most often was that of eradication therapy: 134 (37.6%) papers analysed the efficiency of antimicrobial regimens against H. pylori, followed by the extraintestinal manifestations of the infection (49 studies, 13.7%) and genetics (32 articles, 8.9%). The association of the infection with tumours other than gastric cancer also elicited high interest with 26 studies (7.3%) (laryngeal cancer: 1, oesophageal: 8, pancreatic: 5, colon: 7, liver and biliary tract: 2, lung: 1). Although peptic ulcer disease is the most important complication of H. pylori infection, it merited only 9 studies (2.5%).

Of the 356 studies, 153 (42.97%) were published in 7 core journals (Table 3). The rest of the articles (203, 57.29%) were found in 128 journals, mostly publishing 1-2 meta-analyses on H. pylori. Top-ranked journals such as Gastroenterology, Gut and Lancet published a small number of studies on this topic (editorial policy? high rate of rejection?). Impact factors and the number of meta-analysis published were seemingly not related.

The citation rate of meta-analysis in recent reviews on H. pylori is also low, achieving a mean of only 7.5%/article. By contrast, the citation of meta-analysis in journals of gastroenterology published between 2006 and 2010 is fairly high (43-172, with a mean of 97 ± 28 citations).

DISCUSSION

Inclusion of meta-analysis in the consensus statements

In biomedical research, meta-analyses are considered the highest level of evidence. The importance of these studies was recently summarised by Gisbert[12] of Madrid, who performed 36 meta-analyses and systematic reviews with his team between 2003 and 2013, concluding that “meta-analysis provides a means of combining raw statistical data from all eligible primary studies addressing an identical question of interest to arrive at conclusions that are more precise and reliable than those presented in a single study.” By analysing all regimens against the bacterium historically, he stated that “meta-analysis has contributed in a relevant way to our understanding of the management of patients with H. pylori infection”.

It could be expected that their results would be included in the recommendations of expert panels. Surprisingly, meta-analysis and systematic reviews represents only 10.6% of the citations in international guidelines (Table 4), and 34% of the cited meta-analyses are identical (i.e., cited in ≥ 3 consensus materials). One can conclude, that meta-analyses are underused in formulating consensus statements. Experts probably prefer to express their opinion based on randomised controlled trials and basic science.

In some areas, meta-analysis seems to be overused (extraintestinal manifestations of H. pylori infection, its associations with extragastric cancers, genetics) resulting in little practical use. Their release in the medical press could be explained by publication pressure too. According to most consensus statements, however, eradication of the infection is only recommended in cases of iron deficiency anaemia and idiopathic thrombcytopenic purpura. Although genetics was studied extensively, genetic counselling and tests are neither available nor recommended in diseases associated with H. pylori infection.

The association of extragastric cancer with the infection is largely documented, but there is no recommendation to screen and treat the infection in high risk patients, as it is in first-degree relatives of gastric carcinoma patients. In all these cases, however, association does not mean causation, further studies are necessary to see if the associations are casual or causal.

In a random selection of recent expert reviews, meta-analyses are again barely cited[20-23] (Table 5), excepting the Spanish team, which is the most active in this field[29].

Table 5 Citation of meta-analyses in recent expert reviews.

Ref.	Year	Journal	No. ref.	No. and % of meta-analyses cited
[20]	2014	World J Gastroenterol	115	5 (4.3)
[21]	2014	World J Gastroenterol	137	8 (5.8)
[22]	2014	World J Gastroenterol	158	14 (8.8)
[23]	2014	World J Gastroenterol	69	1 (1.4)
[24]	2014	World J Gastroenterol	79	14 (17.7)
		Total	558	42 (7.5)

In contrast with this, meta-analyses are adequately cited generally speaking. The data suggests, that meta-analyses are as frequently cited as other clinical studies in the literature, but not in consensus materials, where they really should be (Table 6)[24-28]. The reason for this discordance is not known.

Table 6 Citation of randomly selected meta-analyses on Helicobacter pylori (Web of Science, accessed on September 4, 2014).

Ref.	Year	Journal	Total citations	Independent citations
[24]	2006	Aliment Pharmacol Ther	172	172
[25]	2009	Am J Gasteroenterol	127	127
[26]	2009	Helicobacter	84	84
[27]	2010	Am J Gastroenterol	62	62
[28]	2010	Am J Gastroenterol	43	43

In conclusion, meta-analysis represent the highest level of evidence in medical research and are themselves under continuous mathematical and statistical development. In the field of H. pylori research, 356 meta-analyses and systematic reviews or both were published between 1992 and 2014. Although these studies are widely cited in literature, their implementation in the national/international consensus quidelines is rather rare. Other topics, of less practical importance, benefit from many meta-analyses. In the future, a more extensive use of meta-analyses would be welcome, to maintain the scientific significance of the guidelines and statements: otherwise, they will proliferate simply as a result of publication pressure and will progressively loss their scientific significance.

ACKNOWLEDGMENTS

The English correction of the manuscript by Mr. Douglas Arnott (EDMF Language Services Ltd., Budapest), and the secretarial work and Web of Science search by Jolán Józan (Semmelweis University, Department of Physiology, Budapest, Hungary) is highly acknowledged.

COMMENTS

Background

Meta-analyses have come strongly to the fore in the past 3 decades and are considered the highest grade of evidence in medical research. Their further use and implementation in the guidelines and consensus statements is unknown.

Research frontiers

The article provides an analysis of the spectrum of meta-analysis published between 1992 and 2014 in the field of Helicobacter pylori (H. pylori) research, providing the distribution of topics, ranking of core journals publishing meta-analysis, giving examples of under- and overuse of meta-analysis in some areas. The author’s main conclusion is that meta-analysis are underused in the formulation of statements from recent international guidelines for diagnosing and treating the infection.

Applications

The article suggests that meta-analysis must be more widely read, used and cited, especially when experts formulate their opinions/recommendations for treating the H. pylori infection. On the other side, their overuse in some topics (genetics, extraintestinal manifestations) did not resulted any benefit.

Peer review

The manuscript “Fate of meta-analysis: The case of Helicobacter pylori” is very interesting and original in its contents.