|
1
|
Cano BHS, da Silva GFMV, Bottari GD, Balbinot EL, Uggioni MLR, Roever L, da Rosa MI, Grande AJ. Evaluation of intervention systematic reviews on chronic non-communicable diseases and lifestyle risk factors in low-middle income countries: meta-research. BMC Med Res Methodol 2025; 25:90. [PMID: 40188041 PMCID: PMC11972496 DOI: 10.1186/s12874-025-02501-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2024] [Accepted: 02/10/2025] [Indexed: 04/07/2025] Open
Abstract
BACKGROUND Systematic Reviews (SRs) rigorously synthesize findings on a theme, but some articles with this design are redundant due to errors and conflicts. Meta-research aims to rigorously analyze research, assessing SRs' methodological quality and result reliability. This study evaluates SRs' overall quality in low- and middle-income countries (LMICs) on chronic non-communicable Diseases (NCDs) and key modifiable risk factors, using assessment tools. METHODS A search strategy was conducted in the following databases: MEDLINE (via PubMed), Embase, (via Elsevier), Cochrane Library, and Grey Literature for published studies from January 1, 2014 - April 5, 2024. SRs addressing the association between at least one of the four most important modifiable behavioral risk factors (tobacco use, inadequate diet, alcohol consumption, and physical inactivity) and chronic NCDs in populations classified as LMICs according to the 'World Bank list of countries' were included. The selected studies were imported into the EndNote 20 software and analyzed using a form for the extraction of their main data and four tools were chosen to assess each of the most important domains of scientific evidence: Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) for article writing; Template for Intervention Description and Replication (TIDieR) for intervention description; A Measurement Tool for Evaluating Systematic Reviews (AMSTAR-2) for methodological assessment; and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) for certainty of evidence. RESULTS Nine studies were included in this analysis. The average Overall Score on the PRISMA 2020 checklist was 13.5 for articles published before 2020 and 25.67 for those published after 2020. TIDieR analysis revealed complete correspondence (100%) for item 'Brief Name', while other items, like 'Why' (89%), and 'What', 'Who Provided', and 'How' (78%), were partially met but significantly so. Regarding AMSTAR-2 criteria, only one study fulfilled all critical items, meeting item 7 by providing a detailed list of excluded studies and justifying each exclusion motive. Additionally, among critical items applicable to multiple articles, only item 11 was consistently fulfilled by all studies. In the final classification, one article achieved a moderate quality rating, three were critically low quality, and five had low quality among the nine evaluated articles. In the GRADE tool evaluation, limitations resulted in estimations for only 19 outcomes and 8 intervention-exposure sets. CONCLUSION The results demonstrated that the writing of recent scientific articles meets most of the PRISMA 2020 criteria, with a checklist being the most used tool. Interventions and exposure were also very well reported, with the TIDieR checklist not being cited in any study as a guiding tool. AMSTAR-2 revealed a methodological approach of varied quality, mainly low and critically low. The GRADE approach classified the certainty of the evidence as generally very low. Therefore, it is necessary to encourage adherence to these approaches to improve the methodological quality in SR studies on chronic NCDs and behavioral factors in LMICs.
Collapse
Affiliation(s)
| | | | - Gustavo Dias Bottari
- Laboratory of Evidence-based Practice, State University of Mato Grosso do Sul, Campo Grande, MS, Brazil
| | - Eduarda Letícia Balbinot
- Laboratory of Translational Biomedicine, Postgraduate Program in Health Sciences, University of ExtremoSulCatarinense, Criciúma, SC, Brazil
| | - Maria Laura Rodrigues Uggioni
- Laboratory of Translational Biomedicine, Postgraduate Program in Health Sciences, University of ExtremoSulCatarinense, Criciúma, SC, Brazil
| | - Leonardo Roever
- Department of Clinical Research, Brazilian Evidence -Based Health Network, Uberlândia, Brazil
- Gilbert and Rose -Marie Chagoury School of Medicine, Lebanese American University, Beirut, Lebanon
| | - Maria Inês da Rosa
- Laboratory of Translational Biomedicine, Postgraduate Program in Health Sciences, University of ExtremoSulCatarinense, Criciúma, SC, Brazil
| | - Antonio José Grande
- Laboratory of Evidence-based Practice, State University of Mato Grosso do Sul, Campo Grande, MS, Brazil.
- Post-graduate Program in Infectious Disease and Parasites, Federal University of Mato Grosso do Sul, Campo Grande, MS, Brazil.
| |
Collapse
|
|
2
|
Warmenhoven J, Menaspà P, Borg DN, Vazire S, White N, Sainani K, Nimphius S, Coutts AJ, Impellizzeri FM. Sports Metaresearch: An Emerging Discipline of Sport Science and Medicine. Sports Med 2025; 55:845-856. [PMID: 40169510 PMCID: PMC12011897 DOI: 10.1007/s40279-025-02181-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/24/2025] [Indexed: 04/03/2025]
Abstract
Inadequacies in the conduct and quality of research are well established across many research domains, including sport science and medicine. Metaresearch-the practice of performing research on research-is presented as a practical vehicle for improving research quality through evaluating the research processes. This article introduces the concept of metaresearch to sport as a new sub-field of sport science. The broad types of metaresearch are introduced, with a mapping of current sports metaresearch activity across these areas. Interdisciplinary centres aimed at improving scientific quality across other fields are also introduced to sport, and specific considerations for beginning metaresearch are provided for sport. This includes, for example, not performing metaresearch poorly, beginning evaluative metaresearch early to intervene before bad practice becomes normalised, leveraging required interdisciplinary expertise depending on the metaresearch question and undertaking an ethical approach for carrying out evaluation of research quality.
Collapse
Affiliation(s)
- John Warmenhoven
- School of Sport, Exercise and Rehabilitation, Faculty of Health & University of Technology Sydney (UTS), Human Performance Research Centre, University of Technology Sydney (UTS), Moore Park, NSW, Australia.
| | - Paolo Menaspà
- Australian Sports Commission, Australian Institute of Sport, Bruce, ACT, Australia
| | - David N Borg
- School of Exercise Nutrition Sciences, Queensland University of Technology, Brisbane, QLD, Australia
| | - Simine Vazire
- Melbourne School of Psychological Sciences, University of Melbourne, Parkville, VIC, Australia
| | - Nicole White
- Australian Centre for Health Services Innovation, School of Public Health and Social Work, Queensland University of Technology, Brisbane, Australia
| | - Kristin Sainani
- Division of Epidemiology, Department of Health Research and Policy, Stanford University, Stanford, USA
| | - Sophia Nimphius
- School of Medical and Health Science, Edith Cowan University, Joondalup, WA, Australia
| | - Aaron J Coutts
- School of Sport, Exercise and Rehabilitation, Faculty of Health & University of Technology Sydney (UTS), Human Performance Research Centre, University of Technology Sydney (UTS), Moore Park, NSW, Australia
| | - Franco M Impellizzeri
- School of Sport, Exercise and Rehabilitation, Faculty of Health & University of Technology Sydney (UTS), Human Performance Research Centre, University of Technology Sydney (UTS), Moore Park, NSW, Australia
| |
Collapse
|
|
3
|
Duranté EK, Ribeiro A, Gaspard-Boulinc L, Boutron I, Henry C, Petit AC, Houenou J, Lemogne C, Chevance A. Biological research on mental pain, social pain and other pains not primarily felt in the body: methodological systematic review. Br J Psychiatry 2025:1-11. [PMID: 40116276 DOI: 10.1192/bjp.2024.292] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/23/2025]
Abstract
BACKGROUND Researchers explore the biology of painful experiences not primarily felt in the body ('non-physical pain'), sometimes referred to as mental, social or emotional pain. A critical challenge lies in how to operationalise this subjective experience for biological research, a crucial process for translating findings into clinical practice. AIMS To map studies investigating biological features of non-physical pain, focusing on their conceptual features (i.e. terms and definitions of non-physical pain) and methodological characteristics (e.g. experimental paradigms and measures). METHOD This methodological systematic review searched reports of primary research on the biological features of non-physical pain across Embase, MEDLINE and Web of Science. Using a meta-research approach, we synthetised results on terms, definitions, populations, experimental paradigms, confounders, measures of non-physical pain and investigation methods (e.g. functional magnetic resonance imaging). RESULTS We identified 92 human studies, involving 7778 participants. Overall, 59.1% of the studies did not report any definition of non-physical pain, and 82% of studies did not use a specific measure. Regarding the possibility of translating results to clinical settings, most of the human studies involved only healthy participants (71.7%) and the seven different experimental paradigms used to induce non-physical pain had unknown external validity. Confounders were not considered by 32.4% of the experimental studies. Animal studies were rare, with only four rodent studies. CONCLUSIONS Biomedical studies of non-physical pain use heterogeneous concepts with unclear overlaps and methods with unknown external validity. As has been done for physical pain, priority actions include establishing an agreed definition and measurement of non-physical pain and developing experimental paradigms with good external validity.
Collapse
Affiliation(s)
- Etienne K Duranté
- Université de Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel Dieu, Paris, France
| | - Alexandre Ribeiro
- Université de Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel Dieu, Paris, France
| | | | - Isabelle Boutron
- Université de Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel Dieu, Paris, France
| | - Chantal Henry
- Department of Psychiatry, Service Hospitalo-Universitaire, GHU Paris Psychiatrie & Neurosciences, Paris, France
- Université Paris-Cité, Paris, France
| | - Anne-Cecile Petit
- Department of Psychiatry, Service Hospitalo-Universitaire, GHU Paris Psychiatrie & Neurosciences, Paris, France
- Université Paris-Cité, Paris, France
- Institut Pasteur, Université Paris Cité, Paris, France
| | - Josselin Houenou
- NeuroSpin, CEA, Université Paris-Saclay, Gif-sur Yvette, France
- APHP, CHU Mondor, DMU IMPACT, INSERM U955 Team 'Neuropsychiatrie Translationnelle', IMRB, Université Paris Est Créteil, Créteil, France
| | - Cedric Lemogne
- Université de Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Department of Psychiatry, AP-HP, Hôpital Hôtel Dieu, Paris, France
| | - Astrid Chevance
- Université de Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel Dieu, Paris, France
| |
Collapse
|
|
4
|
Lozada-Martinez ID, Neira-Rodado D, Martinez-Guevara D, Cruz-Soto HS, Sanchez-Echeverry MP, Liscano Y. Why is it important to implement meta-research in universities and institutes with medical research activities? Front Res Metr Anal 2025; 10:1497280. [PMID: 40177472 PMCID: PMC11962015 DOI: 10.3389/frma.2025.1497280] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2024] [Accepted: 02/25/2025] [Indexed: 04/05/2025] Open
Abstract
In recent years, there has been a growing concern over questionable practices and a lack of rigor in scientific activities, particularly in health and medical sciences. Universities and research institutes are key players in the development of science, technology, and innovation. Academic institutions, whose primary mission is to generate and disseminate knowledge, bear the responsibility in many parts of the world to act as consultants and guardians of scientific integrity in health research. Then, universities and research institutes must act as guardians of the research and technological development process, utilizing methodological and operational evaluation tools to validate the rigor and quality of medical research. Meta-research is defined as the research of research itself. Some of the most important specific objectives of meta-research include the assessment of research relevance, the evaluation of evidence validity, and the exploration of scientific integrity. A significant portion of evidence in the medical and health sciences literature has been found to be redundant, misleading, or inconsistent. Although this issue is of great importance in global health, discussions about practical and tangible solutions remain fragmented and limited. The aim of this manuscript is to highlight the significance of employing meta-research within universities and research institutes as a tool to monitor scientific rigor and promote responsible practices in medical research.
Collapse
Affiliation(s)
- Ivan David Lozada-Martinez
- Biomedical Scientometrics and Evidence-Based Research Unit, Department of Health Sciences, Universidad de la Costa, Barranquilla, Colombia
| | | | - Darly Martinez-Guevara
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali, Colombia
| | - Hary Salome Cruz-Soto
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali, Colombia
| | - Maria Paula Sanchez-Echeverry
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali, Colombia
| | - Yamil Liscano
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali, Colombia
| |
Collapse
|
|
5
|
Zhong J, Liu X, Lu J, Yang J, Zhang G, Mao S, Chen H, Yin Q, Cen Q, Jiang R, Song Y, Lu M, Chu J, Xing Y, Hu Y, Ding D, Ge X, Zhang H, Yao W. Overlooked and underpowered: a meta-research addressing sample size in radiomics prediction models for binary outcomes. Eur Radiol 2025; 35:1146-1156. [PMID: 39789271 PMCID: PMC11835977 DOI: 10.1007/s00330-024-11331-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Revised: 11/10/2024] [Accepted: 11/30/2024] [Indexed: 01/12/2025]
Abstract
OBJECTIVES To investigate how studies determine the sample size when developing radiomics prediction models for binary outcomes, and whether the sample size meets the estimates obtained by using established criteria. METHODS We identified radiomics studies that were published from 01 January 2023 to 31 December 2023 in seven leading peer-reviewed radiological journals. We reviewed the sample size justification methods, and actual sample size used. We calculated and compared the actual sample size used to the estimates obtained by using three established criteria proposed by Riley et al. We investigated which characteristics factors were associated with the sufficient sample size that meets the estimates obtained by using established criteria proposed by Riley et al. RESULTS: We included 116 studies. Eleven out of one hundred sixteen studies justified the sample size, in which 6/11 performed a priori sample size calculation. The median (first and third quartile, Q1, Q3) of the total sample size is 223 (130, 463), and those of sample size for training are 150 (90, 288). The median (Q1, Q3) difference between total sample size and minimum sample size according to established criteria are -100 (-216, 183), and those differences between total sample size and a more restrictive approach based on established criteria are -268 (-427, -157). The presence of external testing and the specialty of the topic were associated with sufficient sample size. CONCLUSION Radiomics studies are often designed without sample size justification, whose sample size may be too small to avoid overfitting. Sample size justification is encouraged when developing a radiomics model. KEY POINTS Question Sample size justification is critical to help minimize overfitting in developing a radiomics model, but is overlooked and underpowered in radiomics research. Findings Few of the radiomics models justified, calculated, or reported their sample size, and most of them did not meet the recent formal sample size criteria. Clinical relevance Radiomics models are often designed without sample size justification. Consequently, many models are too small to avoid overfitting. It should be encouraged to justify, perform, and report the considerations on sample size when developing radiomics models.
Collapse
Affiliation(s)
- Jingyu Zhong
- Laboratory of Key Technology and Materials in Minimally Invasive Spine Surgery, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
- Center for Spinal Minimally Invasive Research, Shanghai Jiao Tong University, Shanghai, China.
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Xianwei Liu
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Junjie Lu
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA
| | - Jiarui Yang
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Guangcheng Zhang
- Department of Orthopedics, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Shiqi Mao
- Department of Medical Oncology, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Haoda Chen
- Department of General Surgery, Pancreatic Disease Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Qian Yin
- Department of Pathology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Qingqing Cen
- Department of Dermatology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Run Jiang
- Department of Pharmacovigilance, SciClone Pharmaceuticals (Holdings) Ltd., Shanghai, China
| | - Yang Song
- MR Scientific Marketing, Siemens Healthineers Ltd., Shanghai, China
| | - Minda Lu
- MR Application, Siemens Healthineers Ltd., Shanghai, China
| | - Jingshen Chu
- Editorial Office of Journal of Diagnostics Concepts & Practice, Department of Science and Technology Development, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yue Xing
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yangfan Hu
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Defang Ding
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xiang Ge
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Huan Zhang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Weiwu Yao
- Laboratory of Key Technology and Materials in Minimally Invasive Spine Surgery, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
- Center for Spinal Minimally Invasive Research, Shanghai Jiao Tong University, Shanghai, China.
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| |
Collapse
|
|
6
|
Moraes MA, Arabe LB, Resende BL, Codo BC, Reis ALAL, Souza BR. The gold standard control groups in physiological and pharmacological research are not that shiny: Intraperitoneal saline injection and needle pricking affect prepubescent mice's behavior in a sex-specific manner. Horm Behav 2025; 169:105707. [PMID: 39965530 DOI: 10.1016/j.yhbeh.2025.105707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2024] [Revised: 01/12/2025] [Accepted: 02/12/2025] [Indexed: 02/20/2025]
Abstract
Study design and experimental tools are crucial for good quality science, and an essential part of it is the choice of control groups to best test the hypothesis. Two of the standard control groups in physiological and pharmacological research are needle pricking without substance injection (Sham) and/or vehicle injection (Saline). However, both needle pricking and saline injection can act as stressors, potentially influencing the analyzed outcome. This raises the question of whether the dependent variable remains unaffected by the stress induced by these procedures. Despite the significance of this issue, very few studies have investigated the behavioral effects of a single intraperitoneal (I.P.) Sham and/or single I.P. Saline injection in mice, and those that have used mostly adult males. In this study, we investigated if a single I.P. Sham and/or I.P. Saline injection affects female and male prepubertal (4-weeks-old) mice behavior. After Sham or Saline injection, we examined exploratory/motor behavior (open field test - OFT), anxiety-like behavior (elevated plus-maze - EPM), and behavioral despair/depressive-like behavior (forced swimming test - FST). We observed that both Sham prepubertal females and males showed behavioral alterations in OFT and EPM, and Saline males showed behavioral alterations in OFT and FST. On the other hand, prepubertal Saline females showed an increase in exploratory behavior, risk assessment/anxiety-like behavior, and behavioral despair/depressive-like behavior. Thus, our findings indicate that control procedures commonly used in physiological and pharmacological experimental designs affect the behavior of prepubescent mice, with more pronounced effects in females than in males. This study suggests considering Naïve animals together with Sham and/or Vehicle for a better and more honest interpretation of the data.
Collapse
Affiliation(s)
- Muiara Aparecida Moraes
- Laboratory of Neurodevelopment and Evolution - Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, 31270-901, Brazil
| | - Laila Blanc Arabe
- Laboratory of Neurodevelopment and Evolution - Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, 31270-901, Brazil
| | - Bruna Lopes Resende
- Laboratory of Neurodevelopment and Evolution - Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, 31270-901, Brazil
| | - Beatriz Campos Codo
- Laboratory of Neurodevelopment and Evolution - Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, 31270-901, Brazil
| | - Ana Luiza Araújo Lima Reis
- Laboratory of Neurodevelopment and Evolution - Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, 31270-901, Brazil
| | - Bruno Rezende Souza
- Laboratory of Neurodevelopment and Evolution - Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, 31270-901, Brazil.
| |
Collapse
|
|
7
|
Sati P, Chandola V, Chandra S, Trivedi VL, Purohit VK, Nautiyal MC. Global environmental change mediated response of wetland plants: Evidence from past decades. THE SCIENCE OF THE TOTAL ENVIRONMENT 2025; 966:178668. [PMID: 39904209 DOI: 10.1016/j.scitotenv.2025.178668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Revised: 01/25/2025] [Accepted: 01/27/2025] [Indexed: 02/06/2025]
Abstract
Wetland ecosystems are critically affected by global environmental changes, yet understanding the impact of these changes on wetland plants remains a challenge. This review article employs a comprehensive approach, including bibliographic analysis, utilization of various climate models for historical data retrieval, and extensive literature survey, to investigate the response of wetland plants to environmental shifts over the past decades. The analysis conducted in this study uncovers a multitude of climatic parameters that exhibit an influence on the dynamics of wetland vegetation. Results indicated a significant positive trend in atmospheric CO2 concentration, leading to increased water use efficiency in some plant species, particularly C3 plants. However, C4 plants did not show the same positive response. Nitrous oxide growth rate showed a weaker, less consistent trend than CO2, highlighting the need for further investigation into the complex factors influencing Nitrous oxide emissions from wetlands. Methane growth rate and global mean sea level demonstrated a strong positive linear trend. Ocean pH exhibited a statistically significant downward trend (acidification), while sea surface temperature showed a moderate but statistically significant upward trend. Glacier mass balance revealed a significant negative trend. Although some plants may benefit from increased CO2 initially, but the combined effects of rising sea levels, ocean acidification, and temperature changes pose substantial threats to the overall health and diversity of wetland plant life.
Collapse
Affiliation(s)
- Pallavi Sati
- High Altitude Plant Physiology Research Centre (HAPPRC), H.N.B. Garhwal University, Post Box: 14, Srinagar Garhwal 246174, Uttarakhand, India
| | - Vaishali Chandola
- High Altitude Plant Physiology Research Centre (HAPPRC), H.N.B. Garhwal University, Post Box: 14, Srinagar Garhwal 246174, Uttarakhand, India
| | - Sudeep Chandra
- High Altitude Plant Physiology Research Centre (HAPPRC), H.N.B. Garhwal University, Post Box: 14, Srinagar Garhwal 246174, Uttarakhand, India
| | - Vijay Laxmi Trivedi
- High Altitude Plant Physiology Research Centre (HAPPRC), H.N.B. Garhwal University, Post Box: 14, Srinagar Garhwal 246174, Uttarakhand, India.
| | - Vijay Kant Purohit
- High Altitude Plant Physiology Research Centre (HAPPRC), H.N.B. Garhwal University, Post Box: 14, Srinagar Garhwal 246174, Uttarakhand, India
| | - M C Nautiyal
- High Altitude Plant Physiology Research Centre (HAPPRC), H.N.B. Garhwal University, Post Box: 14, Srinagar Garhwal 246174, Uttarakhand, India
| |
Collapse
|
|
8
|
Wu Y, Krueger F. Charting the neuroscience of interpersonal trust: A bibliographic literature review. Neurosci Biobehav Rev 2024; 167:105930. [PMID: 39433115 DOI: 10.1016/j.neubiorev.2024.105930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2024] [Revised: 10/16/2024] [Accepted: 10/17/2024] [Indexed: 10/23/2024]
Abstract
Interpersonal trust is essential for societal well-being, underpinning relationships from individuals to institutions. Neuroscience research on trust has advanced swiftly since 2001. While quantitative reviews, meta-analyses, and theoretical frameworks have effectively synthesized trust neuroscience research, bibliometric analysis remains underutilized. Our bibliometric analysis aimed to provide a comprehensive overview of trust neuroscience's current state and future directions by examining its historical development, key contributors, geographic distribution, methodological paradigms, influential works, thematic trends, and overall impact. This field has been characterized by the input of a few key contributors through international collaboration, with significant contributions from the U.S., China, the Netherlands, and Germany. Research predominantly utilizes the trust game and fMRI, with a rising focus on neural networks, general trust, and differentiating behavioral from attitudinal trust. Integrating insights from psychology, economics, and sociology, this interdisciplinary field holds promise for advancing our understanding of trust through a neurobiological lens. In conclusion, our bibliographic literature review provides valuable insights and guidance for scholars, spotlighting potential avenues for further investigation in this fast-growing field.
Collapse
Affiliation(s)
- Yan Wu
- Department of Psychology, College of Education, Hangzhou Normal University, Hangzhou 311121, China; Zhejiang Philosophy and Social Science Laboratory for Research in Early Development and Childcare, Hangzhou Normal University, Hangzhou 311121, China
| | - Frank Krueger
- School of Systems Biology, George Mason University, Fairfax, VA, USA; Department of Psychology, University of Mannheim, Mannheim, Germany.
| |
Collapse
|
|
9
|
Fairchild AJ, Yin Y, Baraldi AN, Astivia OLO, Shi D. Many nonnormalities, one simulation: Do different data generation algorithms affect study results? Behav Res Methods 2024; 56:6464-6484. [PMID: 38389030 DOI: 10.3758/s13428-024-02364-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/07/2024] [Indexed: 02/24/2024]
Abstract
Monte Carlo simulation studies are among the primary scientific outputs contributed by methodologists, guiding application of various statistical tools in practice. Although methodological researchers routinely extend simulation study findings through follow-up work, few studies are ever replicated. Simulation studies are susceptible to factors that can contribute to replicability failures, however. This paper sought to conduct a meta-scientific study by replicating one highly cited simulation study (Curran et al., Psychological Methods, 1, 16-29, 1996) that investigated the robustness of normal theory maximum likelihood (ML)-based chi-square fit statistics under multivariate nonnormality. We further examined the generalizability of the original study findings across different nonnormal data generation algorithms. Our replication results were generally consistent with original findings, but we discerned several differences. Our generalizability results were more mixed. Only two results observed under the original data generation algorithm held completely across other algorithms examined. One of the most striking findings we observed was that results associated with the independent generator (IG) data generation algorithm vastly differed from other procedures examined and suggested that ML was robust to nonnormality for the particular factor model used in the simulation. Findings point to the reality that extant methodological recommendations may not be universally valid in contexts where multiple data generation algorithms exist for a given data characteristic. We recommend that researchers consider multiple approaches to generating a specific data or model characteristic (when more than one is available) to optimize the generalizability of simulation results.
Collapse
Affiliation(s)
- Amanda J Fairchild
- Department of Psychology, University of South Carolina, Columbia, SC, USA.
| | - Yunhang Yin
- Department of Psychology, University of South Carolina, Columbia, SC, USA
| | - Amanda N Baraldi
- Department of Psychology, Oklahoma State University, Stillwater, OK, USA
| | | | - Dexin Shi
- Department of Psychology, University of South Carolina, Columbia, SC, USA
| |
Collapse
|
|
10
|
Krauss A. Science of science: A multidisciplinary field studying science. Heliyon 2024; 10:e36066. [PMID: 39296115 PMCID: PMC11408022 DOI: 10.1016/j.heliyon.2024.e36066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2023] [Revised: 07/24/2024] [Accepted: 08/08/2024] [Indexed: 09/21/2024] Open
Abstract
Science and knowledge are studied by researchers across many disciplines, examining how they are developed, what their current boundaries are and how we can advance them. By integrating evidence across disparate disciplines, the holistic field of science of science can address these foundational questions. This field illustrates how science is shaped by many interconnected factors: the cognitive processes of scientists, the historical evolution of science, economic incentives, institutional influences, computational approaches, statistical, mathematical and instrumental foundations of scientific inference, scientometric measures, philosophical and ethical dimensions of scientific concepts, among other influences. Achieving a comprehensive overview of a multifaceted field like the science of science requires pulling together evidence from the many sub-fields studying science across the natural and social sciences and humanities. This enables developing an interdisciplinary perspective of scientific practice, a more holistic understanding of scientific processes and outcomes, and more nuanced perspectives to how scientific research is conducted, influenced and evolves. It enables leveraging the strengths of various disciplines to create a holistic view of the foundations of science. Different researchers study science from their own disciplinary perspective and use their own methods, and there is a large divide between quantitative and qualitative researchers as they commonly do not read or cite research using other methodological approaches. A broader, synthesizing paper employing a qualitative approach can however help provide a bridge between disciplines by pulling together aspects of science (economic, scientometric, psychological, philosophical etc.). Such an approach enables identifying, across the range of fields, the powerful role of our scientific methods and instruments in shaping most aspects of our knowledge and science, whereas economic, social and historical influences help shape what knowledge we pursue. A unifying theory is then outlined for science of science - the new-methods-drive-science theory.
Collapse
Affiliation(s)
- Alexander Krauss
- London School of Economics, London, UK
- Institute for Economic Analysis, Spanish National Research Council, Barcelona, Spain
| |
Collapse
|
|
11
|
Lozada-Martinez ID, Hernandez-Paez DA, Palacios Velasco I, Martinez Guevara D, Liscano Y. Meta-Research in Geriatric Surgery: Improving the Quality of Surgical Evidence for Older Persons in a Multidimensional-Scale Research Field. J Clin Med 2024; 13:5441. [PMID: 39336933 PMCID: PMC11432730 DOI: 10.3390/jcm13185441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2024] [Revised: 09/07/2024] [Accepted: 09/12/2024] [Indexed: 09/30/2024] Open
Abstract
The world is facing a significant demographic transition, with a substantial increase in the proportion of older persons, as well as long-lived persons (especially nonagenarians and centenarians). One of the popular beliefs is that old age is synonymous with disease and disability. However, the successful aging hypothesis suggests that those older persons with advanced chronological age who maintain their functional capacity derive from it a delay in biological aging, enhancing the quality of organic aging and regulation. Therefore, regardless of chronological age, even in cases of extreme longevity, those older adults with a successful aging phenotype and favorable functional capacity would be expected to have satisfactory post-surgical recovery with a low risk of morbidity and mortality. Currently, there is a significant gap between the availability of high-certainty surgical evidence that allows for evidence-based interventions applicable to the long-lived population-taking into account the actual conditioning factors of the health phenotype in older persons-and, above all, predictors of satisfactory post-surgical evolution. The application of meta-research to geriatric surgery emerges as a fundamental tool to address this knowledge gap and reveals opportunities and limitations that need to be resolved in the near future to establish evidence-based surgical care for older persons. The aim of this manuscript was to present a real and globally relevant scenario related to surgical care, addressing the longevity, the availability, and the quality of surgical evidence applicable to this population, and also to present variables to consider in analysis and future perspectives in research and meta-research in geriatric surgery.
Collapse
Affiliation(s)
- Ivan David Lozada-Martinez
- Biomedical Scientometrics and Evidence-Based Research Unit, Department of Health Sciences, Universidad de la Costa, Barranquilla 500366, Colombia;
| | - David A. Hernandez-Paez
- Center for Meta-Research and Scientometrics in Biomedical Sciences, Barranquilla 500366, Colombia;
- Grupo Prometheus y Biomedicina Aplicada a las Ciencias Clinicas, School of Medicine, Universidad de Cartagena, Cartagena 130001, Colombia
| | - Isabela Palacios Velasco
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali 518300, Colombia; (I.P.V.); (D.M.G.)
| | - Darly Martinez Guevara
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali 518300, Colombia; (I.P.V.); (D.M.G.)
| | - Yamil Liscano
- Grupo de Investigación en Salud Integral (GISI), Departamento Facultad de Salud, Universidad Santiago de Cali, Cali 518300, Colombia; (I.P.V.); (D.M.G.)
| |
Collapse
|
|
12
|
Astaneh B, Abdullah R, Astaneh V, Gupta S, Brignardello-Petersen R, Levine MAH, Guaytt G. Pedagogic Strategies and Contents in Medical Writing/Publishing Education: A Comprehensive Systematic Survey. Eur J Investig Health Psychol Educ 2024; 14:2491-2508. [PMID: 39329833 PMCID: PMC11431838 DOI: 10.3390/ejihpe14090165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2024] [Revised: 08/17/2024] [Accepted: 08/26/2024] [Indexed: 09/28/2024] Open
Abstract
Workshops or training sessions on medical writing and publishing exist worldwide. We aimed to evaluate published articles about such workshops and examine both the content and teaching strategies employed. We searched ISI Web of Science, Ovid EMBASE, ERIC, Ovid Medline, and the grey literature. We considered no language, geographical location, or time period limitations. We included randomized controlled trials, before-after studies, surveys, cohort studies, and program evaluation and development studies. We descriptively reported the results. Out of 222 articles that underwent a full-text review, 30 were deemed eligible. The educational sessions were sporadic, with researchers often developing their own content and methods. Fifteen articles reported teaching the standard structure of medical articles, ten articles reported on teaching optimal English language use for writing articles, nine articles discussed publication ethics issues, and three articles discussed publication strategies to enhance the chance of publication. Most reports lacked in-depth descriptions of the content and strategies used, and the approach to those topics was relatively superficial. Existing workshops have covered topics such as the standard structure of articles, publication ethics, techniques for improving publication rates, and how to use the English language. However, many other topics are left uncovered. The reports and practice of academic-teaching courses should be improved.
Collapse
Affiliation(s)
- Behrooz Astaneh
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON L8S 4K1, Canada
| | - Ream Abdullah
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A1, Canada
| | - Vala Astaneh
- Faculty of Kinesiology and Health Sciences, York University, Toronto, ON M3J 1P3, Canada
| | - Sana Gupta
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON L8S 4K1, Canada
| | - Romina Brignardello-Petersen
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON L8S 4K1, Canada
| | - Mitchell A H Levine
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON L8S 4K1, Canada
- Department of Medicine, Division of Clinical Pharmacology and Toxicology, McMaster University, Hamilton, ON L8S 4K1, Canada
| | - Gordon Guaytt
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON L8S 4K1, Canada
- Department of Medicine, McMaster University, Hamilton, ON L8S 4K1, Canada
- MAGIC Evidence Ecosystem Foundation, 0456 Oslo, Norway
| |
Collapse
|
|
13
|
Ng JY, Lee MS, Liu JP, Steel A, Wieland LS, Witt CM, Moher D, Cramer H. How can meta-research be used to evaluate and improve the quality of research in the field of traditional, complementary, and integrative medicine? Integr Med Res 2024; 13:101068. [PMID: 39253695 PMCID: PMC11381986 DOI: 10.1016/j.imr.2024.101068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 06/19/2024] [Accepted: 07/07/2024] [Indexed: 09/11/2024] Open
Abstract
The field of traditional, complementary, and integrative medicine (TCIM) has garnered increasing attention due to its holistic approach to health and well-being. While the quantity of published research about TCIM has increased exponentially, critics have argued that the field faces challenges related to methodological rigour, reproducibility, and overall quality. This article proposes meta-research as one approach to evaluating and improving the quality of TCIM research. Meta-research, also known as research about research, can be defined as "the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives". By systematically evaluating methodological rigour, identifying biases, and promoting transparency, meta-research can enhance the reliability and credibility of TCIM research. Specific topics of interest that are discussed in this article include the following: 1) study design and research methodology, 2) reporting of research, 3) research ethics, integrity, and misconduct, 4) replicability and reproducibility, 5) peer review and journal editorial practices, 6) research funding: grants and awards, and 7) hiring, promotion, and tenure. For each topic, we provide case examples to illustrate meta-research applications in TCIM. We argue that meta-research initiatives can contribute to maintaining public trust, safeguarding research integrity, and advancing evidence based TCIM practice, while challenges include navigating methodological complexities, biases, and disparities in funding and academic recognition. Future directions involve tailored research methodologies, interdisciplinary collaboration, policy implications, and capacity building in meta-research.
Collapse
Affiliation(s)
- Jeremy Y Ng
- Institute of General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany
- Robert Bosch Center for Integrative Medicine and Health, Bosch Health Campus, Stuttgart, Germany
- Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Myeong Soo Lee
- KM Science Research Division, Korea Institute of Oriental Medicine, Daejeon, South Korea
| | - Jian-Ping Liu
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Amie Steel
- Australian Research Consortium in Complementary and Integrative Medicine (ARCCIM), School of Public Health, Faculty of Health, University of Technology Sydney, Australia
| | - L Susan Wieland
- Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, United States
- Institute for Complementary and Integrative Medicine, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Claudia M Witt
- Institute for Complementary and Integrative Medicine, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - David Moher
- Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| | - Holger Cramer
- Institute of General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany
- Robert Bosch Center for Integrative Medicine and Health, Bosch Health Campus, Stuttgart, Germany
| |
Collapse
|
|
14
|
Zambonin Mazzoleni G, Bergna A, Buffone F, Sacchi A, Misseroni S, Tramontano M, Dal Farra F. A Critical Appraisal of Reporting in Randomized Controlled Trials Investigating Osteopathic Manipulative Treatment: A Meta-Research Study. J Clin Med 2024; 13:5181. [PMID: 39274394 PMCID: PMC11396362 DOI: 10.3390/jcm13175181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2024] [Revised: 08/26/2024] [Accepted: 08/29/2024] [Indexed: 09/16/2024] Open
Abstract
Background/Objectives: In osteopathy, it becomes necessary to produce high-quality evidence to demonstrate its effectiveness. The aim of this meta-research study is to assess the reporting quality of RCTs published in the osteopathic field. Methods: The protocol was preliminarily registered on the "Open Science Framework (OSF)" website. For reporting, we considered the PRISMA 2020 checklist. We included all the RCTs, published between 2011 and 2023, investigating the effectiveness of Osteopathic Manipulative Treatment (OMT) in any possible condition. The search process was conducted on four major biomedical databases including PubMed, Central, Scopus and Embase. A data extraction form was implemented to collect all relevant information. The completeness of reporting was calculated as the percentage of adherence to the CONSORT checklist; the Cochrane ROB 2 tool was considered to assess the risk of bias (RoB) in the following five major domains: randomization (D1), interventions (D2), missing data (D3), outcome measurement (D4), selective reporting (D5). Results: A total of 131 studies were included and the overall adherence was 57%, with the worst section being "other information" (42%). Studies with a lower RoB showed higher adherence to the CONSORT. The "results" section presented the highest differences as follows: D1 (-36.7%), D2 (-27.2%), D3 (-21.5%) and D5 (-25.5%). Significant correlations were also found between the preliminary protocol registration, higher journal quartile, publication in hybrid journals and the completeness of reporting (β: 19.22, CI: 14.45-24.00, p < 0.001; β: 5.41; CI: 2.80-8.02, p ≤ 0.001; β: 5.64, CI: 1.06-10.23, p = 0.016, respectively). Conclusions: The adherence to the CONSORT checklist in osteopathic RCTs is lacking. An association was found between a lower completeness of reporting and a higher RoB, a good journal ranking, publication in hybrid journals and a prospective protocol registration. Journals and authors should adopt all the strategies to adhere to reporting guidelines to guarantee generalization of the results arising from RCTs.
Collapse
Affiliation(s)
- Gabriele Zambonin Mazzoleni
- SOMA-Istituto Osteopatia Milano, Viale Sarca 336 F, 20126 Milan, Italy
- Physiotherapy Degree Course, Department of Medicine and Technology Innovation, Università degli Studi dell'Insubria, 21100 Varese, Italy
| | - Andrea Bergna
- SOMA-Istituto Osteopatia Milano, Viale Sarca 336 F, 20126 Milan, Italy
- AISO-Associazione Italiana Scuole di Osteopatia, 65125 Pescara, Italy
| | - Francesca Buffone
- SOMA-Istituto Osteopatia Milano, Viale Sarca 336 F, 20126 Milan, Italy
- PPCR, Harvard T.H. Chan School of Public Health-ECPE, Boston, MA 02115-6096, USA
| | - Andrea Sacchi
- SOMA-Istituto Osteopatia Milano, Viale Sarca 336 F, 20126 Milan, Italy
| | - Serena Misseroni
- SOMA-Istituto Osteopatia Milano, Viale Sarca 336 F, 20126 Milan, Italy
| | - Marco Tramontano
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy
- Unit of Occupational Medicine, IRCCS Azienda Ospedaliero-Universitaria di Bologna, 40138 Bologna, Italy
| | - Fulvio Dal Farra
- SOMA-Istituto Osteopatia Milano, Viale Sarca 336 F, 20126 Milan, Italy
- Department Information Engineering, University of Brescia, Via Branze 38, 25123 Brescia, Italy
| |
Collapse
|
|
15
|
Zhong J, Xing Y, Hu Y, Lu J, Yang J, Zhang G, Mao S, Chen H, Yin Q, Cen Q, Jiang R, Chu J, Song Y, Lu M, Ding D, Ge X, Zhang H, Yao W. The policies on the use of large language models in radiological journals are lacking: a meta-research study. Insights Imaging 2024; 15:186. [PMID: 39090273 PMCID: PMC11294318 DOI: 10.1186/s13244-024-01769-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2024] [Accepted: 07/07/2024] [Indexed: 08/04/2024] Open
Abstract
OBJECTIVE To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence. METHODS In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated. RESULTS The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001). CONCLUSION The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency. CRITICAL RELEVANCE STATEMENT It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals. KEY POINTS The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.
Collapse
Affiliation(s)
- Jingyu Zhong
- Laboratory of Key Technology and Materials in Minimally Invasive Spine Surgery, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
- Center for Spinal Minimally Invasive Research, Shanghai Jiao Tong University, Shanghai, China.
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Yue Xing
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yangfan Hu
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Junjie Lu
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA
| | - Jiarui Yang
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Guangcheng Zhang
- Department of Orthopedics, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Shiqi Mao
- Department of Medical Oncology, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Haoda Chen
- Department of General Surgery, Pancreatic Disease Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Qian Yin
- Department of Pathology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Qingqing Cen
- Department of Dermatology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Run Jiang
- Department of Pharmacovigilance, Shanghai Hansoh BioMedical Co., Ltd., Shanghai, China
| | - Jingshen Chu
- Editorial Office of Journal of Diagnostics Concepts & Practice, Department of Science and Technology Development, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yang Song
- MR Scientific Marketing, Siemens Healthineers Ltd., Shanghai, China
| | - Minda Lu
- MR Application, Siemens Healthineers Ltd., Shanghai, China
| | - Defang Ding
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xiang Ge
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Huan Zhang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Weiwu Yao
- Laboratory of Key Technology and Materials in Minimally Invasive Spine Surgery, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
- Center for Spinal Minimally Invasive Research, Shanghai Jiao Tong University, Shanghai, China.
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| |
Collapse
|
|
16
|
Ng JY, Cramer H. Meta-Research in Traditional, Complementary, and Integrative Medicine: The Value of Standing on the Outside Looking in. JOURNAL OF INTEGRATIVE AND COMPLEMENTARY MEDICINE 2024; 30:709-711. [PMID: 39058632 DOI: 10.1089/jicm.2024.0444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/28/2024]
Affiliation(s)
- Jeremy Y Ng
- Institute of General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany
- Robert Bosch Center for Integrative Medicine and Health, Bosch Health Campus, Stuttgart, Germany
- Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Holger Cramer
- Institute of General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany
- Robert Bosch Center for Integrative Medicine and Health, Bosch Health Campus, Stuttgart, Germany
| |
Collapse
|
|
17
|
de Lima LP, Mello AT, Nascimento GM, Trindade EBSDM. Methodological quality of research on perioperative immunomodulatory supplementation in oncological gastrointestinal tract surgery: a meta-research protocol. BMJ Open 2024; 14:e082112. [PMID: 39059807 PMCID: PMC11284874 DOI: 10.1136/bmjopen-2023-082112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Accepted: 06/14/2024] [Indexed: 07/28/2024] Open
Abstract
INTRODUCTION One of the topics that show differences of opinion in the scientific field of nutrition is the recommendation by clinical practice guidelines (CPGs) of an immunomodulatory diet with arginine, nucleotides and omega-3 for individuals diagnosed with cancer undergoing major surgery. The quality of the recommendations is directly related to credibility, transparency and rigour in their development, but also to the quality of the studies published and available for inclusion in the recommendation, such as systematic reviews (SRs) and randomised clinical trials. The aim of this study is to evaluate the methodological quality of the recommendation of perioperative immunomodulatory supplementation for individuals with gastrointestinal and head and neck cancer, the CPGs, and the studies that support the recommendations. METHODS AND ANALYSIS We will conduct a systematic search for CPGs. Recommendations for nutritional supplementation with immunomodulatory substrates for individuals undergoing major oncological surgery will be analysed using the Appraisal of Guidelines Research and Evaluation-Recommendations Excellence tool. CPGs will be analysed using the Appraisal of Guidelines Research and Evaluation II tool. The SRs cited in the recommendations will be analysed using the A Measurement Tool to Assess Systematic Reviews II tool and additional questions regarding heterogeneity in reviews. The clinical trials cited in the SRs and in the guideline recommendations (when applicable) will be analysed according to questions regarding heterogeneity in trials. The results will be presented in tables or charts using descriptive analyses. ETHICS AND DISSEMINATION The results of this study will be disseminated through relevant conferences and peer-reviewed journals. PROTOCOL REGISTRATION NUMBER 10.17605/OSF.IO/X2GYT.
Collapse
Affiliation(s)
- Luana Pucci de Lima
- Post-Graduate Program in Nutrition, Federal University of Santa Catarina, Florianópolis, Brazil
| | - Arthur Thives Mello
- Post-Graduate Program in Nutrition, Federal University of Santa Catarina, Florianópolis, Brazil
| | | | - Erasmo B S de Moraes Trindade
- Post-Graduate Program in Nutrition, Federal University of Santa Catarina, Florianópolis, Brazil
- Department of Nutrition, Federal University of Santa Catarina, Florianopolis, Brazil
| |
Collapse
|
|
18
|
Bała MM, Poklepović Peričić T, Žuljević MF, Bralić N, Zając J, Motaze NV, Rohwer A, Gajdzica M, Young T. Adherence to the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) of studies on evidence-based healthcare e-learning: a cross-sectional study. BMJ Evid Based Med 2024; 29:229-238. [PMID: 38862202 DOI: 10.1136/bmjebm-2023-112647] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/28/2024] [Indexed: 06/13/2024]
Abstract
OBJECTIVES The objectives of this study are to assess reporting of evidence-based healthcare (EBHC) e-learning interventions using the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and explore factors associated with compliant reporting. DESIGN Methodological cross-sectional study. METHODS Based on the criteria used in an earlier systematic review, we included studies comparing EBHC e-learning and any other form of EBHC training or no EBHC training. We searched Medline, Embase, ERIC, CINAHL, CENTRAL, SCOPUS, Web of Knowledge, PsycInfo, ProQuest and Best Evidence Medical Education up to 4 January 2023. Screening of titles, abstracts, full-text articles and data extraction was done independently by two authors. For each study, we assessed adherence to each of the 17 GREET items and extracted information on possible predictors. Adequacy of reporting for each item of the GREET checklist was judged with yes (provided complete information), no (provided no information), unclear (when insufficient information was provided), or not applicable, when the item was clearly of no relevance to the intervention described (such as for item 8-details about the instructors-in the studies which used electronic, self-paced intervention, without any tutoring). Studies' adherence to the GREET checklist was presented as percentages and absolute numbers. We performed univariate analysis to assess the association of potential adherence predictors with the GREET checklist. We summarised results descriptively. RESULTS We included 40 studies, the majority of which assessed e-learning or blended learning and mostly involved medical and other healthcare students. None of the studies fully reported all the GREET items. Overall, the median number of GREET items met (received yes) per study was 8 and third quartile (Q3) of GREET items met per study was 9 (min. 4 max. 14). When we used Q3 of the number of items met as cut-off point, adherence to the GREET reporting checklist was poor with 7 out of 40 studies (17.5%) reporting items of the checklist on acceptable level (adhered to at least 10 items out of 17). None of the studies reported on all 17 GREET items. For 3 items, 80% of included studies well reported information (received yes for these items): item 1 (brief description of intervention), item 4 (evidence-based practice content) and item 6 (educational strategies). Items for which 50% of included studies reported complete information (received yes for these items) included: item 9 (modes of delivery), item 11 (schedule) and 12 (time spent on learning). The items for which 70% or more of included studies did not provide information (received no for these items) included: item 7 (incentives) and item 13 (adaptations; for both items 70% of studies received no for them), item 14 (modifications of educational interventions-95% of studies received no for this item), item 16 (any processes to determine whether the materials and the educational strategies used in the educational intervention were delivered as originally planned-93% of studies received no for this item) and 17 (intervention delivery according to schedule-100% of studies received no for this item). Studies published after September 2016 showed slight improvements in nine reporting items. In the logistic regression models, using the cut-off point of Q3 (10 points or above) the odds of acceptable adherence to GREET guidelines were 7.5 times higher if adherence to other guideline (Consolidated Standards of Reporting Trials, Strengthening the Reporting of Observational Studies in Epidemiology, etc) was reported for a given study type (p=0.039), also higher number of study authors increased the odds of adherence to GREET guidance by 18% (p=0.037). CONCLUSIONS Studies assessing educational interventions on EBHC e-learning still poorly adhere to the GREET checklist. Using other reporting guidelines increased the odds of better GREET reporting. Journals should call for the use of appropriate use of reporting guidelines of future studies on teaching EBHC to increase transparency of reporting, decrease unnecessary research duplication and facilitate uptake of research evidence or result. STUDY REGISTRATION NUMBER The Open Science Framework (https://doi.org/10.17605/OSF.IO/V86FR).
Collapse
Affiliation(s)
- Małgorzata M Bała
- Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Tina Poklepović Peričić
- Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia
| | - Marija Franka Žuljević
- Department of Medical Humanities, University of Split School of Medicine, Split, Croatia
| | - Nensi Bralić
- Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia
| | - Joanna Zając
- Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Nkengafac Villyen Motaze
- Medicine Usage in South Africa, Faculty of Health Sciences, North-West University, Potchefstroom, South Africa
| | - Anke Rohwer
- Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University Faculty of Medicine and Health Sciences, Cape Town, Western Cape, South Africa
| | - Michalina Gajdzica
- Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Taryn Young
- Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University Faculty of Medicine and Health Sciences, Cape Town, Western Cape, South Africa
| |
Collapse
|
|
19
|
Stevens ER, Cleland CM, Shunk A, El Shahawy O. Evaluating strategies to recruit health researchers to participate in online survey research. BMC Med Res Methodol 2024; 24:153. [PMID: 39026149 PMCID: PMC11256559 DOI: 10.1186/s12874-024-02275-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Accepted: 07/05/2024] [Indexed: 07/20/2024] Open
Abstract
BACKGROUND Engaging researchers as research subjects is key to informing the development of effective and relevant research practices. It is important to understand how best to engage researchers as research subjects. METHODS A 24 factorial experiment, as part of a Multiphase Optimization Strategy, was performed to evaluate effects of four recruitment strategy components on participant opening of an emailed survey link and survey completion. Participants were members of three US-based national health research consortia. A stratified simple random sample was used to assign potential survey participants to one of 16 recruitment scenarios. Recruitment strategy components were intended to address both intrinsic and extrinsic sources of motivation, including: $50 gift, $1,000 raffle, altruistic messaging, and egoistic messaging. Multivariable generalized linear regression analyses adjusting for consortium estimated component effects on outcomes. Potential interactions among components were tested. Results are reported as adjusted odds ratios (aOR) with 95% confidence intervals (95% CI). RESULTS Surveys were collected from June to December 2023. A total of 418 participants were included from the consortia, with final analytical sample of 400 eligible participants. Out of the final sample, 82% (341) opened the survey link and 35% (147) completed the survey. Altruistic messaging increased the odds of opening the survey (aOR 2.02, 95% CI: 1.35-2.69, p = 0.033), while egoistic messaging significantly reduced the odds of opening the survey (aOR 0.56, 95%CI 0.38-0.75, p = 0.08). The receipt of egoistic messaging increased the odds of completing the survey once opened (aOR 1.81, 95%CI: 1.39-2.23, p < 0.05). There was a significant negative interaction effect between the altruistic appeal and egoistic messaging strategies for survey completion outcome. Monetary incentives did not a have a significant impact on survey completion. CONCLUSION Intrinsic motivation is likely to be a greater driver of health researcher participation in survey research than extrinsic motivation. Altruistic and egoistic messaging may differentially impact initial interest and survey completion and when combined may lead to improved rates of recruitment, but not survey completion. Further research is needed to determine how to best optimize message content and whether the effects observed are modified by survey burden.
Collapse
Affiliation(s)
- Elizabeth R Stevens
- Department of Population Health, NYU Grossman School of Medicine, New York, NY, USA.
| | - Charles M Cleland
- Department of Population Health, NYU Grossman School of Medicine, New York, NY, USA
| | - Amelia Shunk
- Department of Population Health, NYU Grossman School of Medicine, New York, NY, USA
| | - Omar El Shahawy
- Department of Population Health, NYU Grossman School of Medicine, New York, NY, USA
- School of Global Public Health, New York University, New York, NY, USA
| |
Collapse
|
|
20
|
Papaefthimiou M, Kontou PI, Bagos PG, Braliou GG. Integration of Antioxidant Activity Assays Data of Stevia Leaf Extracts: A Systematic Review and Meta-Analysis. Antioxidants (Basel) 2024; 13:692. [PMID: 38929131 PMCID: PMC11201069 DOI: 10.3390/antiox13060692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 05/22/2024] [Accepted: 05/28/2024] [Indexed: 06/28/2024] Open
Abstract
Stevia rebaudiana Bertoni, a no-calorie natural sweetener, contains a plethora of polyphenols that exert antioxidant properties with potential medicinal significance. Due to the variety of functional groups, polyphenols exhibit varying solubility depending on the nature of the extraction solvents (water, organic, or their mixtures, defined further on as hydroalcoholic extracts). In the present study, we performed a systematic review, following PRISMA guidelines, and meta-analysis, synthesizing all available data from 45 articles encompassing 250 different studies. Our results showed that the total phenolic content (TPC) of hydroalcoholic and aqueous extracts presents higher values (64.77 and 63.73 mg GAE/g) compared to organic extracts (33.39). Total flavonoid content (TFC) was also higher in aqueous and hydroalcoholic extracts; meta-regression analysis revealed that outcomes in different measuring units (mg QE/g, mg CE/g, and mg RUE/g) do not present statistically significant differences and can be synthesized in meta-analysis. Using meta-regression analysis, we showed that outcomes from the chemical-based ABTS, FRAP, and ORAC antioxidant assays for the same extract type can be combined in meta-analysis because they do not differ statistically significantly. Meta-analysis of ABTS, FRAP, and ORAC assays outcomes revealed that the antioxidant activity profile of various extract types follows that of their phenolic and flavonoid content. Using regression meta-analysis, we also presented that outcomes from SOD, CAT, and POX enzymatic antioxidant assays are independent of the assay type (p-value = 0.905) and can be combined. Our study constitutes the first effort to quantitatively and statistically synthesize the research results of individual studies using all methods measuring the antioxidant activity of stevia leaf extracts. Our results, in light of evidence-based practice, uncover the need for a broadly accepted, unified, methodological strategy to perform antioxidant tests, and offer documentation that the use of ethanol:water 1:1 mixtures or pure water can more efficiently extract stevia antioxidant compounds.
Collapse
Affiliation(s)
- Maria Papaefthimiou
- Department of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, Greece; (M.P.); (P.G.B.)
| | | | - Pantelis G. Bagos
- Department of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, Greece; (M.P.); (P.G.B.)
| | - Georgia G. Braliou
- Department of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, Greece; (M.P.); (P.G.B.)
| |
Collapse
|
|
21
|
Rosengaard LO, Andersen MZ, Rosenberg J, Fonnes S. Five aspects of research waste in biomedicine: A scoping review. J Evid Based Med 2024; 17:351-359. [PMID: 38798014 DOI: 10.1111/jebm.12616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 05/12/2024] [Indexed: 05/29/2024]
Abstract
BACKGROUND The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports. METHODS In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste. RESULTS Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination. CONCLUSION MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.
Collapse
Affiliation(s)
- Louise Olsbro Rosengaard
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Mikkel Zola Andersen
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Jacob Rosenberg
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Siv Fonnes
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| |
Collapse
|
|
22
|
Laynor G, Stevens ER. Identifying meta-research with researchers as study subjects: Protocol for a scoping review. PLoS One 2024; 19:e0303905. [PMID: 38768101 PMCID: PMC11104640 DOI: 10.1371/journal.pone.0303905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 05/02/2024] [Indexed: 05/22/2024] Open
Abstract
BACKGROUND Meta-research in which researchers are the study subjects can illuminate how to better support researchers and enhance the development of research capacity. Comprehensively compiling the literature in this area can help define best practices for research capacity development and reveal gaps in the literature. However, there are challenges to assessing and synthesizing the breadth of the meta-research literature produced. METHODS In this article, we discuss the current barriers to conducting literature reviews on meta-research and strategies to address these barriers. We then outline proposed methods for conducting a scoping review on meta-research with researchers as study subjects. DISCUSSION Due to its interdisciplinary nature, broad scope, and difficult to pinpoint terminology, little is known about the state of meta-research with researchers as the study subjects. For this reason, there is a need for a scoping review that will identify research performed in which researchers were the study subjects.
Collapse
Affiliation(s)
- Gregory Laynor
- NYU Health Sciences Library, NYU Grossman School of Medicine, New York, NY, United States of America
| | - Elizabeth R. Stevens
- Department of Population Health, NYU Grossman School of Medicine, New York, NY, United States of America
| |
Collapse
|
|
23
|
Stogiannis D, Siannis F, Androulakis E. Heterogeneity in meta-analysis: a comprehensive overview. Int J Biostat 2024; 20:169-199. [PMID: 36961993 DOI: 10.1515/ijb-2022-0070] [Citation(s) in RCA: 18] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 02/10/2023] [Indexed: 03/26/2023]
Abstract
In recent years, meta-analysis has evolved to a critically important field of Statistics, and has significant applications in Medicine and Health Sciences. In this work we briefly present existing methodologies to conduct meta-analysis along with any discussion and recent developments accompanying them. Undoubtedly, studies brought together in a systematic review will differ in one way or another. This yields a considerable amount of variability, any kind of which may be termed heterogeneity. To this end, reports of meta-analyses commonly present a statistical test of heterogeneity when attempting to establish whether the included studies are indeed similar in terms of the reported output or not. We intend to provide an overview of the topic, discuss the potential sources of heterogeneity commonly met in the literature and provide useful guidelines on how to address this issue and to detect heterogeneity. Moreover, we review the recent developments in the Bayesian approach along with the various graphical tools and statistical software that are currently available to the analyst. In addition, we discuss sensitivity analysis issues and other approaches of understanding the causes of heterogeneity. Finally, we explore heterogeneity in meta-analysis for time to event data in a nutshell, pointing out its unique characteristics.
Collapse
Affiliation(s)
| | - Fotios Siannis
- Department of Mathematics, National and Kapodistrian University, Athens, Greece
| | - Emmanouil Androulakis
- Mathematical Modeling and Applications Laboratory, Section of Mathematics, Hellenic Naval Academy, Piraeus, Greece
| |
Collapse
|
|
24
|
Aguilar L, Gath-Morad M, Grübel J, Ermatinger J, Zhao H, Wehrli S, Sumner RW, Zhang C, Helbing D, Hölscher C. Experiments as Code and its application to VR studies in human-building interaction. Sci Rep 2024; 14:9883. [PMID: 38688980 PMCID: PMC11061313 DOI: 10.1038/s41598-024-60791-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 04/26/2024] [Indexed: 05/02/2024] Open
Abstract
Experiments as Code (ExaC) is a concept for reproducible, auditable, debuggable, reusable, & scalable experiments. Experiments are a crucial tool to understand Human-Building Interactions (HBI) and build a coherent theory around it. However, a common concern for experiments is their auditability and reproducibility. Experiments are usually designed, provisioned, managed, and analyzed by diverse teams of specialists (e.g., researchers, technicians, engineers) and may require many resources (e.g., cloud infrastructure, specialized equipment). Although researchers strive to document experiments accurately, this process is often lacking. Consequently, it is difficult to reproduce these experiments. Moreover, when it is necessary to create a similar experiment, the "wheel is very often reinvented". It appears easier to start from scratch than trying to reuse existing work. Thus valuable embedded best practices and previous experiences are lost. In behavioral studies, such as in HBI, this has contributed to the reproducibility crisis. To tackle these challenges, we propose the ExaC paradigm, which not only documents the whole experiment, but additionally provides the automation code to provision, deploy, manage, and analyze the experiment. To this end, we define the ExaC concept, provide a taxonomy for the components of a practical implementation, and provide a proof of concept with an HBI desktop VR experiment that demonstrates the benefits of its "as code" representation, that is, reproducibility, auditability, debuggability, reusability, & scalability.
Collapse
Affiliation(s)
- Leonel Aguilar
- Chair of Cognitive Science, ETH Zürich, Zurich, Switzerland.
- Data Science, Systems and Services Group, ETH Zürich, Zurich, Switzerland.
| | - Michal Gath-Morad
- Chair of Cognitive Science, ETH Zürich, Zurich, Switzerland
- Cambridge Cognitive Architecture, University of Cambridge, Cambridge, UK
| | - Jascha Grübel
- Chair of Cognitive Science, ETH Zürich, Zurich, Switzerland
- Geo-information Science and Remote Sensing Laboratory, Wageningen University, Wageningen, The Netherlands
- Game Technology Center, ETH Zürich, Zurich, Switzerland
- Visual Computing Group, Harvard University, Cambridge, USA
- Center for Sustainable Future Mobility, ETH Zürich, Zurich, Switzerland
- Geoinformation Engineering Group, ETH Zürich, Zurich, Switzerland
| | | | - Hantao Zhao
- School of Cyber Science and Engineering, Southeast University, Nanjing, China
- Purple Mountain Laboratories, Nanjing, China
| | - Stefan Wehrli
- Decision Science Laboratory, ETH Zürich, Zurich, Switzerland
| | - Robert W Sumner
- Geo-information Science and Remote Sensing Laboratory, Wageningen University, Wageningen, The Netherlands
| | - Ce Zhang
- Data Science, Systems and Services Group, ETH Zürich, Zurich, Switzerland
| | - Dirk Helbing
- Decision Science Laboratory, ETH Zürich, Zurich, Switzerland
- Chair of Computational Social Science, ETH Zr̈ich, Zurich, Switzerland
| | - Christoph Hölscher
- Chair of Cognitive Science, ETH Zürich, Zurich, Switzerland
- Decision Science Laboratory, ETH Zürich, Zurich, Switzerland
| |
Collapse
|
|
25
|
Lozada-Martinez ID, Lozada-Martinez LM, Fiorillo-Moreno O. Leiden manifesto and evidence-based research: Are the appropriate standards being used for the correct evaluation of pluralism, gaps and relevance in medical research? J R Coll Physicians Edinb 2024; 54:4-6. [PMID: 38352991 DOI: 10.1177/14782715241227991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/13/2024] Open
Affiliation(s)
- Ivan David Lozada-Martinez
- Epidemiology Program, Department of Graduate Studies in Health Sciences, Universidad Autónoma de Bucaramanga, Bucaramanga, Colombia
| | | | | |
Collapse
|
|
26
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Br J Pharmacol 2024; 181:180-210. [PMID: 37282770 DOI: 10.1111/bph.16100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 04/26/2023] [Indexed: 06/08/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, New York, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California, USA
| |
Collapse
|
|
27
|
Zhong J, Xing Y, Lu J, Zhang G, Mao S, Chen H, Yin Q, Cen Q, Jiang R, Hu Y, Ding D, Ge X, Zhang H, Yao W. The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study. BMC Med Res Methodol 2023; 23:292. [PMID: 38093215 PMCID: PMC10717715 DOI: 10.1186/s12874-023-02117-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 12/01/2023] [Indexed: 12/17/2023] Open
Abstract
BACKGROUND Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables. METHODS This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis. RESULTS We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals. CONCLUSIONS The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.
Collapse
Affiliation(s)
- Jingyu Zhong
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Yue Xing
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Junjie Lu
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - Guangcheng Zhang
- Department of Orthopedics, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China
| | - Shiqi Mao
- Department of Medical Oncology, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, 200433, China
| | - Haoda Chen
- Department of General Surgery, Pancreatic Disease Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Qian Yin
- Department of Pathology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China
| | - Qingqing Cen
- Department of Dermatology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200011, China
| | - Run Jiang
- Department of Pharmacovigilance, Shanghai Hansoh BioMedical Co., Ltd., Shanghai, 201203, China
| | - Yangfan Hu
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Defang Ding
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Xiang Ge
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Huan Zhang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China.
| | - Weiwu Yao
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China.
| |
Collapse
|
|
28
|
Stevens ER, Laynor G. Recognizing the value of meta-research and making it easier to find. J Med Libr Assoc 2023; 111:839-843. [PMID: 37928126 PMCID: PMC10621717 DOI: 10.5195/jmla.2023.1758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2023] Open
Abstract
Meta-research is a bourgeoning field studying topics with significant relevance to health sciences librarianship, such as research reproducibility, peer review, and open access. As a discipline that studies research itself and the practices of researchers, meta-research spans disciplines and encompasses a broad spectrum of topics and methods. The breadth of meta-research presents a significant challenge for identifying published meta-research studies. Introducing a subject heading for meta-research in the controlled vocabularies of literature databases has the potential to increase the visibility of meta-research, further advance the field, and expand its impact on research practices. Given the relatively recent designation of meta-research as a field and its expanding use as a term, now is the time to develop appropriate indexing vocabulary. We seek to call attention to the value of meta-research for health sciences librarianship, describe the challenges of identifying meta-research literature with currently available key terms, and highlight the need to establish controlled vocabulary specific to meta-research.
Collapse
Affiliation(s)
- Elizabeth R Stevens
- , Department of Population Health, NYU Grossman School of Medicine, New York, NY
| | - Gregory Laynor
- , NYU Health Sciences Library, NYU Grossman School of Medicine, New York, NY
| |
Collapse
|
|
29
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Acta Anaesthesiol Scand 2023; 67:1148-1177. [PMID: 37288997 DOI: 10.1111/aas.14295] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 04/26/2023] [Indexed: 06/09/2023]
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, New York, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California, USA
| |
Collapse
|
|
30
|
Sarkis-Onofre R, Sofi-Mahmudi A, Puljak L, Moraes RR. The importance of meta-research in dentistry. Evid Based Dent 2023; 24:98-99. [PMID: 37737329 DOI: 10.1038/s41432-023-00880-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/23/2023]
Affiliation(s)
| | - Ahmad Sofi-Mahmudi
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Livia Puljak
- Center for Evidence-Based Medicine, Catholic University of Croatia, Zagreb, Croatia
| | - Rafael R Moraes
- School of Dentistry, Universidade Federal de Pelotas, Pelotas, RS, Brazil
| |
Collapse
|
|
31
|
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P.A. Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
|
32
|
Zhang P. Which headache disorders can be diagnosed concurrently? An analysis of ICHD3 criteria using prime encoding system. Front Neurol 2023; 14:1221209. [PMID: 37670775 PMCID: PMC10475541 DOI: 10.3389/fneur.2023.1221209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 07/31/2023] [Indexed: 09/07/2023] Open
Abstract
Introduction Real-life headache presentations may fit more than one ICHD3 diagnosis. This project seeks to exhaustively list all logically consistent "co-diagnoses" according to the ICHD3 criteria. We limited our project to cases of two concurrent diagnoses. Methods We included the criteria for "Migraine" (1.1, 1.2, 1.3), "Tension-type headache" (2.1, 2.2, 2.3, 2.4), "Trigeminal autonomic cephalalgias" (3.1, 3.2, 3.3, 3.4, 3.5), and "Other primary headache disorders." We also excluded "probable" diagnosis criteria. Each characteristic in the above criteria is assigned a unique prime number. We then encoded each ICHD3 criteria into integers through multiplication in a list format; we called these criteria representations. "Codiagnoses representations" were generated by multiplying all possible pairings of criteria representations. We then manually encoded a list of logically inconsistent characteristics through multiplication. All co-diagnoses representations divisible by any inconsistency representations were filtered out, generating a list of co-diagnoses representations that were logically consistent. This list was then translated back into ICHD3 diagnoses. Results We used a total of 103 prime numbers to encode 578 ICHD3 criteria. Once illogical characteristics were excluded, we obtained 145 dual diagnoses. Of the dual diagnoses, two contained intersecting characteristics due to subset relationships, 14 contained intersecting characteristics without subset relationships, and 129 contained dual diagnoses as a result of non-intersecting characteristics. Conclusion Analysis of dual diagnosis in headaches offers insight into "loopholes" in the ICHD3 as well as a potential explanation for the source of a number of controversies regarding headache disorders. The existence of dual diagnoses and their identification may carry implications for future developments and testing of machine-learning diagnostic algorithms for headaches.
Collapse
Affiliation(s)
- Pengfei Zhang
- Department of Neurology, Rutgers Robert University Medical School, New Brunswick, NJ, United States
| |
Collapse
|
|
33
|
Bradshaw A, Hughes N, Vallez-Garcia D, Chokoshvili D, Owens A, Hansen C, Emmert K, Maetzler W, Killin L, Barnes R, Brookes AJ, Visser PJ, Hofmann-Apitius M, Diaz C, Steukers L. Data sharing in neurodegenerative disease research: challenges and learnings from the innovative medicines initiative public-private partnership model. Front Neurol 2023; 14:1187095. [PMID: 37545729 PMCID: PMC10397390 DOI: 10.3389/fneur.2023.1187095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 06/02/2023] [Indexed: 08/08/2023] Open
Abstract
Efficient data sharing is hampered by an array of organizational, ethical, behavioral, and technical challenges, slowing research progress and reducing the utility of data generated by clinical research studies on neurodegenerative diseases. There is a particular need to address differences between public and private sector environments for research and data sharing, which have varying standards, expectations, motivations, and interests. The Neuronet data sharing Working Group was set up to understand the existing barriers to data sharing in public-private partnership projects, and to provide guidance to overcome these barriers, by convening data sharing experts from diverse projects in the IMI neurodegeneration portfolio. In this policy and practice review, we outline the challenges and learnings of the WG, providing the neurodegeneration community with examples of good practices and recommendations on how to overcome obstacles to data sharing. These obstacles span organizational issues linked to the unique structure of cross-sectoral, collaborative research initiatives, to technical issues that affect the storage, structure and annotations of individual datasets. We also identify sociotechnical hurdles, such as academic recognition and reward systems that disincentivise data sharing, and legal challenges linked to heightened perceptions of data privacy risk, compounded by a lack of clear guidance on GDPR compliance mechanisms for public-private research. Focusing on real-world, neuroimaging and digital biomarker data, we highlight particular challenges and learnings for data sharing, such as data management planning, development of ethical codes of conduct, and harmonization of protocols and curation processes. Cross-cutting solutions and enablers include the principles of transparency, standardization and co-design - from open, accessible metadata catalogs that enhance findability of data, to measures that increase visibility and trust in data reuse.
Collapse
Affiliation(s)
| | | | - David Vallez-Garcia
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Center, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | | | - Andrew Owens
- Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
| | - Clint Hansen
- Department of Neurology, University Hospital Schleswig-Holstein, Campus Kiel and Kiel University, Kiel, Germany
| | - Kirsten Emmert
- Department of Neurology, University Hospital Schleswig-Holstein, Campus Kiel and Kiel University, Kiel, Germany
| | - Walter Maetzler
- Department of Neurology, University Hospital Schleswig-Holstein, Campus Kiel and Kiel University, Kiel, Germany
| | - Lewis Killin
- Synapse Research Management Partners, Barcelona, Spain
| | | | - Anthony J. Brookes
- Department of Genetics and Genome Biology, University of Leicester, Leicester, United Kingdom
| | - Pieter Jelle Visser
- Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, University of Maastricht, Maastricht, Netherlands
| | | | - Carlos Diaz
- Synapse Research Management Partners, Barcelona, Spain
| | | |
Collapse
|
|
34
|
Dal Santo T, Rice DB, Amiri LSN, Tasleem A, Li K, Boruff JT, Geoffroy MC, Benedetti A, Thombs BD. Methods and results of studies on reporting guideline adherence are poorly reported: a meta-research study. J Clin Epidemiol 2023; 159:225-234. [PMID: 37271424 DOI: 10.1016/j.jclinepi.2023.05.017] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 05/16/2023] [Accepted: 05/22/2023] [Indexed: 06/06/2023]
Abstract
OBJECTIVES We investigated recent meta-research studies on adherence to four reporting guidelines to determine the proportion that provided (1) an explanation for how adherence to guideline items was rated and (2) results from all included individual studies. We examined conclusions of each meta-research study to evaluate possible repetitive and similar findings. STUDY DESIGN AND SETTING A cross-sectional meta-research study. MEDLINE (Ovid) was searched on July 5, 2022 for studies that used any version of the Consolidated Standards of Reporting Trials, Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Standards for the Reporting of Diagnostic Accuracy Studies, or Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines or their extensions to evaluate reporting. RESULTS Of 148 included meta-research studies published between August 2020 and June 2022, 14 (10%, 95% confidence interval [CI] 6%-15%) provided a fully replicable explanation of how they coded the adherence ratings and 49 (33%, 95% CI 26%-41%) completely reported individual study results. Of 90 studies that classified reporting as adequate or inadequate in the study abstract, six (7%, 95% CI 3%-14%) concluded that reporting was adequate, but none of those six studies provided information on how items were coded or provided item-level results for included studies. CONCLUSION Almost all included meta-research studies found that reporting in health research is suboptimal. However, few of these reported enough information for verification or replication.
Collapse
Affiliation(s)
- Tiffany Dal Santo
- Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada; Department of Psychiatry, McGill University, Montreal, Quebec, Canada
| | - Danielle B Rice
- Department of Psychology, St. Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada; Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
| | - Lara S N Amiri
- Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada
| | - Amina Tasleem
- Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada
| | - Kexin Li
- Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada
| | - Jill T Boruff
- Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montreal, Quebec, Canada
| | - Marie-Claude Geoffroy
- Department of Psychiatry, McGill University, Montreal, Quebec, Canada; Department of Educational and Counselling Psychology, McGill University, Montreal, Quebec, Canada; McGill Group for Suicide Studies, Douglas Research Centre, Montreal, Quebec, Canada
| | - Andrea Benedetti
- Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada; Department of Medicine, McGill University, Montreal, Quebec, Canada; Respiratory Epidemiology and Clinical Research Unit, McGill University Health Centre, Montreal, Quebec, Canada; Biomedical Ethics Unit, McGill University, Montreal, Quebec, Canada
| | - Brett D Thombs
- Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada; Department of Psychiatry, McGill University, Montreal, Quebec, Canada; Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada; Department of Medicine, McGill University, Montreal, Quebec, Canada; Biomedical Ethics Unit, McGill University, Montreal, Quebec, Canada; Department of Psychology, McGill University, Montreal, Quebec, Canada.
| |
Collapse
|
|
35
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. BMC Infect Dis 2023; 23:383. [PMID: 37286949 DOI: 10.1186/s12879-023-08304-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 05/03/2023] [Indexed: 06/09/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA.
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
|
36
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Syst Rev 2023; 12:96. [PMID: 37291658 DOI: 10.1186/s13643-023-02255-9] [Citation(s) in RCA: 36] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 02/19/2023] [Indexed: 06/10/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA.
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
|
37
|
Lozada-Martinez ID, Ealo-Cardona CI, Marrugo-Ortiz AC, Picón-Jaimes YA, Cabrera-Vargas LF, Narvaez-Rojas AR. Meta-research studies in surgery: a field that should be encouraged to assess and improve the quality of surgical evidence. Int J Surg 2023; 109:1823-1824. [PMID: 37144675 PMCID: PMC10389356 DOI: 10.1097/js9.0000000000000422] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Accepted: 04/18/2023] [Indexed: 05/06/2023]
Affiliation(s)
- Ivan D. Lozada-Martinez
- Department of Graduate Studies in Health Sciences, Epidemiology Program, Universidad Autónoma de Bucaramanga, Bucaramanga
| | | | | | | | - Luis F. Cabrera-Vargas
- Medical and Surgical Research Center, Future Surgeons Chapter, Colombian Surgery Association, Bogotá, Colombia
| | - Alexis R. Narvaez-Rojas
- International Coalition on Surgical Research, Faculty of Medical Sciences, Universidad Nacional Autónoma de Nicaragua, Managua, Nicaragua
- DeWitt Daughtry Family Department of Surgery, Breast Surgical Oncology Division and Jackson Health System/University of Miami Miller School of Medicine, Miami, Florida, USA
| |
Collapse
|
|
38
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to Best Tools and Practices for Systematic Reviews. JBJS Rev 2023; 11:01874474-202306000-00009. [PMID: 37285444 DOI: 10.2106/jbjs.rvw.23.00077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, North Carolina
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, New York
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California
| |
Collapse
|
|
39
|
Stevens ER, Brody AA, Epps F, Sloan DH, Sherman SE. Using meta-research to foster diverse, equitable, and inclusive collaborative research networks. J Am Geriatr Soc 2023; 71:1028-1033. [PMID: 36585905 PMCID: PMC10089973 DOI: 10.1111/jgs.18217] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Revised: 11/18/2022] [Accepted: 12/15/2022] [Indexed: 01/01/2023]
Abstract
Fostering diverse, equitable, and inclusive collaborative research networks is important for advancing the field of aging research. Despite sizeable investment in research consortia and career development programs, there has been only moderate progress toward diversifying the research workforce studying aging. Without critically examining what works and what does not, continuing to place more resources into these same strategies may not result in a substantial improvement in diversity or the creation of collaborative networks. Using meta-research to rigorously evaluate potential strategies to promote diversity and collaboration may yield important insights that can be used to improve upon current efforts. For this reason, we sought to describe meta-research and highlight how its principles can be used to achieve the aging research community's collaboration and diversity goals.
Collapse
Affiliation(s)
| | - Abraham A. Brody
- NYU Rory Meyers College of Nursing, Hartford Institute for Geriatric Nursing, New York, NY
| | - Fayron Epps
- Emory University, Nell Hodgson Woodruff School of Nursing, Atlanta, GA
| | | | - Scott E. Sherman
- NYU Grossman School of Medicine, Department of Population Health, New York, NY
- Department of Medicine, VA New York Harbor Healthcare System, New York, NY
| |
Collapse
|
|
40
|
Kolaski K, Romeiser Logan L, Ioannidis JPA. Guidance to best tools and practices for systematic reviews1. J Pediatr Rehabil Med 2023; 16:241-273. [PMID: 37302044 DOI: 10.3233/prm-230019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/12/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
|
41
|
Ioannidis JPA. Prolific non-research authors in high impact scientific journals: meta-research study. Scientometrics 2023; 128:3171-3184. [PMID: 37101975 PMCID: PMC10089822 DOI: 10.1007/s11192-023-04687-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Accepted: 03/05/2023] [Indexed: 04/28/2023]
Abstract
Journalistic papers published in high impact scientific journals can be very influential, especially in hot fields. This meta-research analysis aimed to evaluate the publication profiles, impact, and disclosures of conflicts of interest of non-research authors who had published > 200 Scopus-indexed papers in Nature, Science, PNAS, Cell, BMJ, Lancet, JAMA or New England Journal of Medicine. 154 prolific authors were identified, 148 of whom had published 67,825 papers in their main affiliated journal in a non-researcher capacity. Nature, Science, and BMJ have the lion's share of such authors. Scopus characterized 35% of the journalistic publications as full articles and another 11% as short surveys. 264 papers had received more than 100 citations. 40/41 most-cited papers in 2020-2022 were on hot COVID-19 topics. Of 25 massively prolific authors with > 700 publications in one of these journals, many were highly-cited (median citations 2273), almost all had published little or nothing in the Scopus-indexed literature other than in their main affiliated journal, and their influential writing covered diverse hot topics over the years. Of the 25, only 3 had a PhD degree in any subject matter, and 7 had a Master's degree in journalism. Only the BMJ offered conflicts of interest disclosures for prolific science writers in its website, but even then only 2 of the 25 massively prolific authors disclosed potential conflicts with some specificity. The practice of assigning so much power to non-researchers in shaping scientific discourse should be further debated and disclosures of potential conflicts of interest should be emphasized.
Collapse
Affiliation(s)
- John P. A. Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA 94305 USA
| |
Collapse
|
|
42
|
McDougald WA, Mannheim JG. Understanding the importance of quality control and quality assurance in preclinical PET/CT imaging. EJNMMI Phys 2022; 9:77. [PMID: 36315337 PMCID: PMC9622967 DOI: 10.1186/s40658-022-00503-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Accepted: 10/20/2022] [Indexed: 11/12/2022] Open
Abstract
The fundamental principle of experimental design is to ensure efficiency and efficacy of the performed experiments. Therefore, it behoves the researcher to gain knowledge of the technological equipment to be used. This should include an understanding of the instrument quality control and assurance requirements to avoid inadequate or spurious results due to instrumentation bias whilst improving reproducibility. Here, the important role of preclinical positron emission tomography/computed tomography and the scanner's required quality control and assurance is presented along with the suggested guidelines for quality control and assurance. There are a multitude of factors impeding the continuity and reproducibility of preclinical research data within a single laboratory as well as across laboratories. A more robust experimental design incorporating validation or accreditation of the scanner performance can reduce inconsistencies. Moreover, the well-being and welfare of the laboratory animals being imaged is prime justification for refining experimental designs to include verification of instrumentation quality control and assurance. Suboptimal scanner performance is not consistent with the 3R principle (Replacement, Reduction, and Refinement) and potentially subjects animals to unnecessary harm. Thus, quality assurance and control should be of paramount interest to any scientist conducting animal studies. For this reason, through this work, we intend to raise the awareness of researchers using PET/CT regarding quality control/quality assurance (QC/QA) guidelines and instil the importance of confirming that these are routinely followed. We introduce a basic understanding of the PET/CT scanner, present the purpose of QC/QA as well as provide evidence of imaging data biases caused by lack of QC/QA. This is shown through a review of the literature, QC/QA accepted standard protocols and our research. We also want to encourage researchers to have discussions with the PET/CT facilities manager and/or technicians to develop the optimal designed PET/CT experiment for obtaining their scientific objective. Additionally, this work provides an easy gateway to multiple resources not only for PET/CT knowledge but for guidelines and assistance in preclinical experimental design to enhance scientific integrity of the data and ensure animal welfare.
Collapse
Affiliation(s)
- Wendy A. McDougald
- grid.4305.20000 0004 1936 7988BHF-Centre for Cardiovascular Science, College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, UK ,grid.4305.20000 0004 1936 7988Edinburgh Preclinical Imaging (EPI), Edinburgh Imaging, University of Edinburgh, Edinburgh, UK ,grid.4305.20000 0004 1936 7988Centre for Cardiovascular Science, Queen’s Medical Research Institute, University of Edinburgh, Edinburgh BioQuarter, 47 Little France Crescent, Edinburgh, EH16 4TJ UK
| | - Julia G. Mannheim
- grid.10392.390000 0001 2190 1447Department of Preclinical Imaging and Radiopharmacy, Werner Siemens Imaging Center, Eberhard-Karls University Tübingen, Tübingen, Germany ,grid.10392.390000 0001 2190 1447Cluster of Excellence iFIT (EXC 2180) “Image Guided and Functionally Instructed Tumor Therapies”, University of Tuebingen, Tübingen, Germany
| |
Collapse
|
|
43
|
Schiavone SR, Vazire S. Reckoning With Our Crisis: An Agenda for the Field of Social and Personality Psychology. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2022; 18:710-722. [PMID: 36301777 DOI: 10.1177/17456916221101060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The replication crisis and credibility revolution in the 2010s brought a wave of doubts about the credibility of social and personality psychology. We argue that as a field, we must reckon with the concerns brought to light during this critical decade. How the field responds to this crisis will reveal our commitment to self-correction. If we do not take the steps necessary to address our problems and simply declare the crisis to be over or the problems to be fixed without evidence, we risk further undermining our credibility. To fully reckon with this crisis, we must empirically assess the state of the field to take stock of how credible our science actually is and whether it is improving. We propose an agenda for metascientific research, and we review approaches to empirically evaluate and track where we are as a field (e.g., analyzing the published literature, surveying researchers). We describe one such project (Surveying the Past and Present State of Published Studies in Social and Personality Psychology) underway in our research group. Empirical evidence about the state of our field is necessary if we are to take self-correction seriously and if we hope to avert future crises.
Collapse
Affiliation(s)
| | - Simine Vazire
- Melbourne School of Psychological Sciences, University of Melbourne
| |
Collapse
|
|
44
|
Jaric I, Voelkl B, Clerc M, Schmid MW, Novak J, Rosso M, Rufener R, von Kortzfleisch VT, Richter SH, Buettner M, Bleich A, Amrein I, Wolfer DP, Touma C, Sunagawa S, Würbel H. The rearing environment persistently modulates mouse phenotypes from the molecular to the behavioural level. PLoS Biol 2022; 20:e3001837. [PMID: 36269766 PMCID: PMC9629646 DOI: 10.1371/journal.pbio.3001837] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 11/02/2022] [Accepted: 09/20/2022] [Indexed: 11/07/2022] Open
Abstract
The phenotype of an organism results from its genotype and the influence of the environment throughout development. Even when using animals of the same genotype, independent studies may test animals of different phenotypes, resulting in poor replicability due to genotype-by-environment interactions. Thus, genetically defined strains of mice may respond differently to experimental treatments depending on their rearing environment. However, the extent of such phenotypic plasticity and its implications for the replicability of research findings have remained unknown. Here, we examined the extent to which common environmental differences between animal facilities modulate the phenotype of genetically homogeneous (inbred) mice. We conducted a comprehensive multicentre study, whereby inbred C57BL/6J mice from a single breeding cohort were allocated to and reared in 5 different animal facilities throughout early life and adolescence, before being transported to a single test laboratory. We found persistent effects of the rearing facility on the composition and heterogeneity of the gut microbial community. These effects were paralleled by persistent differences in body weight and in the behavioural phenotype of the mice. Furthermore, we show that environmental variation among animal facilities is strong enough to influence epigenetic patterns in neurons at the level of chromatin organisation. We detected changes in chromatin organisation in the regulatory regions of genes involved in nucleosome assembly, neuronal differentiation, synaptic plasticity, and regulation of behaviour. Our findings demonstrate that common environmental differences between animal facilities may produce facility-specific phenotypes, from the molecular to the behavioural level. Furthermore, they highlight an important limitation of inferences from single-laboratory studies and thus argue that study designs should take environmental background into account to increase the robustness and replicability of findings. The phenotype of an organism results not only from its genotype but also the influence of its environment throughout development. This study shows that common environmental differences between animal facilities can induce substantial variation in the phenotype of mice, thereby highlighting an important limitation of inferences from single-laboratory studies in animal research.
Collapse
Affiliation(s)
- Ivana Jaric
- Animal Welfare Division, Vetsuisse Faculty, University of Bern, Bern, Switzerland
- * E-mail: (IJ); (HW)
| | - Bernhard Voelkl
- Animal Welfare Division, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| | - Melanie Clerc
- Department of Biology, Institute of Microbiology and Swiss Institute of Bioinformatics, ETH Zürich, Zürich, Switzerland
| | | | - Janja Novak
- Animal Welfare Division, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| | - Marianna Rosso
- Animal Welfare Division, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| | - Reto Rufener
- Department of Oncology-Pathology, Karolinska Institutet, Solna, Sweden
| | | | - S. Helene Richter
- Department of Behavioural Biology, University of Münster, Münster, Germany
| | - Manuela Buettner
- Institute for Laboratory Animal Science and Central Animal Facility, Hannover Medical School, Hannover, Germany
| | - André Bleich
- Institute for Laboratory Animal Science and Central Animal Facility, Hannover Medical School, Hannover, Germany
| | - Irmgard Amrein
- Institute of Anatomy, Division of Functional Neuroanatomy, University of Zürich, Zürich, Switzerland; Department of Health Sciences and Technology, ETH Zürich, Zürich, Switzerland
| | - David P. Wolfer
- Institute of Anatomy, Division of Functional Neuroanatomy, University of Zürich, Zürich, Switzerland; Department of Health Sciences and Technology, ETH Zürich, Zürich, Switzerland
| | - Chadi Touma
- Department of Behavioural Biology, Osnabrück University, Osnabrück, Germany
| | - Shinichi Sunagawa
- Department of Biology, Institute of Microbiology and Swiss Institute of Bioinformatics, ETH Zürich, Zürich, Switzerland
| | - Hanno Würbel
- Animal Welfare Division, Vetsuisse Faculty, University of Bern, Bern, Switzerland
- * E-mail: (IJ); (HW)
| |
Collapse
|
|
45
|
Lawson DO, Wang MK, Kim K, Eikelboom R, Rodrigues M, Trapsa D, Thabane L, Moher D. Lessons from the COVID-19 pandemic and recent developments on the communication of clinical trials, publishing practices, and research integrity: in conversation with Dr. David Moher. Trials 2022; 23:671. [PMID: 35978325 PMCID: PMC9383655 DOI: 10.1186/s13063-022-06624-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Accepted: 07/29/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The torrent of research during the coronavirus (COVID-19) pandemic has exposed the persistent challenges with reporting trials, open science practices, and scholarship in academia. These real-world examples provide unique learning opportunities for research methodologists and clinical epidemiologists-in-training. Dr. David Moher, a recognized expert on the science of research reporting and one of the founders of the Consolidated Standards of Reporting Trials (CONSORT) statement, was a guest speaker for the 2021 Hooker Distinguished Visiting Professor Lecture series at McMaster University and shared his insights about these issues. MAIN TEXT This paper covers a discussion on the influence of reporting guidelines on trials and issues with the use of CONSORT as a measure of quality. Dr. Moher also addresses how the overwhelming body of COVID-19 research reflects the "publish or perish" paradigm in academia and why improvement in the reporting of trials requires policy initiatives from research institutions and funding agencies. We also discuss the rise of publication bias and other questionable reporting practices. To combat this, Dr. Moher believes open science and training initiatives led by institutions can foster research integrity, including the trustworthiness of researchers, institutions, and journals, as well as counter threats posed by predatory journals. He highlights how metrics like journal impact factor and quantity of publications also harm research integrity. Dr. Moher also discussed the importance of meta-science, the study of how research is carried out, which can help to evaluate audit and feedback systems and their effect on open science practices. CONCLUSION Dr. Moher advocates for policy to further improve the reporting of trials and health research. The COVID-19 pandemic has exposed how a lack of open science practices and flawed systems incentivizing researchers to publish can harm research integrity. There is a need for a culture shift in assessing careers and "productivity" in academia, and this requires collaborative top-down and bottom-up approaches.
Collapse
Affiliation(s)
- Daeria O Lawson
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Michael K Wang
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, Canada
- Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada
| | - Kevin Kim
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Rachel Eikelboom
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Myanca Rodrigues
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Daniela Trapsa
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
- Department of Medicine, McMaster University, Hamilton, ON, Canada.
- Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada.
- Biostatistics Unit, Father Sean O'Sullivan Research Centre, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada.
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| |
Collapse
|
|
46
|
Rajtmajer SM, Errington TM, Hillary FG. How failure to falsify in high-volume science contributes to the replication crisis. eLife 2022; 11:e78830. [PMID: 35939392 PMCID: PMC9398444 DOI: 10.7554/elife.78830] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 07/28/2022] [Indexed: 11/29/2022] Open
Abstract
The number of scientific papers published every year continues to increase, but scientific knowledge is not progressing at the same rate. Here we argue that a greater emphasis on falsification - the direct testing of strong hypotheses - would lead to faster progress by allowing well-specified hypotheses to be eliminated. We describe an example from neuroscience where there has been little work to directly test two prominent but incompatible hypotheses related to traumatic brain injury. Based on this example, we discuss how building strong hypotheses and then setting out to falsify them can bring greater precision to the clinical neurosciences, and argue that this approach could be beneficial to all areas of science.
Collapse
Affiliation(s)
- Sarah M Rajtmajer
- College of Information Sciences and Technology, The Pennsylvania State UniversityUniversity ParkUnited States
| | | | - Frank G Hillary
- Department of Psychology and the Social Life and Engineering Sciences Imaging Center, The Pennsylvania State UniversityUniversity ParkUnited States
| |
Collapse
|
|
47
|
Purgar M, Klanjscek T, Culina A. Quantifying research waste in ecology. Nat Ecol Evol 2022; 6:1390-1397. [PMID: 35864230 DOI: 10.1038/s41559-022-01820-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 05/31/2022] [Indexed: 11/09/2022]
Abstract
Research inefficiencies can generate huge waste: evidence from biomedical research has shown that most research is avoidably wasted and steps have been taken to tackle this costly problem. Although other scientific fields could also benefit from identifying and quantifying waste and acting to reduce it, no other estimates of research waste are available. Given that ecological issues interweave most of the United Nations Sustainable Development Goals, we argue that tackling research waste in ecology should be prioritized. Our study leads the way. We estimate components of waste in ecological research based on a literature review and a meta-analysis. Shockingly, our results suggest only 11-18% of conducted ecological research reaches its full informative value. All actors within the research system-including academic institutions, policymakers, funders and publishers-have a duty towards science, the environment, study organisms and the public, to urgently act and reduce this considerable yet preventable loss. We discuss potential ways forward and call for two major actions: (1) further research into waste in ecology (and beyond); (2) focused development and implementation of solutions to reduce unused potential of ecological research.
Collapse
Affiliation(s)
- Marija Purgar
- Rudjer Boskovic Institute, Zagreb, Croatia.,Department of Biology, University of Osijek, Osijek, Croatia
| | | | - Antica Culina
- Rudjer Boskovic Institute, Zagreb, Croatia. .,Netherlands Institute of Ecology, NIOO-KNAW, Wageningen, the Netherlands.
| |
Collapse
|
|
48
|
Ioannidis JPA. Educating educators on research on research. PERSPECTIVES ON MEDICAL EDUCATION 2022; 11:137-138. [PMID: 33877586 PMCID: PMC9240132 DOI: 10.1007/s40037-021-00662-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/24/2021] [Indexed: 05/04/2023]
Affiliation(s)
- John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA.
| |
Collapse
|
|
49
|
Meta-research studies should improve and evaluate their own data sharing practices. J Clin Epidemiol 2022; 149:183-189. [PMID: 35623542 DOI: 10.1016/j.jclinepi.2022.05.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Revised: 05/04/2022] [Accepted: 05/16/2022] [Indexed: 11/20/2022]
Abstract
Data sharing is gradually becoming a requirement across all fields of science, owing to its key benefits in verifying the reproducibility of findings and reusing existent data for new purposes. Although meta-research studies are complex, time-consuming and hinge on the availability of data produced and curated by others, there has been little focus on how they make their own data available. This is in stark contrast with the heightened attention data sharing has received in clinical research. Yet, as secondary data users par excellence, meta-researchers are ethically bound to both improving and evaluating data sharing practices, as well as correctly sharing their own data. We contrast particularities of data sharing in meta-research and clinical research, such as benefits, barriers, inadequate and potentially pervasive sharing practices. We conclude with an array of concrete and tailored recommendations for improvement.
Collapse
|
|
50
|
Béchard B, Kimmerle J, Lawarée J, Bédard PO, Straus SE, Ouimet M. The Impact of Information Presentation and Cognitive Dissonance on Processing Systematic Review Summaries: A Randomized Controlled Trial on Bicycle Helmet Legislation. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:6234. [PMID: 35627776 PMCID: PMC9140747 DOI: 10.3390/ijerph19106234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 05/16/2022] [Accepted: 05/18/2022] [Indexed: 12/02/2022]
Abstract
Background: Summaries of systematic reviews are a reference method for the dissemination of research evidence on the effectiveness of public health interventions beyond the scientific community. Motivated reasoning and cognitive dissonance may interfere with readers' ability to process the information included in such summaries. Methods: We conducted a web experiment on a panel of university-educated North Americans (N = 259) using a systematic review of the effectiveness of bicycle helmet legislation as a test case. The outcome variables were the perceived tentativeness of review findings and attitude toward bicycle helmet legislation. We manipulated two types of uncertainty: (i) deficient uncertainty (inclusion vs. non-inclusion of information on limitations of the studies included in the review) and (ii) consensus uncertainty (consensual findings showing legislation effectiveness vs. no evidence of effectiveness). We also examined whether reported expertise in helmet legislation and the frequency of wearing a helmet while cycling interact with the experimental factors. Results: None of the experimental manipulations had a main effect on the perceived tentativeness. The presentation of consensual efficacy findings had a positive main effect on the attitude toward the legislation. Self-reported expertise had a significant main effect on the perceived tentativeness, and exposing participants with reported expertise to results showing a lack of evidence of efficacy increased their favorable attitude toward the legislation. Participants' helmet use was positively associated with their attitude toward the legislation (but not with perceived tentativeness). Helmet use did not interact with the experimental manipulations. Conclusions: Motivated reasoning and cognitive dissonance influence a reader's ability to process information contained in a systematic review summary.
Collapse
Affiliation(s)
- Benoît Béchard
- PolitiCo, School of Psychology, Université Laval, Québec, QC G1V 0A6, Canada;
| | | | - Justin Lawarée
- International Observatory on the Societal Impact of AI and Digital Technology, Department of Political Science, Université Laval, Québec, QC G1V 0A6, Canada;
| | - Pierre-Oliver Bédard
- GC Experimentation Team, Treasury Board of Canada Secretariat, Government of Canada, Ottawa, ON K1A OR5, Canada;
| | - Sharon E. Straus
- Knowledge Translation Program, St. Michael’s Hospital, Toronto, ON M5B 1W8, Canada;
| | - Mathieu Ouimet
- PolitiCo, Department of Political Science, Université Laval, Québec, QC G1V 0A6, Canada;
| |
Collapse
|