|
1
|
Chyrmang G, Barua B, Bora K, Ahmed GN, Das AK, Kakoti L, Lemos B, Mallik S. Self-HER2Net: A generative self-supervised framework for HER2 classification in IHC histopathology of breast cancer. Pathol Res Pract 2025; 270:155961. [PMID: 40245674 DOI: 10.1016/j.prp.2025.155961] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2025] [Accepted: 04/08/2025] [Indexed: 04/19/2025]
Abstract
Breast cancer is a significant global health concern, where precise identification of proteins like Human Epidermal Growth Factor Receptor 2 (HER2) in cancer cells via Immunohistochemistry (IHC) is pivotal for treatment decisions. HER2 overexpression is evaluated through HER2 scoring on a scale from 0 to 3 + based on staining patterns and intensity. Recent efforts have been made to automate HER2 scoring using image processing and AI techniques. However, existing methods require large manually annotated datasets as these follow supervised learning paradigms. Therefore, we proposed a generative self-supervised learning (SSL) framework "Self-HER2Net" for the classification of HER2 scoring, to reduce dependence on large manually annotated data by leveraging one of best performing four novel generative self-supervised tasks, that we proposed. The first two SSL tasks HER2hsl and HER2hsv are domain-agnostic and the other two HER2dab and HER2hae are domain-specific SSL tasks focusing on domain-agnostic and domain-specific staining patterns and intensity representation. Our approach is evaluated under different budget scenarios (2 %, 15 %, & 100 % labeled datasets) and also out distribution test. For tile-level assessment, HER2hsv achieved the best performance with AUC-ROC of 0.965 ± 0.037. Our self-supervised learning approach shows potential for application in scenarios with limited annotated data for HER2 analysis.
Collapse
Affiliation(s)
- Genevieve Chyrmang
- Department of Computer Science and Information Technology, Cotton University, Guwahati, Assam, India.
| | - Barun Barua
- Department of Computer Science and Information Technology, Cotton University, Guwahati, Assam, India.
| | - Kangkana Bora
- Department of Computer Science and Information Technology, Cotton University, Guwahati, Assam, India.
| | - Gazi N Ahmed
- North East Cancer Hospital and Research Institute, Guwahati, Assam, India.
| | - Anup Kr Das
- Arya Wellness centre, Guwahati, Assam, India.
| | | | - Bernardo Lemos
- Department of Environmental Health, Harvard T H Chan School of Public Health, Boston, MA 02115, USA; Department of Pharmacology & Toxicology, University of Arizona, AZ 85721, USA.
| | - Saurav Mallik
- Department of Environmental Health, Harvard T H Chan School of Public Health, Boston, MA 02115, USA; Department of Pharmacology & Toxicology, University of Arizona, AZ 85721, USA.
| |
Collapse
|
|
2
|
Marra A, Morganti S, Pareja F, Campanella G, Bibeau F, Fuchs T, Loda M, Parwani A, Scarpa A, Reis-Filho JS, Curigliano G, Marchiò C, Kather JN. Artificial intelligence entering the pathology arena in oncology: current applications and future perspectives. Ann Oncol 2025:S0923-7534(25)00112-7. [PMID: 40307127 DOI: 10.1016/j.annonc.2025.03.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2024] [Revised: 02/19/2025] [Accepted: 03/07/2025] [Indexed: 05/02/2025] Open
Abstract
BACKGROUND Artificial intelligence (AI) is rapidly transforming the fields of pathology and oncology, offering novel opportunities for advancing diagnosis, prognosis, and treatment of cancer. METHODS Through a systematic review-based approach, the representatives from the European Society for Medical Oncology (ESMO) Precision Oncology Working Group (POWG) and international experts identified studies in pathology and oncology that applied AI-based algorithms for tumour diagnosis, molecular biomarker detection, and cancer prognosis assessment. These findings were synthesised to provide a comprehensive overview of current AI applications and future directions in cancer pathology. RESULTS The integration of AI tools in digital pathology is markedly improving the accuracy and efficiency of image analysis, allowing for automated tumour detection and classification, identification of prognostic molecular biomarkers, and prediction of treatment response and patient outcomes. Several barriers for the adoption of AI in clinical workflows, such as data availability, explainability, and regulatory considerations, still persist. There are currently no prognostic or predictive AI-based biomarkers supported by level IA or IB evidence. The ongoing advancements in AI algorithms, particularly foundation models, generalist models and transformer-based deep learning, offer immense promise for the future of cancer research and care. AI is also facilitating the integration of multi-omics data, leading to more precise patient stratification and personalised treatment strategies. CONCLUSIONS The application of AI in pathology is poised to not only enhance the accuracy and efficiency of cancer diagnosis and prognosis but also facilitate the development of personalised treatment strategies. Although barriers to implementation remain, ongoing research and development in this field coupled with addressing ethical and regulatory considerations will likely lead to a future where AI plays an integral role in cancer management and precision medicine. The continued evolution and adoption of AI in pathology and oncology are anticipated to reshape the landscape of cancer care, heralding a new era of precision medicine and improved patient outcomes.
Collapse
Affiliation(s)
- A Marra
- Division of Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, Milan, Italy
| | - S Morganti
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, USA; Department of Medicine, Harvard Medical School, Boston, USA; Gerstner Center for Cancer Diagnostics, Broad Institute of MIT and Harvard, Boston, USA
| | - F Pareja
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, USA
| | - G Campanella
- Hasso Plattner Institute for Digital Health, Mount Sinai Medical School, New York, USA; Department of AI and Human Health, Icahn School of Medicine at Mount Sinai, New York, USA
| | - F Bibeau
- Department of Pathology, University Hospital of Besançon, Besancon, France
| | - T Fuchs
- Hasso Plattner Institute for Digital Health, Mount Sinai Medical School, New York, USA; Department of AI and Human Health, Icahn School of Medicine at Mount Sinai, New York, USA
| | - M Loda
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, USA; Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK; Department of Oncologic Pathology, Dana-Farber Cancer Institute and Harvard Medical School, Boston, USA
| | - A Parwani
- Department of Pathology, Wexner Medical Center, Ohio State University, Columbus, USA
| | - A Scarpa
- Department of Diagnostics and Public Health, Section of Pathology, University and Hospital Trust of Verona, Verona, Italy; ARC-Net Research Center, University of Verona, Verona, Italy
| | - J S Reis-Filho
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, USA
| | - G Curigliano
- Division of Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, Milan, Italy; Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - C Marchiò
- Candiolo Cancer Institute, FPO IRCCS, Candiolo, Italy; Department of Medical Sciences, University of Turin, Turin, Italy
| | - J N Kather
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany; Department of Medicine I, University Hospital and Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany; Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
|
3
|
Atallah NM, Makhlouf S, Nabil M, Ibrahim A, Toss MS, Mongan NP, Rakha E. Characterisation of HER2-Driven Morphometric Signature in Breast Cancer and Prediction of Risk of Recurrence. Cancer Med 2025; 14:e70852. [PMID: 40243160 PMCID: PMC12004275 DOI: 10.1002/cam4.70852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2024] [Revised: 03/17/2025] [Accepted: 03/26/2025] [Indexed: 04/18/2025] Open
Abstract
INTRODUCTION Human epidermal growth factor receptor 2-positive (HER2-positive) breast cancer (BC) is a heterogeneous disease. In this study, we hypothesised that the degree of HER2 oncogenic activity, and hence response to anti-HER2 therapy is translated into a morphological signature that can be of prognostic/predictive value. METHODS We developed a HER2-driven signature based on a set of morphometric features identified through digital image analysis and visual assessment in a sizable cohort of BC patients. HER2-enriched molecular sub-type (HER2-E) was used for validation, and pathway enrichment analysis was performed to assess HER2 pathway activity in the signature-positive cases. The predictive utility of this signature was evaluated in post-adjuvant HER2-positive BC patients. RESULTS A total of 57 morphometric features were evaluated; of them, 22 features were significantly associated with HER2 positivity. HER2 IHC score 3+/oestrogen receptor-negative tumours were significantly associated with HER2-related morphometric features compared to other HER2 classes including HER2 IHC 2+ with gene amplification, and they showed the least intra-tumour morphological heterogeneity. Tumours displaying HER2-driven morphometric signature showed the strongest association with PAM50 HER2-E sub-type and were enriched with ERBB signalling pathway compared to signature-negative cases. BC patients with positive HER2 morphometric signature showed prolonged distant metastasis-free survival post-adjuvant anti-HER2 therapy (p = 0.007). The clinico-morphometric prognostic index demonstrated an 87% accuracy in predicting recurrence risk. CONCLUSION Our findings underscore the strong prognostic and predictive correlation between HER2 histo-morphometric features and response to targeted anti-HER2 therapy.
Collapse
Affiliation(s)
- N. M. Atallah
- Translational Medical Science, School of MedicineThe University of Nottingham and Nottingham University Hospitals NHS TrustNottinghamUK
- Department of Pathology, Faculty of MedicineMenoufia UniversityShebin El‐KomEgypt
| | - S. Makhlouf
- Translational Medical Science, School of MedicineThe University of Nottingham and Nottingham University Hospitals NHS TrustNottinghamUK
- Department of Pathology, Faculty of MedicineAssiut UniversityAssuitEgypt
| | - M. Nabil
- Department of Computer Science, Faculty of MedicineMenoufia UniversityShebin El‐KomEgypt
| | - A. Ibrahim
- Translational Medical Science, School of MedicineThe University of Nottingham and Nottingham University Hospitals NHS TrustNottinghamUK
- Department of PathologySuez Canal UniversityIsmailiaEgypt
| | - M. S. Toss
- Translational Medical Science, School of MedicineThe University of Nottingham and Nottingham University Hospitals NHS TrustNottinghamUK
- Histopathology DepartmentRoyal Hallamshire Hospital, Sheffield Teaching Hospitals NHS Foundation TrustSheffieldUK
| | - N. P. Mongan
- School of Veterinary Medicine and SciencesUniversity of NottinghamSutton BoningtonUK
- Department of PharmacologyWeill Cornell MedicineNew YorkNew YorkUSA
| | - E. Rakha
- Translational Medical Science, School of MedicineThe University of Nottingham and Nottingham University Hospitals NHS TrustNottinghamUK
- Department of Pathology, Faculty of MedicineMenoufia UniversityShebin El‐KomEgypt
- Pathology DepartmentHamad Medical CorporationDohaQatar
| |
Collapse
|
|
4
|
Öttl M, Steenpass J, Wilm F, Qiu J, Rübner M, Lang-Schwarz C, Taverna C, Tava F, Hartmann A, Huebner H, Beckmann MW, Fasching PA, Maier A, Erber R, Breininger K. Fully automatic HER2 tissue segmentation for interpretable HER2 scoring. J Pathol Inform 2025; 17:100435. [PMID: 40236564 PMCID: PMC11999220 DOI: 10.1016/j.jpi.2025.100435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2025] [Revised: 03/03/2025] [Accepted: 03/09/2025] [Indexed: 04/17/2025] Open
Abstract
Breast cancer is the most common cancer in women, with HER2 (human epidermal growth factor receptor 2) overexpression playing a critical role in regulating cell growth and division. HER2 status, assessed according to established scoring guidelines, offers important information for treatment selection. However, the complexity of the task leads to variability in human rater assessments. In this work, we propose a fully automated, interpretable HER2 scoring pipeline based on pixel-level semantic segmentations, designed to align with clinical guidelines. Using polygon annotations, our method balances annotation effort with the ability to capture fine-grained details and larger structures, such as non-invasive tumor tissue. To enhance HER2 segmentation, we propose the use of a Wasserstein Dice loss to model class relationships, ensuring robust segmentation and HER2 scoring performance. Additionally, based on observations of pathologists' behavior in clinical practice, we propose a calibration step to the scoring rules, which positively impacts the accuracy and consistency of automated HER2 scoring. Our approach achieves an F1 score of 0.832 on HER2 scoring, demonstrating its effectiveness. This work establishes a potent segmentation pipeline that can be further leveraged to analyze HER2 expression in breast cancer tissue.
Collapse
Affiliation(s)
- Mathias Öttl
- Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jana Steenpass
- Institute of Pathology, University Hospital Erlangen, Erlangen, Germany
| | - Frauke Wilm
- Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jingna Qiu
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Matthias Rübner
- Department of Gynecology and Obstetrics, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | | - Cecilia Taverna
- Surgical Pathology Unit, Azienda Sanitaria Locale, Presidio Ospedaliero, Ospedale San Giacomo, Novi Ligure, Italy
| | - Francesca Tava
- Surgical Pathology Unit, Azienda Sanitaria Locale, Presidio Ospedaliero, Ospedale San Giacomo, Novi Ligure, Italy
| | - Arndt Hartmann
- Institute of Pathology, University Hospital Erlangen, Erlangen, Germany
| | - Hanna Huebner
- Department of Gynecology and Obstetrics, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Matthias W. Beckmann
- Department of Gynecology and Obstetrics, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Peter A. Fasching
- Department of Gynecology and Obstetrics, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Andreas Maier
- Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Ramona Erber
- Institute of Pathology, University Hospital Erlangen, Erlangen, Germany
- Institute of Pathology, University Regensburg, Regensburg, Germany
| | - Katharina Breininger
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Center for AI and Data Science (CAIDAS), Universität Würzburg, Würzburg, Germany
| |
Collapse
|
|
5
|
Wu S, Shang J, Li Z, Liu H, Xu X, Zhang Z, Wang Y, Zhao M, Yue M, He J, Miao J, Sang Y, Yan J, Pang W, Shao Q, Zhang Y, Zhao M, Liu X, Wang P, Cai C, Liu B, Wang X, Liu Y. Interobserver consistency and diagnostic challenges in HER2-ultralow breast cancer: a multicenter study. ESMO Open 2025; 10:104127. [PMID: 39891991 PMCID: PMC11841085 DOI: 10.1016/j.esmoop.2024.104127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2024] [Revised: 11/20/2024] [Accepted: 12/23/2024] [Indexed: 02/03/2025] Open
Abstract
BACKGROUND Recent advancements in novel antibody-drug conjugates (ADCs) have demonstrated efficacy in patients with human epidermal growth factor receptor 2 (HER2)-ultralow breast cancer (BC), expanding the eligibility for anti-HER2 targeted therapy to include some patients previously categorized as HER2 immunohistochemistry (IHC) 0. This expansion underscores the need for pathologists to accurately differentiate HER2-null and HER2-ultralow. MATERIALS AND METHODS Thirty-six pathologists from four centers nationwide conducted microscopic visual assessments on HER2 IHC slides from 50 consecutive BC surgical specimens, all previously diagnosed as HER2 IHC 0. RESULTS The interobserver consistency in differentiating HER2-null from HER2-ultralow, measured by Fleiss κ, was only 0.230-lower than the consistency for combined HER2 IHC 0 cases (Fleiss κ = 0.344) and binary classification (HER2-null versus HER2-non-null; Fleiss κ = 0.292). High agreement for HER2-null versus HER2-ultralow differentiation was achieved in only 4% of cases, while combining them into HER2 IHC 0 raised high agreement cases to 32%, higher than the 18% seen in the binary classification. Consensus among the 36 pathologists aligned with historical scores in 72% of cases; however, when subdividing HER2 IHC 0 into HER2-null and HER2-ultralow, the consistency dropped to 54%. CONCLUSIONS The low consistency among pathologists in distinguishing HER2-null, -ultralow, and 1+ cases may impact patient eligibility for new ADC therapies. To address this challenge, there is a need for improved detection methods, artificial intelligence-assisted quantitative assessments, and larger clinical datasets to refine the definition of HER2-ultralow.
Collapse
Affiliation(s)
- S Wu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - J Shang
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - Z Li
- Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, USA
| | - H Liu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - X Xu
- Department of Pathology, Xingtai People's Hospital, Xingtai, China
| | - Z Zhang
- Department of Pathology, Cangzhou People's Hospital, Cangzhou, China
| | - Y Wang
- Department of Pathology, Affiliated Hospital of Hebei University, Baoding, China
| | - M Zhao
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - M Yue
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - J He
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - J Miao
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - Y Sang
- Department of Pathology, Cangzhou People's Hospital, Cangzhou, China
| | - J Yan
- Department of Pathology, Xingtai People's Hospital, Xingtai, China
| | - W Pang
- Department of Pathology, Xingtai People's Hospital, Xingtai, China
| | - Q Shao
- Department of Pathology, Cangzhou People's Hospital, Cangzhou, China
| | - Y Zhang
- Department of Pathology, Cangzhou People's Hospital, Cangzhou, China
| | - M Zhao
- Department of Pathology, Xingtai People's Hospital, Xingtai, China
| | - X Liu
- Department of Pathology, Xingtai People's Hospital, Xingtai, China
| | - P Wang
- Department of Pathology, Cangzhou People's Hospital, Cangzhou, China
| | - C Cai
- Department of Pathology, Cangzhou People's Hospital, Cangzhou, China
| | - B Liu
- Department of Pathology, Xingtai People's Hospital, Xingtai, China
| | - X Wang
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China
| | - Y Liu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China.
| |
Collapse
|
|
6
|
Fountzilas E, Pearce T, Baysal MA, Chakraborty A, Tsimberidou AM. Convergence of evolving artificial intelligence and machine learning techniques in precision oncology. NPJ Digit Med 2025; 8:75. [PMID: 39890986 PMCID: PMC11785769 DOI: 10.1038/s41746-025-01471-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Accepted: 01/19/2025] [Indexed: 02/03/2025] Open
Abstract
The confluence of new technologies with artificial intelligence (AI) and machine learning (ML) analytical techniques is rapidly advancing the field of precision oncology, promising to improve diagnostic approaches and therapeutic strategies for patients with cancer. By analyzing multi-dimensional, multiomic, spatial pathology, and radiomic data, these technologies enable a deeper understanding of the intricate molecular pathways, aiding in the identification of critical nodes within the tumor's biology to optimize treatment selection. The applications of AI/ML in precision oncology are extensive and include the generation of synthetic data, e.g., digital twins, in order to provide the necessary information to design or expedite the conduct of clinical trials. Currently, many operational and technical challenges exist related to data technology, engineering, and storage; algorithm development and structures; quality and quantity of the data and the analytical pipeline; data sharing and generalizability; and the incorporation of these technologies into the current clinical workflow and reimbursement models.
Collapse
Affiliation(s)
- Elena Fountzilas
- Department of Medical Oncology, St Luke's Clinic, Panorama, Thessaloniki, Greece
| | | | - Mehmet A Baysal
- Department of Investigational Cancer Therapeutics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd., Houston, TX, USA
| | - Abhijit Chakraborty
- Department of Investigational Cancer Therapeutics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd., Houston, TX, USA
| | - Apostolia M Tsimberidou
- Department of Investigational Cancer Therapeutics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd., Houston, TX, USA.
| |
Collapse
|
|
7
|
Chyrmang G, Bora K, Das AK, Ahmed GN, Kakoti L. Insights into AI advances in immunohistochemistry for effective breast cancer treatment: a literature review of ER, PR, and HER2 scoring. Curr Med Res Opin 2025; 41:115-134. [PMID: 39705612 DOI: 10.1080/03007995.2024.2445142] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2024] [Revised: 12/15/2024] [Accepted: 12/17/2024] [Indexed: 12/22/2024]
Abstract
Breast cancer is a significant health challenge, with accurate and timely diagnosis being critical to effective treatment. Immunohistochemistry (IHC) staining is a widely used technique for the evaluation of breast cancer markers, but manual scoring is time-consuming and can be subject to variability. With the rise of Artificial Intelligence (AI), there is an increasing interest in using machine learning and deep learning approaches to automate the scoring of ER, PR, and HER2 biomarkers in IHC-stained images for effective treatment. This narrative literature review focuses on AI-based techniques for the automated scoring of breast cancer markers in IHC-stained images, specifically Allred, Histochemical (H-Score) and HER2 scoring. We aim to identify the current state-of-the-art approaches, challenges, and potential future research prospects for this area of study. By conducting a comprehensive review of the existing literature, we aim to contribute to the ultimate goal of improving the accuracy and efficiency of breast cancer diagnosis and treatment.
Collapse
Affiliation(s)
- Genevieve Chyrmang
- Department of Computer Science and Information Technology, Cotton University, Guwahati, Assam, India
| | - Kangkana Bora
- Department of Computer Science and Information Technology, Cotton University, Guwahati, Assam, India
| | - Anup Kr Das
- Arya Wellness Centre, Guwahati, Assam, India
| | - Gazi N Ahmed
- North East Cancer Hospital and Research Institute, Guwahati, Assam, India
| | | |
Collapse
|
|
8
|
Brevet M, Li Z, Parwani A. Computational pathology in the identification of HER2-low breast cancer: Opportunities and challenges. J Pathol Inform 2024; 15:100343. [PMID: 38125925 PMCID: PMC10730362 DOI: 10.1016/j.jpi.2023.100343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 09/18/2023] [Accepted: 11/01/2023] [Indexed: 12/23/2023] Open
Abstract
For the past 2 decades, pathologists have been accustomed to reporting the HER2 status of breast cancer as either positive or negative, based on HER2 IHC. Today, however, there is a clinical imperative to employ a 3-tier approach to interpreting HER2 IHC that can also identify tumours categorised as HER2-low. Meeting this need for a finer degree of discrimination may be challenging, and in this article, we consider the potential for the integration of computational approaches to support pathologists in achieving accurate and reproducible HER2 IHC scoring as well as outlining some of the practicalities involved.
Collapse
Affiliation(s)
| | - Zaibo Li
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Anil Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| |
Collapse
|
|
9
|
Wu Z, Park J, Steiner PR, Zhu B, Zhang JXJ. Generative adversarial network model to classify human induced pluripotent stem cell-cardiomyocytes based on maturation level. Sci Rep 2024; 14:27016. [PMID: 39506030 PMCID: PMC11541591 DOI: 10.1038/s41598-024-77943-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Accepted: 10/28/2024] [Indexed: 11/08/2024] Open
Abstract
Our study develops a generative adversarial network (GAN)-based method that generates faithful synthetic image data of human cardiomyocytes at varying stages in their maturation process, as a tool to significantly enhance the classification accuracy of cells and ultimately assist the throughput of computational analysis of cellular structure and functions. Human induced pluripotent stem cell derived cardiomyocytes (hiPSC-CMs) were cultured on micropatterned collagen coated hydrogels of physiological stiffnesses to facilitate maturation and optical measurements were performed for their structural and functional analyses. Control groups were cultured on collagen coated glass well plates. These image recordings were used as the real data to train the GAN model. The results show the GAN approach is able to replicate true features from the real data, and inclusion of such synthetic data significantly improves the classification accuracy compared to usage of only real experimental data that is often limited in scale and diversity. The proposed model outperformed four conventional machine learning algorithms with respect to improved data generalization ability and data classification by incorporating synthetic data. This work demonstrates the importance of integrating synthetic data in situations where there are limited sample sizes and thus, effectively addresses the challenges imposed by data availability.
Collapse
Affiliation(s)
- Ziqian Wu
- Thayer School of Engineering, Dartmouth College, Hanover, 03755, USA
| | - Jiyoon Park
- Thayer School of Engineering, Dartmouth College, Hanover, 03755, USA
| | - Paul R Steiner
- Geisel School of Medicine, Dartmouth College, Hanover, 03755, USA
| | - Bo Zhu
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, 30332, USA
| | - John X J Zhang
- Thayer School of Engineering, Dartmouth College, Hanover, 03755, USA.
| |
Collapse
|
|
10
|
Dunenova G, Kalmataeva Z, Kaidarova D, Dauletbaev N, Semenova Y, Mansurova M, Grjibovski A, Kassymbekova F, Sarsembayev A, Semenov D, Glushkova N. The Performance and Clinical Applicability of HER2 Digital Image Analysis in Breast Cancer: A Systematic Review. Cancers (Basel) 2024; 16:2761. [PMID: 39123488 PMCID: PMC11311684 DOI: 10.3390/cancers16152761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2024] [Revised: 07/28/2024] [Accepted: 07/30/2024] [Indexed: 08/12/2024] Open
Abstract
This systematic review aims to address the research gap in the performance of computational algorithms for the digital image analysis of HER2 images in clinical settings. While numerous studies have explored various aspects of these algorithms, there is a lack of comprehensive evaluation regarding their effectiveness in real-world clinical applications. We conducted a search of the Web of Science and PubMed databases for studies published from 31 December 2013 to 30 June 2024, focusing on performance effectiveness and components such as dataset size, diversity and source, ground truth, annotation, and validation methods. The study was registered with PROSPERO (CRD42024525404). Key questions guiding this review include the following: How effective are current computational algorithms at detecting HER2 status in digital images? What are the common validation methods and dataset characteristics used in these studies? Is there standardization of algorithm evaluations of clinical applications that can improve the clinical utility and reliability of computational tools for HER2 detection in digital image analysis? We identified 6833 publications, with 25 meeting the inclusion criteria. The accuracy rate with clinical datasets varied from 84.19% to 97.9%. The highest accuracy was achieved on the publicly available Warwick dataset at 98.8% in synthesized datasets. Only 12% of studies used separate datasets for external validation; 64% of studies used a combination of accuracy, precision, recall, and F1 as a set of performance measures. Despite the high accuracy rates reported in these studies, there is a notable absence of direct evidence supporting their clinical application. To facilitate the integration of these technologies into clinical practice, there is an urgent need to address real-world challenges and overreliance on internal validation. Standardizing study designs on real clinical datasets can enhance the reliability and clinical applicability of computational algorithms in improving the detection of HER2 cancer.
Collapse
Affiliation(s)
- Gauhar Dunenova
- Department of Epidemiology, Biostatistics and Evidence-Based Medicine, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
| | - Zhanna Kalmataeva
- Rector Office, Asfendiyarov Kazakh National Medical University, Almaty 050000, Kazakhstan;
| | - Dilyara Kaidarova
- Kazakh Research Institute of Oncology and Radiology, Almaty 050022, Kazakhstan;
| | - Nurlan Dauletbaev
- Department of Internal, Respiratory and Critical Care Medicine, Philipps University of Marburg, 35037 Marburg, Germany;
- Department of Pediatrics, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC H4A 3J1, Canada
- Faculty of Medicine and Health Care, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
| | - Yuliya Semenova
- School of Medicine, Nazarbayev University, Astana 010000, Kazakhstan;
| | - Madina Mansurova
- Department of Artificial Intelligence and Big Data, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan;
| | - Andrej Grjibovski
- Central Scientific Research Laboratory, Northern State Medical University, Arkhangelsk 163000, Russia;
- Department of Epidemiology and Modern Vaccination Technologies, I.M. Sechenov First Moscow State Medical University, Moscow 105064, Russia
- Department of Biology, Ecology and Biotechnology, Northern (Arctic) Federal University, Arkhangelsk 163000, Russia
- Department of Health Policy and Management, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
| | - Fatima Kassymbekova
- Department of Public Health and Social Sciences, Kazakhstan Medical University “KSPH”, Almaty 050060, Kazakhstan;
| | - Aidos Sarsembayev
- School of Digital Technologies, Almaty Management University, Almaty 050060, Kazakhstan;
- Health Research Institute, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan;
| | - Daniil Semenov
- Computer Science and Engineering Program, Astana IT University, Astana 020000, Kazakhstan;
| | - Natalya Glushkova
- Department of Epidemiology, Biostatistics and Evidence-Based Medicine, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
- Health Research Institute, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan;
| |
Collapse
|
|
11
|
Selcuk SY, Yang X, Bai B, Zhang Y, Li Y, Aydin M, Unal AF, Gomatam A, Guo Z, Angus DM, Kolodney G, Atlan K, Haran TK, Pillar N, Ozcan A. Automated HER2 Scoring in Breast Cancer Images Using Deep Learning and Pyramid Sampling. BME FRONTIERS 2024; 5:0048. [PMID: 39045139 PMCID: PMC11265840 DOI: 10.34133/bmef.0048] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Accepted: 06/14/2024] [Indexed: 07/25/2024] Open
Abstract
Objective and Impact Statement: Human epidermal growth factor receptor 2 (HER2) is a critical protein in cancer cell growth that signifies the aggressiveness of breast cancer (BC) and helps predict its prognosis. Here, we introduce a deep learning-based approach utilizing pyramid sampling for the automated classification of HER2 status in immunohistochemically (IHC) stained BC tissue images. Introduction: Accurate assessment of IHC-stained tissue slides for HER2 expression levels is essential for both treatment guidance and understanding of cancer mechanisms. Nevertheless, the traditional workflow of manual examination by board-certified pathologists encounters challenges, including inter- and intra-observer inconsistency and extended turnaround times. Methods: Our deep learning-based method analyzes morphological features at various spatial scales, efficiently managing the computational load and facilitating a detailed examination of cellular and larger-scale tissue-level details. Results: This approach addresses the tissue heterogeneity of HER2 expression by providing a comprehensive view, leading to a blind testing classification accuracy of 84.70%, on a dataset of 523 core images from tissue microarrays. Conclusion: This automated system, proving reliable as an adjunct pathology tool, has the potential to enhance diagnostic precision and evaluation speed, and might substantially impact cancer treatment planning.
Collapse
Affiliation(s)
- Sahan Yoruc Selcuk
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Xilin Yang
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Bijie Bai
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Yuzhu Li
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Musa Aydin
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Aras Firat Unal
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Aditya Gomatam
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Zhen Guo
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Darrow Morgan Angus
- Department of Pathology and Laboratory Medicine,
University of California at Davis, Sacramento, CA, USA
| | | | - Karine Atlan
- Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | | | - Nir Pillar
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department,
University of California, Los Angeles, Los Angeles, CA, USA
- California NanoSystems Institute,
University of California, Los Angeles, Los Angeles, CA, USA
- David Geffen School of Medicine,
University of California, Los Angeles, Los Angeles, CA, USA
| |
Collapse
|
|
12
|
Wu S, Li X, Miao J, Xian D, Yue M, Liu H, Fan S, Wei W, Liu Y. Artificial intelligence for assisted HER2 immunohistochemistry evaluation of breast cancer: A systematic review and meta-analysis. Pathol Res Pract 2024; 260:155472. [PMID: 39053133 DOI: 10.1016/j.prp.2024.155472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/22/2024] [Revised: 07/05/2024] [Accepted: 07/14/2024] [Indexed: 07/27/2024]
Abstract
Accurate assessment of HER2 expression in tumor tissue is crucial for determining HER2-targeted treatment options. Nevertheless, pathologists' assessments of HER2 status are less objective than automated, computer-based evaluations. Artificial Intelligence (AI) promises enhanced accuracy and reproducibility in HER2 interpretation. This study aimed to systematically evaluate current AI algorithms for HER2 immunohistochemical diagnosis, offering insights to guide the development of more adaptable algorithms in response to evolving HER2 assessment practices. A comprehensive data search of the PubMed, Embase, Cochrane, and Web of Science databases was conducted using a combination of subject terms and free text. A total of 4994 computational pathology articles published from inception to September 2023 identifying HER2 expression in breast cancer were retrieved. After applying predefined inclusion and exclusion criteria, seven studies were selected. These seven studies comprised 6867 HER2 identification tasks, with two studies employing the HER2-CONNECT algorithm, two using the CNN algorithm, one with the multi-class logistic regression algorithm, and two using the HER2 4B5 algorithm. AI's sensitivity and specificity for distinguishing HER2 0/1+ were 0.98 [0.92-0.99] and 0.92 [0.80-0.97] respectively. For distinguishing HER2 2+, the sensitivity and specificity were 0.78 [0.50-0.92] and 0.98 [0.93-0.99], respectively. For HER2 3+ distinction, AI exhibited a sensitivity of 0.99 [0.98-1.00] and specificity of 0.99 [0.97-1.00]. Furthermore, due to the lack of HER2-targeted therapies for HER2-negative patients in the past, pathologists may have neglected to distinguish between HER2 0 and 1+, leaving room for improvement in the performance of artificial intelligence (AI) in this differentiation. AI excels in automating the assessment of HER2 immunohistochemistry, showing promising results despite slight variations in performance across different HER2 status. While incorporating AI algorithms into the pathology workflow for HER2 assessment poses challenges in standardization, application patterns, and ethical considerations, ongoing advancements suggest its potential as a widely effective tool for pathologists in clinical practice in the near future.
Collapse
Affiliation(s)
- Si Wu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, No. 12 Jiankang Road, Shijiazhuang, Hebei 050011, China
| | - Xiang Li
- Medical Affairs Department, Betrue AI Lab, Guangzhou 510700, China
| | - Jiaxian Miao
- Department of Pathology, The Fourth Hospital of Hebei Medical University, No. 12 Jiankang Road, Shijiazhuang, Hebei 050011, China
| | - Dongyi Xian
- Medical Affairs Department, Betrue AI Lab, Guangzhou 510700, China
| | - Meng Yue
- Department of Pathology, The Fourth Hospital of Hebei Medical University, No. 12 Jiankang Road, Shijiazhuang, Hebei 050011, China
| | - Hongbo Liu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, No. 12 Jiankang Road, Shijiazhuang, Hebei 050011, China
| | - Shishun Fan
- Department of Pathology, The Fourth Hospital of Hebei Medical University, No. 12 Jiankang Road, Shijiazhuang, Hebei 050011, China
| | - Weiwei Wei
- Medical Affairs Department, Betrue AI Lab, Guangzhou 510700, China
| | - Yueping Liu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, No. 12 Jiankang Road, Shijiazhuang, Hebei 050011, China.
| |
Collapse
|
|
13
|
Lorenzo G, Ahmed SR, Hormuth DA, Vaughn B, Kalpathy-Cramer J, Solorio L, Yankeelov TE, Gomez H. Patient-Specific, Mechanistic Models of Tumor Growth Incorporating Artificial Intelligence and Big Data. Annu Rev Biomed Eng 2024; 26:529-560. [PMID: 38594947 DOI: 10.1146/annurev-bioeng-081623-025834] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/11/2024]
Abstract
Despite the remarkable advances in cancer diagnosis, treatment, and management over the past decade, malignant tumors remain a major public health problem. Further progress in combating cancer may be enabled by personalizing the delivery of therapies according to the predicted response for each individual patient. The design of personalized therapies requires the integration of patient-specific information with an appropriate mathematical model of tumor response. A fundamental barrier to realizing this paradigm is the current lack of a rigorous yet practical mathematical theory of tumor initiation, development, invasion, and response to therapy. We begin this review with an overview of different approaches to modeling tumor growth and treatment, including mechanistic as well as data-driven models based on big data and artificial intelligence. We then present illustrative examples of mathematical models manifesting their utility and discuss the limitations of stand-alone mechanistic and data-driven models. We then discuss the potential of mechanistic models for not only predicting but also optimizing response to therapy on a patient-specific basis. We describe current efforts and future possibilities to integrate mechanistic and data-driven models. We conclude by proposing five fundamental challenges that must be addressed to fully realize personalized care for cancer patients driven by computational models.
Collapse
Affiliation(s)
- Guillermo Lorenzo
- Oden Institute for Computational Engineering and Sciences, University of Texas, Austin, Texas, USA
- Department of Civil Engineering and Architecture, University of Pavia, Pavia, Italy
| | - Syed Rakin Ahmed
- Geisel School of Medicine, Dartmouth College, Hanover, New Hampshire, USA
- Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
- Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Massachusetts, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - David A Hormuth
- Livestrong Cancer Institutes, University of Texas, Austin, Texas, USA
- Oden Institute for Computational Engineering and Sciences, University of Texas, Austin, Texas, USA
| | - Brenna Vaughn
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA;
| | | | - Luis Solorio
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA;
| | - Thomas E Yankeelov
- Department of Imaging Physics, MD Anderson Cancer Center, Houston, Texas, USA
- Department of Biomedical Engineering, Department of Oncology, and Department of Diagnostic Medicine, University of Texas, Austin, Texas, USA
- Livestrong Cancer Institutes, University of Texas, Austin, Texas, USA
- Oden Institute for Computational Engineering and Sciences, University of Texas, Austin, Texas, USA
| | - Hector Gomez
- School of Mechanical Engineering and Purdue Center for Cancer Research, Purdue University, West Lafayette, Indiana, USA
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA;
| |
Collapse
|
|
14
|
Chang J, Hatfield B. Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond. Adv Cancer Res 2024; 161:431-478. [PMID: 39032956 DOI: 10.1016/bs.acr.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/23/2024]
Abstract
The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.
Collapse
Affiliation(s)
- Justin Chang
- Virginia Commonwealth University Health System, Richmond, VA, United States
| | - Bryce Hatfield
- Virginia Commonwealth University Health System, Richmond, VA, United States.
| |
Collapse
|
|
15
|
Yücel Z, Akal F, Oltulu P. Automated AI-based grading of neuroendocrine tumors using Ki-67 proliferation index: comparative evaluation and performance analysis. Med Biol Eng Comput 2024; 62:1899-1909. [PMID: 38409645 DOI: 10.1007/s11517-024-03045-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 02/03/2024] [Indexed: 02/28/2024]
Abstract
Early detection is critical for successfully diagnosing cancer, and timely analysis of diagnostic tests is increasingly important. In the context of neuroendocrine tumors, the Ki-67 proliferation index serves as a fundamental biomarker, aiding pathologists in grading and diagnosing these tumors based on histopathological images. The appropriate treatment plan for the patient is determined based on the tumor grade. An artificial intelligence-based method is proposed to aid pathologists in the automated calculation and grading of the Ki-67 proliferation index. The proposed system first performs preprocessing to enhance image quality. Then, segmentation process is performed using the U-Net architecture, which is a deep learning algorithm, to separate the nuclei from the background. The identified nuclei are then evaluated as Ki-67 positive or negative based on basic color space information and other features. The Ki-67 proliferation index is then calculated, and the neuroendocrine tumor is graded accordingly. The proposed system's performance was evaluated on a dataset obtained from the Department of Pathology at Meram Faculty of Medicine Hospital, Necmettin Erbakan University. The results of the pathologist and the proposed system were compared, and the proposed system was found to have an accuracy of 95% in tumor grading when compared to the pathologist's report.
Collapse
Affiliation(s)
- Zehra Yücel
- Necmettin Erbakan University, Department of Computer Technologies, Konya, Turkey.
- Hacettepe University, Graduate School of Science and Engineering, Ankara, Turkey.
| | - Fuat Akal
- Hacettepe University, Faculty of Engineering, Department of Computer Engineering, Ankara, Turkey
| | - Pembe Oltulu
- Necmettin Erbakan University, Faculty of Medicine, Department of Pathology, Konya, Turkey
| |
Collapse
|
|
16
|
Koziarski M, Cyganek B, Niedziela P, Olborski B, Antosz Z, Żydak M, Kwolek B, Wąsowicz P, Bukała A, Swadźba J, Sitkowski P. DiagSet: a dataset for prostate cancer histopathological image classification. Sci Rep 2024; 14:6780. [PMID: 38514661 PMCID: PMC10958036 DOI: 10.1038/s41598-024-52183-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 01/15/2024] [Indexed: 03/23/2024] Open
Abstract
Cancer diseases constitute one of the most significant societal challenges. In this paper, we introduce a novel histopathological dataset for prostate cancer detection. The proposed dataset, consisting of over 2.6 million tissue patches extracted from 430 fully annotated scans, 4675 scans with assigned binary diagnoses, and 46 scans with diagnoses independently provided by a group of histopathologists can be found at https://github.com/michalkoziarski/DiagSet . Furthermore, we propose a machine learning framework for detection of cancerous tissue regions and prediction of scan-level diagnosis, utilizing thresholding to abstain from the decision in uncertain cases. The proposed approach, composed of ensembles of deep neural networks operating on the histopathological scans at different scales, achieves 94.6% accuracy in patch-level recognition and is compared in a scan-level diagnosis with 9 human histopathologists showing high statistical agreement.
Collapse
Affiliation(s)
- Michał Koziarski
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland.
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland.
- Mila - Quebec AI Institute, 6666 Rue Saint-Urbain, Montréal, QC H2S 3H1, Canada.
| | - Bogusław Cyganek
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Przemysław Niedziela
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Bogusław Olborski
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Zbigniew Antosz
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Marcin Żydak
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Bogdan Kwolek
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Paweł Wąsowicz
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Andrzej Bukała
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Jakub Swadźba
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
- Andrzej Frycz Modrzewski Krakow University, Gustawa Herlinga-Grudzińskiego 1, 30-705, Kraków, Poland
| | - Piotr Sitkowski
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| |
Collapse
|
|
17
|
Wu Z, Park J, Steiner PR, Zhu B, Zhang JXJ. Generative Adversarial Network Model to Classify Human Induced Pluripotent Stem Cell-Cardiomyocytes based on Maturation Level. RESEARCH SQUARE 2024:rs.3.rs-4061531. [PMID: 38559233 PMCID: PMC10980104 DOI: 10.21203/rs.3.rs-4061531/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Objective Our study develops a generative adversarial network (GAN)-based method that generates faithful synthetic image data of human cardiomyocytes at varying stages in their maturation process, as a tool to significantly enhance the classification accuracy of cells and ultimately assist the throughput of computational analysis of cellular structure and functions. Methods Human induced pluripotent stem cell derived cardiomyocytes (hiPSC-CMs) were cultured on micropatterned collagen coated hydrogels of physiological stiffnesses to facilitate maturation and optical measurements were performed for their structural and functional analyses. Control groups were cultured on collagen coated glass well plates. These image recordings were used as the real data to train the GAN model. Results The results show the GAN approach is able to replicate true features from the real data, and inclusion of such synthetic data significantly improves the classification accuracy compared to usage of only real experimental data that is often limited in scale and diversity. Conclusion The proposed model outperformed four conventional machine learning algorithms with respect to improved data generalization ability and data classification accuracy by incorporating synthetic data. Significance This work demonstrates the importance of integrating synthetic data in situations where there are limited sample sizes and thus, effectively addresses the challenges imposed by data availability.
Collapse
Affiliation(s)
- Ziqian Wu
- Thayer School of Engineering, Dartmouth College, Hanover, NH USA
| | - Jiyoon Park
- Thayer School of Engineering, Dartmouth College, Hanover, NH USA
| | | | - Bo Zhu
- Department of Computer Science, Dartmouth College, Hanover, NH USA. He is now with the School of Interactive Computing, Georgia Institute of Technology, GA USA
| | - John X J Zhang
- Thayer School of Engineering, Dartmouth College, Hanover, NH USA
| |
Collapse
|
|
18
|
Ayana G, Lee E, Choe SW. Vision Transformers for Breast Cancer Human Epidermal Growth Factor Receptor 2 Expression Staging without Immunohistochemical Staining. THE AMERICAN JOURNAL OF PATHOLOGY 2024; 194:402-414. [PMID: 38096984 DOI: 10.1016/j.ajpath.2023.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 10/10/2023] [Accepted: 11/20/2023] [Indexed: 12/31/2023]
Abstract
Accurate staging of human epidermal growth factor receptor 2 (HER2) expression is vital for evaluating breast cancer treatment efficacy. However, it typically involves costly and complex immunohistochemical staining, along with hematoxylin and eosin staining. This work presents customized vision transformers for staging HER2 expression in breast cancer using only hematoxylin and eosin-stained images. The proposed algorithm comprised three modules: a localization module for weakly localizing critical image features using spatial transformers, an attention module for global learning via vision transformers, and a loss module to determine proximity to a HER2 expression level based on input images by calculating ordinal loss. Results, reported with 95% CIs, reveal the proposed approach's success in HER2 expression staging: area under the receiver operating characteristic curve, 0.9202 ± 0.01; precision, 0.922 ± 0.01; sensitivity, 0.876 ± 0.01; and specificity, 0.959 ± 0.02 over fivefold cross-validation. Comparatively, this approach significantly outperformed conventional vision transformer models and state-of-the-art convolutional neural network models (P < 0.001). Furthermore, it surpassed existing methods when evaluated on an independent test data set. This work holds great importance, aiding HER2 expression staging in breast cancer treatment while circumventing the costly and time-consuming immunohistochemical staining procedure, thereby addressing diagnostic disparities in low-resource settings and low-income countries.
Collapse
Affiliation(s)
- Gelan Ayana
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea; School of Biomedical Engineering, Jimma University, Jimma, Ethiopia
| | - Eonjin Lee
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea
| | - Se-Woon Choe
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea; Department of IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea.
| |
Collapse
|
|
19
|
Valenza C, Guidi L, Battaiotto E, Trapani D, Sartore Bianchi A, Siena S, Curigliano G. Targeting HER2 heterogeneity in breast and gastrointestinal cancers. Trends Cancer 2024; 10:113-123. [PMID: 38008666 DOI: 10.1016/j.trecan.2023.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 10/29/2023] [Accepted: 11/01/2023] [Indexed: 11/28/2023]
Abstract
About 20% of breast and gastric cancers and 3% of colorectal carcinomas overexpress the human epidermal growth factor receptor 2 (HER2) and are sensitive to HER2-directed agents. The expression of HER2 may differ within the same tumoral lesion (spatial intralesional heterogeneity), from different tumor locations (spatial interlesional heterogeneity), and throughout treatments (temporal heterogeneity). Spatial and temporal heterogeneity may impact on response and resistance to HER2-targeting agents and its prevalence and predictive role changes across HER2-overexpressing solid tumors. Therefore, the definition and the characterization of HER2 heterogeneity pose many challenges and its implementation as a reproducible predictive biomarker would help in guiding treatment modulation.
Collapse
Affiliation(s)
- Carmine Valenza
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Lorenzo Guidi
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Elena Battaiotto
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Dario Trapani
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Andrea Sartore Bianchi
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy; Niguarda Cancer Center, Grande Ospedale Metropolitano Niguarda, Milan, Italy
| | - Salvatore Siena
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy; Niguarda Cancer Center, Grande Ospedale Metropolitano Niguarda, Milan, Italy
| | - Giuseppe Curigliano
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.
| |
Collapse
|
|
20
|
Kumarasinghe MP, Houghton D, Allanson BM, Price TJ. What Therapeutic Biomarkers in Gastro-Esophageal Junction and Gastric Cancer Should a Pathologist Know About? Surg Pathol Clin 2023; 16:659-672. [PMID: 37863558 DOI: 10.1016/j.path.2023.05.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2023]
Abstract
Malignancies of upper gastrointestinal tract are aggressive, and most locally advanced unresectable and metastatic cancers are managed by a combination of surgery and neoadjuvant/adjuvant chemotherapy and radiotherapy. Current therapeutic recommendations include targeted therapies based on biomarker expression of an individual tumor. All G/gastro-esophageal junction (GEJ) cancers should be tested for HER2 status as a reflex test at the time of diagnosis. Currently, testing for PDL 1 and mismatch repair protein status is optional. HER2 testing is restricted to adenocarcinomas only and endoscopic biopsies, resections, or cellblocks. Facilities should be available for performing validated immunohistochemical stains and in-situ hybridization techniques, and importantly, pathologists should be experienced with preanalytical and analytical issues and scoring criteria. Genomic profiling via next-generation sequencing (NGS) is another strategy that assess numerous mutations and other molecular events simultaneously, including HER2 amplification, MSS status, tumor mutation burden, and neurotrophic tropomyosin-receptor kinases gene fusions.
Collapse
Affiliation(s)
- Marian Priyanthi Kumarasinghe
- Anatomical Pathology, PathWest, QEII Medical Centre, School of Pathology and Laboratory Medicine, UWA and Curtin Medical School, J Block, Hospital Avenue, Nedlands, Western Australia 6009, Australia.
| | - Daniel Houghton
- Department of Anatomical Pathology, PathWest, QEII Medical Centre, J Block, Hospital Avenue, Nedlands, Western Australia 6009, Australia
| | - Benjamin Michael Allanson
- Department of Anatomical Pathology, PathWest, QEII Medical Centre, J Block, Hospital Avenue, Nedlands, Western Australia 6009, Australia
| | - Timothy J Price
- Department Medical Oncology, The Queen Elizabeth Hospital and University of Adelaide, Adelaide, South Australia, Australia
| |
Collapse
|
|
21
|
Gupta P, Khare V, Srivastava A, Agarwal J, Mittal V, Sonkar V, Saxena S, Agarwal A, Jain A. A prospective observational multicentric clinical trial to evaluate microscopic examination of acid-fast bacilli in sputum by artificial intelligence-based microscopy system. J Investig Med 2023; 71:716-721. [PMID: 37158073 DOI: 10.1177/10815589231171402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Microscopy-based tuberculosis (TB) diagnosis i.e., Ziehl-Neelsen (ZN) stained smear screening still remains the primary diagnostic method in resource poor and high TB burden countries, however itrequires considerable experience and is bound to human errors. In remote areas, wherever expert microscopist is not available, timely diagnosis at initial level is not possible. Artificial intelligence (AI)-based microscopy may be a solution to this problem. A prospective observational multi-centric clinical trial to evaluate microscopic examination of acid-fast bacilli (AFB) in sputum by the AI based system was done in three hospitals in Northern India. Sputum samples from 400 clinically suspected cases of pulmonary tuberculosis were collected from three centres. Ziehl-Neelsen staining of smears was done. All the smears were observed by 3 microscopist and the AI based microscopy system. AI based microscopy was found to have a sensitivity, specificity, positive predictive value, negative predictive value and diagnostic accuracy of 89.25%, 92.15%, 75.45%, 96.94%, 91.53% respectively. AI based sputum microscopy has an acceptable degree of accuracy, PPV, NPV, specificity and sensitivity and thus may be used as a screening tool for the diagnosis of pulmonary tuberculosis.
Collapse
Affiliation(s)
- Prashant Gupta
- Department of Microbiology, King George's Medical University, Lucknow, Uttar Pradesh, India
| | - Vineeta Khare
- Department of Microbiology, Era's Lucknow Medical College & hospitals, Era University, Lucknow, Uttar Pradesh, India
| | - Anand Srivastava
- Department of Respiratory Medicine, King George's Medical University, Lucknow, Uttar Pradesh, India
| | - Jyotsna Agarwal
- Department of Microbiology, Dr. Ram Manohar Lohia Institute of Medical Sciences, Lucknow, Uttar Pradesh, India
| | - Vineeta Mittal
- Department of Microbiology, Dr. Ram Manohar Lohia Institute of Medical Sciences, Lucknow, Uttar Pradesh, India
| | - Vijay Sonkar
- Department of Microbiology, King George's Medical University, Lucknow, Uttar Pradesh, India
| | - Shelly Saxena
- Sevamob Ventures Limited, Lucknow, Uttar Pradesh, India
| | - Ankit Agarwal
- Sevamob Ventures Limited, Lucknow, Uttar Pradesh, India
| | - Amita Jain
- Department of Microbiology, King George's Medical University, Lucknow, Uttar Pradesh, India
| |
Collapse
|
|
22
|
Liu Y, Han D, Parwani AV, Li Z. Applications of Artificial Intelligence in Breast Pathology. Arch Pathol Lab Med 2023; 147:1003-1013. [PMID: 36800539 DOI: 10.5858/arpa.2022-0457-ra] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/15/2022] [Indexed: 02/19/2023]
Abstract
CONTEXT.— Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology. OBJECTIVE.— To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes. DATA SOURCES.— We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience. CONCLUSIONS.— With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.
Collapse
Affiliation(s)
- Yueping Liu
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| | - Dandan Han
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| | - Anil V Parwani
- The Department of Pathology, The Ohio State University, Columbus (Parwani, Li)
| | - Zaibo Li
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| |
Collapse
|
|
23
|
Pham MD, Balezo G, Tilmant C, Petit S, Salmon I, Hadj SB, Fick RHJ. Interpretable HER2 scoring by evaluating clinical guidelines through a weakly supervised, constrained deep learning approach. Comput Med Imaging Graph 2023; 108:102261. [PMID: 37356357 DOI: 10.1016/j.compmedimag.2023.102261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Revised: 04/30/2023] [Accepted: 06/11/2023] [Indexed: 06/27/2023]
Abstract
The evaluation of the Human Epidermal growth factor Receptor-2 (HER2) expression is an important prognostic biomarker for breast cancer treatment selection. However, HER2 scoring has notoriously high interobserver variability due to stain variations between centers and the need to estimate visually the staining intensity in specific percentages of tumor area. In this paper, focusing on the interpretability of HER2 scoring by a pathologist, we propose a semi-automatic, two-stage deep learning approach that directly evaluates the clinical HER2 guidelines defined by the American Society of Clinical Oncology/ College of American Pathologists (ASCO/CAP). In the first stage, we segment the invasive tumor over the user-indicated Region of Interest (ROI). Then, in the second stage, we classify the tumor tissue into four HER2 classes. For the classification stage, we use weakly supervised, constrained optimization to find a model that classifies cancerous patches such that the tumor surface percentage meets the guidelines specification of each HER2 class. We end the second stage by freezing the model and refining its output logits in a supervised way to all slide labels in the training set. To ensure the quality of our dataset's labels, we conducted a multi-pathologist HER2 scoring consensus. For the assessment of doubtful cases where no consensus was found, our model can help by interpreting its HER2 class percentages output. We achieve a performance of 0.78 in F1-score on the test set while keeping our model interpretable for the pathologist, hopefully contributing to interpretable AI models in digital pathology.
Collapse
Affiliation(s)
- Manh-Dan Pham
- Tribun Health, 2 Rue du Capitaine Scott, 75015 Paris, France
| | | | | | | | | | - Saïma Ben Hadj
- Tribun Health, 2 Rue du Capitaine Scott, 75015 Paris, France
| | - Rutger H J Fick
- Tribun Health, 2 Rue du Capitaine Scott, 75015 Paris, France.
| |
Collapse
|
|
24
|
Li Y, Liu S. Adversarial Attack and Defense in Breast Cancer Deep Learning Systems. Bioengineering (Basel) 2023; 10:973. [PMID: 37627858 PMCID: PMC10451783 DOI: 10.3390/bioengineering10080973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 08/14/2023] [Indexed: 08/27/2023] Open
Abstract
Deep-learning-assisted medical diagnosis has brought revolutionary innovations to medicine. Breast cancer is a great threat to women's health, and deep-learning-assisted diagnosis of breast cancer pathology images can save manpower and improve diagnostic accuracy. However, researchers have found that deep learning systems based on natural images are vulnerable to attacks that can lead to errors in recognition and classification, raising security concerns about deep systems based on medical images. We used the adversarial attack algorithm FGSM to reveal that breast cancer deep learning systems are vulnerable to attacks and thus misclassify breast cancer pathology images. To address this problem, we built a deep learning system for breast cancer pathology image recognition with better defense performance. Accurate diagnosis of medical images is related to the health status of patients. Therefore, it is very important and meaningful to improve the security and reliability of medical deep learning systems before they are actually deployed.
Collapse
Affiliation(s)
- Yang Li
- Graduate School of Advanced Science and Engineering, Hiroshima University, Higashihiroshima 739-8511, Japan
| | | |
Collapse
|
|
25
|
Bardia A, Viale G. HER2-Low Breast Cancer-Diagnostic Challenges and Opportunities for Insights from Ongoing Studies: A Podcast. Target Oncol 2023; 18:313-319. [PMID: 37133651 DOI: 10.1007/s11523-023-00964-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/27/2023] [Indexed: 05/04/2023]
Abstract
Breast cancer has been traditionally classified as either human epidermal growth factor receptor 2 (HER2)-positive or HER2-negative based on immunohistochemistry (IHC) scoring and/or gene amplification. HER2-positive breast cancer (defined as IHC 3+ or IHC 2+ and in situ hybridization [ISH]+) is routinely treated with HER2-targeted therapies, while HER2-negative breast cancer (defined as IHC 0, IHC 1+, or IHC 2+/ISH-) was not previously eligible for HER2-targeted therapy. Some tumors traditionally defined as HER2-negative express low levels of HER2 (i.e., HER2-low breast cancer, defined as IHC 1+ or IHC 2+/ISH-). Recently reported results from the DESTINY-Breast04 trial demonstrated that the HER2-targeted antibody-drug conjugate trastuzumab deruxtecan (T-DXd) improved survival in patients with previously treated advanced or metastatic HER2-low breast cancer, leading to the approval of T-DXd in the US and EU for patients with unresectable or metastatic HER2-low breast cancer after prior chemotherapy in the metastatic setting or disease recurrence within 6 months of adjuvant chemotherapy. As the first HER2-targeted therapy approved for the treatment of HER2-low breast cancer, this represents a change in the clinical landscape and presents new challenges, including identifying patients with HER2-low breast cancer. In this podcast, we discuss the strengths and limitations of current methodologies for classifying HER2 expression and future research that will help refine the identification of patients expected to benefit from HER2-targeted therapy, such as T‑DXd or other antibody-drug conjugates. Although current methodologies are not optimized to identify all patients with HER2-low breast cancer who may potentially benefit from HER2-targeted antibody-drug conjugates, they are likely to identify many. Ongoing studies-including the DESTINY-Breast06 trial evaluating T-DXd in patients with HER2-low breast cancer and those with tumors expressing very low levels of HER2 (IHC > 0 to < 1+)-will provide insights that may improve the identification of patient populations expected to benefit from HER2-targeted antibody-drug conjugates. Supplementary file1 (MP4 123466 KB).
Collapse
Affiliation(s)
| | - Giuseppe Viale
- University of Milan, Milan, Italy
- European Institute of Oncology, IRCCS, Milan, Italy
| |
Collapse
|
|
26
|
Che Y, Ren F, Zhang X, Cui L, Wu H, Zhao Z. Immunohistochemical HER2 Recognition and Analysis of Breast Cancer Based on Deep Learning. Diagnostics (Basel) 2023; 13:263. [PMID: 36673073 PMCID: PMC9858188 DOI: 10.3390/diagnostics13020263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 01/05/2023] [Accepted: 01/06/2023] [Indexed: 01/13/2023] Open
Abstract
Breast cancer is one of the common malignant tumors in women. It seriously endangers women's life and health. The human epidermal growth factor receptor 2 (HER2) protein is responsible for the division and growth of healthy breast cells. The overexpression of the HER2 protein is generally evaluated by immunohistochemistry (IHC). The IHC evaluation criteria mainly includes three indexes: staining intensity, circumferential membrane staining pattern, and proportion of positive cells. Manually scoring HER2 IHC images is an error-prone, variable, and time-consuming work. To solve these problems, this study proposes an automated predictive method for scoring whole-slide images (WSI) of HER2 slides based on a deep learning network. A total of 95 HER2 pathological slides from September 2021 to December 2021 were included. The average patch level precision and f1 score were 95.77% and 83.09%, respectively. The overall accuracy of automated scoring for slide-level classification was 97.9%. The proposed method showed excellent specificity for all IHC 0 and 3+ slides and most 1+ and 2+ slides. The evaluation effect of the integrated method is better than the effect of using the staining result only.
Collapse
Affiliation(s)
- Yuxuan Che
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
- School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 101408, China
- Jinfeng Laboratory, Chongqing 401329, China
| | - Fei Ren
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
| | - Xueyuan Zhang
- Beijing Zhijian Life Technology Co., Ltd., Beijing 100036, China
| | - Li Cui
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
| | - Huanwen Wu
- Department of Pathology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China
| | - Ze Zhao
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
| |
Collapse
|
|
27
|
Using Whole Slide Gray Value Map to Predict HER2 Expression and FISH Status in Breast Cancer. Cancers (Basel) 2022; 14:cancers14246233. [PMID: 36551720 PMCID: PMC9777488 DOI: 10.3390/cancers14246233] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 12/12/2022] [Accepted: 12/14/2022] [Indexed: 12/24/2022] Open
Abstract
Accurate detection of HER2 expression through immunohistochemistry (IHC) is of great clinical significance in the treatment of breast cancer. However, manual interpretation of HER2 is challenging, due to the interobserver variability among pathologists. We sought to explore a deep learning method to predict HER2 expression level and gene status based on a Whole Slide Image (WSI) of the HER2 IHC section. When applied to 228 invasive breast carcinoma of no special type (IBC-NST) DAB-stained slides, our GrayMap+ convolutional neural network (CNN) model accurately classified HER2 IHC level with mean accuracy 0.952 ± 0.029 and predicted HER2 FISH status with mean accuracy 0.921 ± 0.029. Our result also demonstrated strong consistency in HER2 expression score between our system and experienced pathologists (intraclass correlation coefficient (ICC) = 0.903, Cohen's κ = 0.875). The discordant cases were found to be largely caused by high intra-tumor staining heterogeneity in the HER2 IHC group and low copy number in the HER2 FISH group.
Collapse
|
|
28
|
Artificial intelligence for prediction of response to cancer immunotherapy. Semin Cancer Biol 2022; 87:137-147. [PMID: 36372326 DOI: 10.1016/j.semcancer.2022.11.008] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 11/02/2022] [Accepted: 11/08/2022] [Indexed: 11/13/2022]
Abstract
Artificial intelligence (AI) indicates the application of machines to imitate intelligent behaviors for solving complex tasks with minimal human intervention, including machine learning and deep learning. The use of AI in medicine improves health-care systems in multiple areas such as diagnostic confirmation, risk stratification, analysis, prognosis prediction, treatment surveillance, and virtual health support, which has considerable potential to revolutionize and reshape medicine. In terms of immunotherapy, AI has been applied to unlock underlying immune signatures to associate with responses to immunotherapy indirectly as well as predict responses to immunotherapy responses directly. The AI-based analysis of high-throughput sequences and medical images can provide useful information for management of cancer immunotherapy considering the excellent abilities in selecting appropriate subjects, improving therapeutic regimens, and predicting individualized prognosis. In present review, we aim to evaluate a broad framework about AI-based computational approaches for prediction of response to cancer immunotherapy on both indirect and direct manners. Furthermore, we summarize our perspectives about challenges and opportunities of further AI applications on cancer immunotherapy relating to clinical practicability.
Collapse
|
|
29
|
Ahmed AA, Abouzid M, Kaczmarek E. Deep Learning Approaches in Histopathology. Cancers (Basel) 2022; 14:5264. [PMID: 36358683 PMCID: PMC9654172 DOI: 10.3390/cancers14215264] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 10/10/2022] [Accepted: 10/24/2022] [Indexed: 10/06/2023] Open
Abstract
The revolution of artificial intelligence and its impacts on our daily life has led to tremendous interest in the field and its related subtypes: machine learning and deep learning. Scientists and developers have designed machine learning- and deep learning-based algorithms to perform various tasks related to tumor pathologies, such as tumor detection, classification, grading with variant stages, diagnostic forecasting, recognition of pathological attributes, pathogenesis, and genomic mutations. Pathologists are interested in artificial intelligence to improve the diagnosis precision impartiality and to minimize the workload combined with the time consumed, which affects the accuracy of the decision taken. Regrettably, there are already certain obstacles to overcome connected to artificial intelligence deployments, such as the applicability and validation of algorithms and computational technologies, in addition to the ability to train pathologists and doctors to use these machines and their willingness to accept the results. This review paper provides a survey of how machine learning and deep learning methods could be implemented into health care providers' routine tasks and the obstacles and opportunities for artificial intelligence application in tumor morphology.
Collapse
Affiliation(s)
- Alhassan Ali Ahmed
- Department of Bioinformatics and Computational Biology, Poznan University of Medical Sciences, 60-812 Poznan, Poland
- Doctoral School, Poznan University of Medical Sciences, 60-812 Poznan, Poland
| | - Mohamed Abouzid
- Doctoral School, Poznan University of Medical Sciences, 60-812 Poznan, Poland
- Department of Physical Pharmacy and Pharmacokinetics, Faculty of Pharmacy, Poznan University of Medical Sciences, Rokietnicka 3 St., 60-806 Poznan, Poland
| | - Elżbieta Kaczmarek
- Department of Bioinformatics and Computational Biology, Poznan University of Medical Sciences, 60-812 Poznan, Poland
| |
Collapse
|
|
30
|
Using Deep Learning to Predict Final HER2 Status in Invasive Breast Cancers That are Equivocal (2+) by Immunohistochemistry. Appl Immunohistochem Mol Morphol 2022; 30:668-673. [PMID: 36251973 DOI: 10.1097/pai.0000000000001079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 09/17/2022] [Indexed: 11/25/2022]
Abstract
Invasive breast carcinomas are routinely tested for HER2 using immunohistochemistry (IHC), with reflex in situ hybridization (ISH) for those scored as equivocal (2+). ISH testing is expensive, time-consuming, and not universally available. In this study, we trained a deep learning algorithm to directly predict HER2 gene amplification status from HER2 2+ IHC slides. Data included 115 consecutive cases of invasive breast carcinoma scored as 2+ by IHC that had follow-up HER2 ISH testing. An external validation data set was created from 36 HER2 IHC slides prepared at an outside institution. All internal IHC slides were digitized and divided into training (80%), and test (20%) sets with 5-fold cross-validation. Small patches (256×256 pixels) were randomly extracted and used to train convolutional neural networks with EfficientNet B0 architecture using a transfer learning approach. Predictions for slides in the test set were made on individual patches, and these predictions were aggregated to generate an overall prediction for each slide. This resulted in a receiver operating characteristic area under the curve of 0.83 with an overall accuracy of 79% (sensitivity=0.70, specificity=0.82). Analysis of external validation slides resulted in a receiver operating characteristic area under the curve of 0.79 with an overall accuracy of 81% (sensitivity=0.50, specificity=0.82). Although the sensitivity and specificity are not high enough to negate the need for reflexive ISH testing entirely, this approach may be useful for triaging cases more likely to be HER2 positive and initiating treatment planning in centers where HER2 ISH testing is not readily available.
Collapse
|
|
31
|
Vuong TTL, Song B, Kwak JT, Kim K. Prediction of Epstein-Barr Virus Status in Gastric Cancer Biopsy Specimens Using a Deep Learning Algorithm. JAMA Netw Open 2022; 5:e2236408. [PMID: 36205993 PMCID: PMC9547324 DOI: 10.1001/jamanetworkopen.2022.36408] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
IMPORTANCE Epstein-Barr virus (EBV)-associated gastric cancer (EBV-GC) is 1 of 4 molecular subtypes of GC and is confirmed by an expensive molecular test, EBV-encoded small RNA in situ hybridization. EBV-GC has 2 histologic characteristics, lymphoid stroma and lace-like tumor pattern, but projecting EBV-GC at biopsy is difficult even for experienced pathologists. OBJECTIVE To develop and validate a deep learning algorithm to predict EBV status from pathology images of GC biopsy. DESIGN, SETTING, AND PARTICIPANTS This diagnostic study developed a deep learning classifier to predict EBV-GC using image patches of tissue microarray (TMA) and whole slide images (WSIs) of GC and applied it to GC biopsy specimens from GCs diagnosed at Kangbuk Samsung Hospital between 2011 and 2020. For a quantitative evaluation and EBV-GC prediction on biopsy specimens, the area of each class and the fraction in total tissue or tumor area were calculated. Data were analyzed from March 5, 2021, to February 10, 2022. MAIN OUTCOMES AND MEASURES Evaluation metrics of predictive model performance were assessed on accuracy, recall, precision, F1 score, area under the receiver operating characteristic curve (AUC), and κ coefficient. RESULTS This study included 137 184 image patches from 16 TMAs (708 tissue cores), 24 WSIs, and 286 biopsy images of GC. The classifier was able to classify EBV-GC image patches from TMAs and WSIs with 94.70% accuracy, 0.936 recall, 0.938 precision, 0.937 F1 score, and 0.909 κ coefficient. The classifier was used for predicting and measuring the area and fraction of EBV-GC on biopsy tissue specimens. A 10% cutoff value for the predicted fraction of EBV-GC to tissue (EBV-GC/tissue area) produced the best prediction results in EBV-GC biopsy specimens and showed the highest AUC value (0.8723; 95% CI, 0.7560-0.9501). That cutoff also obtained high sensitivity (0.895) and moderate specificity (0.745) compared with experienced pathologist sensitivity (0.842) and specificity (0.854) when using the presence of lymphoid stroma and a lace-like pattern as diagnostic criteria. On prediction maps, EBV-GCs with lace-like pattern and lymphoid stroma showed the same prediction results as EBV-GC, but cases lacking these histologic features revealed heterogeneous prediction results of EBV-GC and non-EBV-GC areas. CONCLUSIONS AND RELEVANCE This study showed the feasibility of EBV-GC prediction using a deep learning algorithm, even in biopsy samples. Use of such an image-based classifier before a confirmatory molecular test will reduce costs and tissue waste.
Collapse
Affiliation(s)
- Trinh Thi Le Vuong
- School of Electrical Engineering, Korea University, Seoul, Republic of Korea
| | - Boram Song
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Jin T. Kwak
- School of Electrical Engineering, Korea University, Seoul, Republic of Korea
| | - Kyungeun Kim
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| |
Collapse
|
|
32
|
Prat A, Bardia A, Curigliano G, Hammond MEH, Loibl S, Tolaney SM, Viale G. An Overview of Clinical Development of Agents for Metastatic or Advanced Breast Cancer Without ERBB2 Amplification (HER2-Low). JAMA Oncol 2022; 8:2796438. [PMID: 36107417 DOI: 10.1001/jamaoncol.2022.4175] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/18/2024]
Abstract
Importance Erb-b2 receptor tyrosine kinase 2 (ERBB2; formerly HER2 [human epidermal growth factor receptor 2]) is an important prognostic and predictive factor in breast cancer. Anti-ERBB2 therapies have improved outcomes in ERBB2-positive breast cancer. However, based on current definitions, tumors with low ERBB2 expression are included in the ERBB2-negative subtype, and therefore, are ineligible for anti-ERBB2 therapies; patients with ERBB2-low (immunohistochemistry [IHC] 1 positive [+] or IHC 2+/in situ hybridization [ISH] negative [-]) tumors account for up to approximately 50% of breast cancer cases. Although the prognostic role of ERBB2-low needs to be defined, ERBB2 offers a potential therapeutic target in these patients. Observations Most breast cancer tumors have some ERBB2 expression, with ERBB2-low being more common in hormone receptor-positive than in hormone receptor-negative breast cancer. Although an early clinical study failed to demonstrate benefit of adjuvant trastuzumab for ERBB2-low disease, several novel anti-ERBB2 therapies have shown efficacy in ERBB2-low breast cancer, including the antibody-drug conjugate trastuzumab deruxtecan in a phase 3 trial, and trastuzumab duocarmazine and the bispecific antibody zenocutuzumab in early-phase studies. Although reports are conflicting, some differences in biology and patient outcomes have been found between ERBB2-low and ERBB2 IHC-0 breast cancer. Currently, no established guidelines exist for scoring ERBB2-low expression in breast cancer because the focus has been on binary classification as ERBB2-positive or ERBB2-negative. Additional interpretive cutoffs may be needed to select patients for treatment with effective agents in ERBB2-low breast cancer, along with standardized laboratory quality assurance programs to ensure consistent patient identification for eligibility for ERBB2-low targeting agents. Conclusions and Relevance This review suggests that ERBB2-low may be a distinct, clinically relevant breast cancer entity warranting reassessment of traditional diagnostic and therapeutic paradigms. Ongoing clinical trials and further investigations may provide optimized strategies for diagnosing and treating ERBB2-low breast cancer, including reproducible, consistent definitions to identify patients in this diagnostic category and demonstration of benefits of emerging therapies.
Collapse
Affiliation(s)
- Aleix Prat
- Department of Medical Oncology, Hospital Clinic of Barcelona, Barcelona, Spain
- Translational Genomics and Targeted Therapies in Solid Tumors, August Pi i Sunyer Biomedical Research Institute, Barcelona, Spain
- Department of Medicine, University of Barcelona, Barcelona, Spain
| | - Aditya Bardia
- Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts
| | - Giuseppe Curigliano
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
- European Institute of Oncology, Istituto di Ricovero e Cura a Carattere Scientifico, Milan, Italy
| | - M Elizabeth H Hammond
- Intermountain Healthcare and University of Utah School of Medicine, Salt Lake City, Utah
| | - Sibylle Loibl
- German Breast Group, Neu-Isenburg, Germany
- Center for Hematology and Oncology Bethanien, Frankfurt, Germany
| | - Sara M Tolaney
- Division of Breast Oncology, Dana-Farber Cancer Institute, Boston, Massachusetts
| | - Giuseppe Viale
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
- European Institute of Oncology, Istituto di Ricovero e Cura a Carattere Scientifico, Milan, Italy
| |
Collapse
|
|
33
|
HEROHE Challenge: Predicting HER2 Status in Breast Cancer from Hematoxylin–Eosin Whole-Slide Imaging. J Imaging 2022; 8:jimaging8080213. [PMID: 36005456 PMCID: PMC9410129 DOI: 10.3390/jimaging8080213] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 07/26/2022] [Accepted: 07/27/2022] [Indexed: 02/06/2023] Open
Abstract
Breast cancer is the most common malignancy in women worldwide, and is responsible for more than half a million deaths each year. The appropriate therapy depends on the evaluation of the expression of various biomarkers, such as the human epidermal growth factor receptor 2 (HER2) transmembrane protein, through specialized techniques, such as immunohistochemistry or in situ hybridization. In this work, we present the HER2 on hematoxylin and eosin (HEROHE) challenge, a parallel event of the 16th European Congress on Digital Pathology, which aimed to predict the HER2 status in breast cancer based only on hematoxylin–eosin-stained tissue samples, thus avoiding specialized techniques. The challenge consisted of a large, annotated, whole-slide images dataset (509), specifically collected for the challenge. Models for predicting HER2 status were presented by 21 teams worldwide. The best-performing models are presented by detailing the network architectures and key parameters. Methods are compared and approaches, core methodologies, and software choices contrasted. Different evaluation metrics are discussed, as well as the performance of the presented models for each of these metrics. Potential differences in ranking that would result from different choices of evaluation metrics highlight the need for careful consideration at the time of their selection, as the results show that some metrics may misrepresent the true potential of a model to solve the problem for which it was developed. The HEROHE dataset remains publicly available to promote advances in the field of computational pathology.
Collapse
|
|
34
|
Mathew T, Niyas S, Johnpaul C, Kini JR, Rajan J. A novel deep classifier framework for automated molecular subtyping of breast carcinoma using immunohistochemistry image analysis. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103657] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
|
35
|
van der Kamp A, Waterlander TJ, de Bel T, van der Laak J, van den Heuvel-Eibrink MM, Mavinkurve-Groothuis AMC, de Krijger RR. Artificial Intelligence in Pediatric Pathology: The Extinction of a Medical Profession or the Key to a Bright Future? Pediatr Dev Pathol 2022; 25:380-387. [PMID: 35238696 DOI: 10.1177/10935266211059809] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Artificial Intelligence (AI) has become of increasing interest over the past decade. While digital image analysis (DIA) is already being used in radiology, it is still in its infancy in pathology. One of the reasons is that large-scale digitization of glass slides has only recently become available. With the advent of digital slide scanners, that digitize glass slides into whole slide images, many labs are now in a transition phase towards digital pathology. However, only few departments worldwide are currently fully digital. Digital pathology provides the ability to annotate large datasets and train computers to develop and validate robust algorithms, similar to radiology. In this opinionated overview, we will give a brief introduction into AI in pathology, discuss the potential positive and negative implications and speculate about the future role of AI in the field of pediatric pathology.
Collapse
Affiliation(s)
- Ananda van der Kamp
- 541199Princess Máxima Center for Pediatric Oncology, Utrecht, the Netherlands
| | - Tomas J Waterlander
- 541199Princess Máxima Center for Pediatric Oncology, Utrecht, the Netherlands
| | - Thomas de Bel
- Department of Pathology, 234134Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jeroen van der Laak
- Department of Pathology, 234134Radboud University Medical Center, Nijmegen, the Netherlands.,Center for Medical Image Science and Visualization, 4566Linköping University, Linköping, Sweden
| | | | | | - Ronald R de Krijger
- 541199Princess Máxima Center for Pediatric Oncology, Utrecht, the Netherlands.,Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| |
Collapse
|
|
36
|
Kouzu K, Nearchou IP, Kajiwara Y, Tsujimoto H, Lillard K, Kishi Y, Ueno H. Deep-learning-based classification of desmoplastic reaction on H&E predicts poor prognosis in oesophageal squamous cell carcinoma. Histopathology 2022; 81:255-263. [PMID: 35758184 DOI: 10.1111/his.14708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 05/16/2022] [Accepted: 05/31/2022] [Indexed: 12/24/2022]
Abstract
AIMS Desmoplastic reaction (DR) categorisation has been shown to be a promising prognostic factor in oesophageal squamous cell carcinoma (ESCC). The usual DR evaluation is performed using semiquantitative scores, which can be subjective. This study aimed to investigate whether a deep-learning classifier could be used for DR classification. We further assessed the prognostic significance of the deep-learning classifier and compared it to that of manual DR reporting and other pathological factors currently used in the clinic. METHODS AND RESULTS From 222 surgically resected ESCC cases, 31 randomly selected haematoxylin-eosin-digitised whole slides of patients with immature DR were used to train and develop a deep-learning classifier. The classifier was trained for 89 370 iterations. The accuracy of the deep-learning classifier was assessed to 30 unseen cases, and the results revealed a Dice coefficient score of 0.81. For survival analysis, the classifier was then applied to the entire cohort of patients, which was split into a training (n = 156) and a test (n = 66) cohort. The automated DR classification had a higher prognostic significance for disease-specific survival than the manually classified DR in both the training and test cohorts. In addition, the automated DR classification outperformed the prognostic accuracy of the gold-standard factors of tumour depth and lymph node metastasis. CONCLUSIONS This study demonstrated that DR can be objectively and quantitatively assessed in ESCC using a deep-learning classifier and that automatically classed DR has a higher prognostic significance than manual DR and other features currently used in the clinic.
Collapse
Affiliation(s)
- Keita Kouzu
- Department of Surgery, National Defense Medical College, Saitama, Japan
| | - Ines P Nearchou
- Department of Surgery, National Defense Medical College, Saitama, Japan
| | - Yoshiki Kajiwara
- Department of Surgery, National Defense Medical College, Saitama, Japan
| | | | | | - Yoji Kishi
- Department of Surgery, National Defense Medical College, Saitama, Japan
| | - Hideki Ueno
- Department of Surgery, National Defense Medical College, Saitama, Japan
| |
Collapse
|
|
37
|
Han Z, Lan J, Wang T, Hu Z, Huang Y, Deng Y, Zhang H, Wang J, Chen M, Jiang H, Lee RG, Gao Q, Du M, Tong T, Chen G. A Deep Learning Quantification Algorithm for HER2 Scoring of Gastric Cancer. Front Neurosci 2022; 16:877229. [PMID: 35706692 PMCID: PMC9190202 DOI: 10.3389/fnins.2022.877229] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Accepted: 04/19/2022] [Indexed: 11/13/2022] Open
Abstract
Gastric cancer is the third most common cause of cancer-related death in the world. Human epidermal growth factor receptor 2 (HER2) positive is an important subtype of gastric cancer, which can provide significant diagnostic information for gastric cancer pathologists. However, pathologists usually use a semi-quantitative assessment method to assign HER2 scores for gastric cancer by repeatedly comparing hematoxylin and eosin (H&E) whole slide images (WSIs) with their HER2 immunohistochemical WSIs one by one under the microscope. It is a repetitive, tedious, and highly subjective process. Additionally, WSIs have billions of pixels in an image, which poses computational challenges to Computer-Aided Diagnosis (CAD) systems. This study proposed a deep learning algorithm for HER2 quantification evaluation of gastric cancer. Different from other studies that use convolutional neural networks for extracting feature maps or pre-processing on WSIs, we proposed a novel automatic HER2 scoring framework in this study. In order to accelerate the computational process, we proposed to use the re-parameterization scheme to separate the training model from the deployment model, which significantly speedup the inference process. To the best of our knowledge, this is the first study to provide a deep learning quantification algorithm for HER2 scoring of gastric cancer to assist the pathologist's diagnosis. Experiment results have demonstrated the effectiveness of our proposed method with an accuracy of 0.94 for the HER2 scoring prediction.
Collapse
Affiliation(s)
- Zixin Han
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Junlin Lan
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Tao Wang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Ziwei Hu
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Yuxiu Huang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Yanglin Deng
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Hejun Zhang
- Department of Pathology, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou, China
| | - Jianchao Wang
- Department of Pathology, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou, China
| | - Musheng Chen
- Department of Pathology, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou, China
| | - Haiyan Jiang
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou, China
| | - Ren-Guey Lee
- Department of Electronic Engineering, National Taipei University of Technology, Taipei, Taiwan
| | - Qinquan Gao
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
- Imperial Vision Technology, Fuzhou, China
| | - Ming Du
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
| | - Tong Tong
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
- Imperial Vision Technology, Fuzhou, China
| | - Gang Chen
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou, China
- Fujian Provincial Key Laboratory of Translational Cancer Medicin, Fuzhou, China
| |
Collapse
|
|
38
|
Rani P, Dutta K, Kumar V. Artificial intelligence techniques for prediction of drug synergy in malignant diseases: Past, present, and future. Comput Biol Med 2022; 144:105334. [DOI: 10.1016/j.compbiomed.2022.105334] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 02/13/2022] [Accepted: 02/13/2022] [Indexed: 12/22/2022]
|
|
39
|
Zhu J, Liu M, Li X. Progress on deep learning in digital pathology of breast cancer: a narrative review. Gland Surg 2022; 11:751-766. [PMID: 35531111 PMCID: PMC9068546 DOI: 10.21037/gs-22-11] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Accepted: 03/04/2022] [Indexed: 01/26/2024]
Abstract
BACKGROUND AND OBJECTIVE Pathology is the gold standard criteria for breast cancer diagnosis and has important guiding value in formulating the clinical treatment plan and predicting the prognosis. However, traditional microscopic examinations of tissue sections are time consuming and labor intensive, with unavoidable subjective variations. Deep learning (DL) can evaluate and extract the most important information from images with less need for human instruction, providing a promising approach to assist in the pathological diagnosis of breast cancer. To provide an informative and up-to-date summary on the topic of DL-based diagnostic systems for breast cancer pathology image analysis and discuss the advantages and challenges to the routine clinical application of digital pathology. METHODS A PubMed search with keywords ("breast neoplasm" or "breast cancer") and ("pathology" or "histopathology") and ("artificial intelligence" or "deep learning") was conducted. Relevant publications in English published from January 2000 to October 2021 were screened manually for their title, abstract, and even full text to determine their true relevance. References from the searched articles and other supplementary articles were also studied. KEY CONTENT AND FINDINGS DL-based computerized image analysis has obtained impressive achievements in breast cancer pathology diagnosis, classification, grading, staging, and prognostic prediction, providing powerful methods for faster, more reproducible, and more precise diagnoses. However, all artificial intelligence (AI)-assisted pathology diagnostic models are still in the experimental stage. Improving their economic efficiency and clinical adaptability are still required to be developed as the focus of further researches. CONCLUSIONS Having searched PubMed and other databases and summarized the application of DL-based AI models in breast cancer pathology, we conclude that DL is undoubtedly a promising tool for assisting pathologists in routines, but further studies are needed to realize the digitization and automation of clinical pathology.
Collapse
Affiliation(s)
- Jingjin Zhu
- School of Medicine, Nankai University, Tianjin, China
| | - Mei Liu
- Department of Pathology, Chinese People’s Liberation Army General Hospital, Beijing, China
| | - Xiru Li
- Department of General Surgery, Chinese People’s Liberation Army General Hospital, Beijing, China
| |
Collapse
|
|
40
|
Lee J, Warner E, Shaikhouni S, Bitzer M, Kretzler M, Gipson D, Pennathur S, Bellovich K, Bhat Z, Gadegbeku C, Massengill S, Perumal K, Saha J, Yang Y, Luo J, Zhang X, Mariani L, Hodgin JB, Rao A. Unsupervised machine learning for identifying important visual features through bag-of-words using histopathology data from chronic kidney disease. Sci Rep 2022; 12:4832. [PMID: 35318420 PMCID: PMC8941143 DOI: 10.1038/s41598-022-08974-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 03/14/2022] [Indexed: 12/22/2022] Open
Abstract
Pathologists use visual classification to assess patient kidney biopsy samples when diagnosing the underlying cause of kidney disease. However, the assessment is qualitative, or semi-quantitative at best, and reproducibility is challenging. To discover previously unknown features which predict patient outcomes and overcome substantial interobserver variability, we developed an unsupervised bag-of-words model. Our study applied to the C-PROBE cohort of patients with chronic kidney disease (CKD). 107,471 histopathology images were obtained from 161 biopsy cores and identified important morphological features in biopsy tissue that are highly predictive of the presence of CKD both at the time of biopsy and in one year. To evaluate the performance of our model, we estimated the AUC and its 95% confidence interval. We show that this method is reliable and reproducible and can achieve 0.93 AUC at predicting glomerular filtration rate at the time of biopsy as well as predicting a loss of function at one year. Additionally, with this method, we ranked the identified morphological features according to their importance as diagnostic markers for chronic kidney disease. In this study, we have demonstrated the feasibility of using an unsupervised machine learning method without human input in order to predict the level of kidney function in CKD. The results from our study indicate that the visual dictionary, or visual image pattern, obtained from unsupervised machine learning can predict outcomes using machine-derived values that correspond to both known and unknown clinically relevant features.
Collapse
Affiliation(s)
- Joonsang Lee
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Elisa Warner
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Salma Shaikhouni
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Markus Bitzer
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Matthias Kretzler
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Debbie Gipson
- Department of Pediatrics, Pediatric Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Subramaniam Pennathur
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Keith Bellovich
- Department of Internal Medicine, Nephrology, St. Clair Nephrology Research, Detroit, MI, USA
| | - Zeenat Bhat
- Department of Internal Medicine, Nephrology, Wayne State University, Detroit, MI, USA
| | - Crystal Gadegbeku
- Department of Internal Medicine, Nephrology, Cleveland Clinic, Cleveland, OH, USA
| | - Susan Massengill
- Department of Pediatrics, Pediatric Nephrology, Levine Children's Hospital, Charlotte, NC, USA
| | - Kalyani Perumal
- Department of Internal Medicine, Nephrology, Department of JH Stroger Hospital, Chicago, IL, USA
| | - Jharna Saha
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Yingbao Yang
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Jinghui Luo
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Xin Zhang
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Laura Mariani
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Jeffrey B Hodgin
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA.
| | - Arvind Rao
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA.
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, USA.
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
|
41
|
Hussan H, Zhao J, Badu-Tawiah AK, Stanich P, Tabung F, Gray D, Ma Q, Kalady M, Clinton SK. Utility of machine learning in developing a predictive model for early-age-onset colorectal neoplasia using electronic health records. PLoS One 2022; 17:e0265209. [PMID: 35271664 PMCID: PMC9064446 DOI: 10.1371/journal.pone.0265209] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Accepted: 02/24/2022] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND AND AIMS The incidence of colorectal cancer (CRC) is increasing in adults younger than 50, and early screening remains challenging due to cost and under-utilization. To identify individuals aged 35-50 years who may benefit from early screening, we developed a prediction model using machine learning and electronic health record (EHR)-derived factors. METHODS We enrolled 3,116 adults aged 35-50 at average-risk for CRC and underwent colonoscopy between 2017-2020 at a single center. Prediction outcomes were (1) CRC and (2) CRC or high-risk polyps. We derived our predictors from EHRs (e.g., demographics, obesity, laboratory values, medications, and zip code-derived factors). We constructed four machine learning-based models using a training set (random sample of 70% of participants): regularized discriminant analysis, random forest, neural network, and gradient boosting decision tree. In the testing set (remaining 30% of participants), we measured predictive performance by comparing C-statistics to a reference model (logistic regression). RESULTS The study sample was 55.1% female, 32.8% non-white, and included 16 (0.05%) CRC cases and 478 (15.3%) cases of CRC or high-risk polyps. All machine learning models predicted CRC with higher discriminative ability compared to the reference model [e.g., C-statistics (95%CI); neural network: 0.75 (0.48-1.00) vs. reference: 0.43 (0.18-0.67); P = 0.07] Furthermore, all machine learning approaches, except for gradient boosting, predicted CRC or high-risk polyps significantly better than the reference model [e.g., C-statistics (95%CI); regularized discriminant analysis: 0.64 (0.59-0.69) vs. reference: 0.55 (0.50-0.59); P<0.0015]. The most important predictive variables in the regularized discriminant analysis model for CRC or high-risk polyps were income per zip code, the colonoscopy indication, and body mass index quartiles. DISCUSSION Machine learning can predict CRC risk in adults aged 35-50 using EHR with improved discrimination. Further development of our model is needed, followed by validation in a primary-care setting, before clinical application.
Collapse
Affiliation(s)
- Hisham Hussan
- Division of Gastroenterology, Hepatology, and Nutrition, Department of
Internal Medicine, The Ohio State University, Columbus, Ohio, United States of
America
- Comprehensive Cancer Center, The Ohio State University, Columbus, Ohio,
United States of America
| | - Jing Zhao
- Department of Biomedical Informatics, College of Medicine, The Ohio State
University, Columbus, Ohio, United States of America
| | - Abraham K. Badu-Tawiah
- Division of Gastroenterology, Hepatology, and Nutrition, Department of
Internal Medicine, The Ohio State University, Columbus, Ohio, United States of
America
- Department of Chemistry and Biochemistry, The Ohio State University,
Columbus, Ohio, United States of America
- Department of Microbial Infection and Immunity, The Ohio State
University, Columbus, Ohio, United States of America
| | - Peter Stanich
- Division of Gastroenterology, Hepatology, and Nutrition, Department of
Internal Medicine, The Ohio State University, Columbus, Ohio, United States of
America
| | - Fred Tabung
- Comprehensive Cancer Center, The Ohio State University, Columbus, Ohio,
United States of America
- Division of Medical Oncology, Department of Internal Medicine, College of
Medicine, The Ohio State University, Columbus, Ohio, United States of
America
| | - Darrell Gray
- Division of Gastroenterology, Hepatology, and Nutrition, Department of
Internal Medicine, The Ohio State University, Columbus, Ohio, United States of
America
- Comprehensive Cancer Center, The Ohio State University, Columbus, Ohio,
United States of America
| | - Qin Ma
- Department of Biomedical Informatics, College of Medicine, The Ohio State
University, Columbus, Ohio, United States of America
| | - Matthew Kalady
- Comprehensive Cancer Center, The Ohio State University, Columbus, Ohio,
United States of America
- Division of Colon and Rectal Surgery, Department of Surgery, The Ohio
State University, Columbus, Ohio, United States of America
| | - Steven K. Clinton
- Comprehensive Cancer Center, The Ohio State University, Columbus, Ohio,
United States of America
- Division of Medical Oncology, Department of Internal Medicine, College of
Medicine, The Ohio State University, Columbus, Ohio, United States of
America
| |
Collapse
|
|
42
|
Mallikarjuna B, Shrivastava G, Sharma M. Blockchain technology: A DNN token-based approach in healthcare and COVID-19 to generate extracted data. EXPERT SYSTEMS 2022; 39:e12778. [PMID: 34511692 PMCID: PMC8420355 DOI: 10.1111/exsy.12778] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Revised: 06/28/2021] [Accepted: 07/07/2021] [Indexed: 05/23/2023]
Abstract
The healthcare technologies in COVID-19 pandemic had grown immensely in various domains. Blockchain technology is one such turnkey technology, which is transforming the data securely; to store electronic health records (EHRs), develop deep learning algorithms, access the data, process the data between physicians and patients to access the EHRs in the form of distributed ledgers. Blockchain technology is also made to supply the data in the cloud and contact the huge amount of healthcare data, which is difficult and complex to process. As the complexity in the analysis of data is increasing day by day, it has become essential to minimize the risk of data complexity. This paper supports deep neural network (DNN) analysis in healthcare and COVID-19 pandemic and gives the smart contract procedure, to identify the feature extracted data (FED) from the existing data. At the same time, the innovation will be useful to analyse future diseases. The proposed method also analyze the existing diseases which had been reported and it is extremely useful to guide physicians in providing appropriate treatment and save lives. To achieve this, the massive data is integrated using Python scripting language under various libraries to perform a wide range of medical and healthcare functions to infer knowledge that assists in the diagnosis of major diseases such as heart disease, blood cancer, gastric and COVID-19.
Collapse
Affiliation(s)
- Basetty Mallikarjuna
- School of Computing Science and EngineeringGalgotias UniversityGreater NoidaIndia
| | - Gulshan Shrivastava
- Department of Computer Science and EngineeringSharda UniversityGreater NoidaIndia
| | - Meenakshi Sharma
- School of Computing Science and EngineeringGalgotias UniversityGreater NoidaIndia
| |
Collapse
|
|
43
|
Burati M, Tagliabue F, Lomonaco A, Chiarelli M, Zago M, Cioffi G, Cioffi U. Artificial intelligence as a future in cancer surgery. Artif Intell Cancer 2022; 3:11-16. [DOI: 10.35713/aic.v3.i1.11] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Revised: 12/24/2021] [Accepted: 01/17/2022] [Indexed: 02/06/2023] Open
Affiliation(s)
- Morena Burati
- Department of Robotic and Emergency Surgery, Ospedale A Manzoni, ASST Lecco, Lecco 23900, Italy
| | - Fulvio Tagliabue
- Department of Robotic and Emergency Surgery, Ospedale A Manzoni, ASST Lecco, Lecco 23900, Italy
| | - Adriana Lomonaco
- Department of Robotic and Emergency Surgery, Ospedale A Manzoni, ASST Lecco, Lecco 23900, Italy
| | - Marco Chiarelli
- Department of Robotic and Emergency Surgery, Ospedale A Manzoni, ASST Lecco, Lecco 23900, Italy
| | - Mauro Zago
- Department of Robotic and Emergency Surgery, Ospedale A Manzoni, ASST Lecco, Lecco 23900, Italy
| | - Gerardo Cioffi
- Department of Sciences and Technologies, Unisannio, Benevento 82100, Italy
| | - Ugo Cioffi
- Department of Surgery, University of Milan, Milano 20122, Italy
| |
Collapse
|
|
44
|
Rabbi F, Dabbagh SR, Angin P, Yetisen AK, Tasoglu S. Deep Learning-Enabled Technologies for Bioimage Analysis. MICROMACHINES 2022; 13:mi13020260. [PMID: 35208385 PMCID: PMC8880650 DOI: 10.3390/mi13020260] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 01/31/2022] [Accepted: 02/03/2022] [Indexed: 02/05/2023]
Abstract
Deep learning (DL) is a subfield of machine learning (ML), which has recently demonstrated its potency to significantly improve the quantification and classification workflows in biomedical and clinical applications. Among the end applications profoundly benefitting from DL, cellular morphology quantification is one of the pioneers. Here, we first briefly explain fundamental concepts in DL and then we review some of the emerging DL-enabled applications in cell morphology quantification in the fields of embryology, point-of-care ovulation testing, as a predictive tool for fetal heart pregnancy, cancer diagnostics via classification of cancer histology images, autosomal polycystic kidney disease, and chronic kidney diseases.
Collapse
Affiliation(s)
- Fazle Rabbi
- Department of Mechanical Engineering, Koç University, Sariyer, Istanbul 34450, Turkey; (F.R.); (S.R.D.)
| | - Sajjad Rahmani Dabbagh
- Department of Mechanical Engineering, Koç University, Sariyer, Istanbul 34450, Turkey; (F.R.); (S.R.D.)
- Koç University Arçelik Research Center for Creative Industries (KUAR), Koç University, Sariyer, Istanbul 34450, Turkey
- Koc University Is Bank Artificial Intelligence Lab (KUIS AILab), Koç University, Sariyer, Istanbul 34450, Turkey
| | - Pelin Angin
- Department of Computer Engineering, Middle East Technical University, Ankara 06800, Turkey;
| | - Ali Kemal Yetisen
- Department of Chemical Engineering, Imperial College London, London SW7 2AZ, UK;
| | - Savas Tasoglu
- Department of Mechanical Engineering, Koç University, Sariyer, Istanbul 34450, Turkey; (F.R.); (S.R.D.)
- Koç University Arçelik Research Center for Creative Industries (KUAR), Koç University, Sariyer, Istanbul 34450, Turkey
- Koc University Is Bank Artificial Intelligence Lab (KUIS AILab), Koç University, Sariyer, Istanbul 34450, Turkey
- Institute of Biomedical Engineering, Boğaziçi University, Çengelköy, Istanbul 34684, Turkey
- Physical Intelligence Department, Max Planck Institute for Intelligent Systems, 70569 Stuttgart, Germany
- Correspondence:
| |
Collapse
|
|
45
|
Tewary S, Mukhopadhyay S. AutoIHCNet: CNN architecture and decision fusion for automated HER2 scoring. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.108572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
|
46
|
McGenity C, Wright A, Treanor D. AIM in Surgical Pathology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
|
47
|
Garberis I, Andre F, Lacroix-Triki M. L’intelligence artificielle pourrait-elle intervenir dans l’aide au diagnostic des cancers du sein ? – L’exemple de HER2. Bull Cancer 2022; 108:11S35-11S45. [PMID: 34969514 DOI: 10.1016/s0007-4551(21)00635-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
HER2 is an important prognostic and predictive biomarker in breast cancer. Its detection makes it possible to define which patients will benefit from a targeted treatment. While assessment of HER2 status by immunohistochemistry in positive vs negative categories is well implemented and reproducible, the introduction of a new "HER2-low" category could raise some concerns about its scoring and reproducibility. We herein described the current HER2 testing methods and the application of innovative machine learning techniques to improve these determinations, as well as the main challenges and opportunities related to the implementation of digital pathology in the up-and-coming AI era.
Collapse
Affiliation(s)
- Ingrid Garberis
- Inserm UMR 981, Gustave Roussy Cancer Campus, Villejuif, France; Université Paris-Saclay, 94270 Le Kremlin-Bicêtre, France.
| | - Fabrice Andre
- Inserm UMR 981, Gustave Roussy Cancer Campus, Villejuif, France; Université Paris-Saclay, 94270 Le Kremlin-Bicêtre, France; Département d'oncologie médicale, Gustave-Roussy, Villejuif, France
| | - Magali Lacroix-Triki
- Inserm UMR 981, Gustave Roussy Cancer Campus, Villejuif, France; Département d'anatomie et cytologie pathologiques, Gustave-Roussy, Villejuif, France
| |
Collapse
|
|
48
|
Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod Pathol 2022; 35:23-32. [PMID: 34611303 PMCID: PMC8491759 DOI: 10.1038/s41379-021-00919-2] [Citation(s) in RCA: 249] [Impact Index Per Article: 83.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Revised: 08/18/2021] [Accepted: 08/30/2021] [Indexed: 02/07/2023]
Abstract
Traditional pathology approaches have played an integral role in the delivery of diagnosis, semi-quantitative or qualitative assessment of protein expression, and classification of disease. Technological advances and the increased focus on precision medicine have recently paved the way for the development of digital pathology-based approaches for quantitative pathologic assessments, namely whole slide imaging and artificial intelligence (AI)-based solutions, allowing us to explore and extract information beyond human visual perception. Within the field of immuno-oncology, the application of such methodologies in drug development and translational research have created invaluable opportunities for deciphering complex pathophysiology and the discovery of novel biomarkers and drug targets. With an increasing number of treatment options available for any given disease, practitioners face the growing challenge of selecting the most appropriate treatment for each patient. The ever-increasing utilization of AI-based approaches substantially expands our understanding of the tumor microenvironment, with digital approaches to patient stratification and selection for diagnostic assays supporting the identification of the optimal treatment regimen based on patient profiles. This review provides an overview of the opportunities and limitations around implementing AI-based methods in biomarker discovery and patient selection and discusses how advances in digital pathology and AI should be considered in the current landscape of translational medicine, touching on challenges this technology may face if adopted in clinical settings. The traditional role of pathologists in delivering accurate diagnoses or assessing biomarkers for companion diagnostics may be enhanced in precision, reproducibility, and scale by AI-powered analysis tools.
Collapse
|
|
49
|
Morelli R, Clissa L, Amici R, Cerri M, Hitrec T, Luppi M, Rinaldi L, Squarcio F, Zoccoli A. Automating cell counting in fluorescent microscopy through deep learning with c-ResUnet. Sci Rep 2021; 11:22920. [PMID: 34824294 PMCID: PMC8617067 DOI: 10.1038/s41598-021-01929-5] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2021] [Accepted: 11/03/2021] [Indexed: 02/06/2023] Open
Abstract
Counting cells in fluorescent microscopy is a tedious, time-consuming task that researchers have to accomplish to assess the effects of different experimental conditions on biological structures of interest. Although such objects are generally easy to identify, the process of manually annotating cells is sometimes subject to fatigue errors and suffers from arbitrariness due to the operator’s interpretation of the borderline cases. We propose a Deep Learning approach that exploits a fully-convolutional network in a binary segmentation fashion to localize the objects of interest. Counts are then retrieved as the number of detected items. Specifically, we introduce a Unet-like architecture, cell ResUnet (c-ResUnet), and compare its performance against 3 similar architectures. In addition, we evaluate through ablation studies the impact of two design choices, (i) artifacts oversampling and (ii) weight maps that penalize the errors on cells boundaries increasingly with overcrowding. In summary, the c-ResUnet outperforms the competitors with respect to both detection and counting metrics (respectively, \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$F_1$$\end{document}F1 score = 0.81 and MAE = 3.09). Also, the introduction of weight maps contribute to enhance performances, especially in presence of clumping cells, artifacts and confounding biological structures. Posterior qualitative assessment by domain experts corroborates previous results, suggesting human-level performance inasmuch even erroneous predictions seem to fall within the limits of operator interpretation. Finally, we release the pre-trained model and the annotated dataset to foster research in this and related fields.
Collapse
Affiliation(s)
- Roberto Morelli
- National Institute for Nuclear Physics, Bologna, Italy. .,Department of Physics and Astronomy, University of Bologna, Bologna, Italy.
| | - Luca Clissa
- National Institute for Nuclear Physics, Bologna, Italy.,Department of Physics and Astronomy, University of Bologna, Bologna, Italy
| | - Roberto Amici
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
| | - Matteo Cerri
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
| | - Timna Hitrec
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
| | - Marco Luppi
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
| | - Lorenzo Rinaldi
- National Institute for Nuclear Physics, Bologna, Italy.,Department of Physics and Astronomy, University of Bologna, Bologna, Italy
| | - Fabio Squarcio
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
| | - Antonio Zoccoli
- National Institute for Nuclear Physics, Bologna, Italy.,Department of Physics and Astronomy, University of Bologna, Bologna, Italy
| |
Collapse
|
|
50
|
Ruini C, Schlingmann S, Jonke Ž, Avci P, Padrón-Laso V, Neumeier F, Koveshazi I, Ikeliani IU, Patzer K, Kunrad E, Kendziora B, Sattler E, French LE, Hartmann D. Machine Learning Based Prediction of Squamous Cell Carcinoma in Ex Vivo Confocal Laser Scanning Microscopy. Cancers (Basel) 2021; 13:cancers13215522. [PMID: 34771684 PMCID: PMC8583634 DOI: 10.3390/cancers13215522] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Revised: 10/22/2021] [Accepted: 10/29/2021] [Indexed: 01/02/2023] Open
Abstract
Image classification with convolutional neural networks (CNN) offers an unprecedented opportunity to medical imaging. Regulatory agencies in the USA and Europe have already cleared numerous deep learning/machine learning based medical devices and algorithms. While the field of radiology is on the forefront of artificial intelligence (AI) revolution, conventional pathology, which commonly relies on examination of tissue samples on a glass slide, is falling behind in leveraging this technology. On the other hand, ex vivo confocal laser scanning microscopy (ex vivo CLSM), owing to its digital workflow features, has a high potential to benefit from integrating AI tools into the assessment and decision-making process. Aim of this work was to explore a preliminary application of CNN in digitally stained ex vivo CLSM images of cutaneous squamous cell carcinoma (cSCC) for automated detection of tumor tissue. Thirty-four freshly excised tissue samples were prospectively collected and examined immediately after resection. After the histologically confirmed ex vivo CLSM diagnosis, the tumor tissue was annotated for segmentation by experts, in order to train the MobileNet CNN. The model was then trained and evaluated using cross validation. The overall sensitivity and specificity of the deep neural network for detecting cSCC and tumor free areas on ex vivo CLSM slides compared to expert evaluation were 0.76 and 0.91, respectively. The area under the ROC curve was equal to 0.90 and the area under the precision-recall curve was 0.85. The results demonstrate a high potential of deep learning models to detect cSCC regions on digitally stained ex vivo CLSM slides and to distinguish them from tumor-free skin.
Collapse
Affiliation(s)
- Cristel Ruini
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
- PhD School in Clinical and Experimental Medicine, University of Modena and Reggio Emilia, 41125 Modena, Italy
- Correspondence:
| | - Sophia Schlingmann
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| | - Žan Jonke
- Munich Innovation Labs GmbH, 80336 Munich, Germany; (Ž.J.); (V.P.-L.)
| | - Pinar Avci
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| | | | - Florian Neumeier
- M3i Industry-in-Clinic-Platform GmbH, 80336 Munich, Germany; (F.N.); (I.K.); (I.U.I.)
| | - Istvan Koveshazi
- M3i Industry-in-Clinic-Platform GmbH, 80336 Munich, Germany; (F.N.); (I.K.); (I.U.I.)
| | - Ikenna U. Ikeliani
- M3i Industry-in-Clinic-Platform GmbH, 80336 Munich, Germany; (F.N.); (I.K.); (I.U.I.)
| | - Kathrin Patzer
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| | - Elena Kunrad
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| | - Benjamin Kendziora
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| | - Elke Sattler
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| | - Lars E. French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
- Dr. Phillip Frost Department of Dermatology & Cutaneous Surgery, Miller School of Medicine, University of Miami, Miami, FL 33136, USA
| | - Daniela Hartmann
- Department of Dermatology and Allergy, University Hospital, LMU Munich, 80337 Munich, Germany; (S.S.); (P.A.); (K.P.); (E.K.); (B.K.); (E.S.); (L.E.F.); (D.H.)
| |
Collapse
|