The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI.

Journal: ArXiv

Volume: 

Issue: 

Year of Publication: 

Affiliated Institutions:  Trinity health Mid Atlantic Hosiptals, Darby, PA, USA. Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, USA. Division of Computational Pathology, Department of Pathology and Laboratory Medicine, School of Medicine, Indiana University, Indianapolis, IN, USA. Department of Electical and Computer Engineering, Cornell University and Cornell Tech, New York, NY, USA. ImagineQuant, Children's Hospital of Philadelphia, Philadelphia, PA, USA. DKFZ Division of Translational Neurooncology at the WTZ, German Cancer Consortium, DKTK Partner Site, University Hospital Essen, Essen, Germany. Medical Artificial Intelligence Lab, Crestview Radiology, Lagos, Nigeria. Sage Bionetworks, Seattle, WA, USA. Montreal Neurological Institute, McGill University, Montreal, Canada. Department of Therapeutic Radiology, Yale School of Medicine, New Haven, CT, USA. Sheikh Zayed Institute for Pediatric Surgical Innovation, Children's National Hospital, Washington, D.C., USA. Department of Radiology, Mayo Clinic, Rochester, MN, USA. Department of Radiology, Duke University Medical Center, Durham, NC, USA. Department of Neurosurgery, Yale School of Medicine, New Haven, CT, USA. Center for Global Health, Perelman School of Medicine, University of Pennsylvania, PA, USA. Department of Informatics, Technical University Munich, Germany. Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, USA. Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA. Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA. Children's National Hospital, Washington, D.C., USA. PrecisionFDA, U.S. Food and Drug Administration, Silver Spring, MD, USA. Department of Neurosurgery, University of Pennsylvania, Philadelphia, PA, USA. Technische Universität München, Munich, Germany. Department of Radiation Oncology, Duke University Medical Center, Durham, NC, USA. Department of Applied Mathematics and Computer Science, Technical University of Denmark, Denmark. New York University School of Medicine, New York, NY, USA. Booz Allen Hamilton, McLean, VA, USA. Biomedical Image Analysis & Machine Learning, Department of Quantitative Biomedicine, University of Zurich, Switzerland. Helmholtz AI, Helmholtz Munich, Germany. Duke University School of Medicine, Durham, NC, USA. Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania, Philadelphia, PA, USA. Radiology Department, University of Missouri, Columbia, MO, USA. Northwestern University, Department of Radiology, Feinberg School of Medicine, Chicago, IL, USA. Department of Neuroradiology, Technical University of Munich, Munich, Germany. Department of Radiology and Imaging Sciences, Indiana University, Indianapolis, IN, USA. Visage Imaging, GmbH, Berlin, Germany. Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, USA. Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, MO, USA. Visage Imaging, Inc, San Diego, CA, USA. University of Missouri, Columbia, MO, USA. Cairo University, Cairo, Egypt. Duke University Medical Center, Durham, NC, USA. Department of Radiology and Biomedical Imaging, University of California San Francisco, CA, USA. Department of Neuroradiology, Heidelberg University Hospital, Heidelberg, Germany. Department of Radiology, Mayo Clinic, Phoenix, AZ, USA. Neuroradiology Section, Mallinckrodt Institute of Radiology at Washington University in St. Louis, St. Louis, MO, USA. Loyola University Medical Center, Hines, IL, USA. Department of Radiology, Queen's University, Kingston, ON, Canada. Department of Radiology, University of Iowa Hospitals and Clinics, Iowa City, IA, USA. Children's healthcare of Atlanta, GA, USA. Carolina Radiology Associates, Myrtle Beach, SC, USA. University of Arkansas medical center, Little Rock, AR, USA. NorthShore Endeavor Health, Evanston, IL, USA. Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong SAR. Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom. Department of Radiology, University of Washington, Seattle, WA, USA. Department of Radiology, Ohio State University College of medicine, Columbus, OH, USA. Albert Einstein Medical Center, Hartford, CT, USA. Amsterdam UMC, location Vrije Universiteir, the Netherlands. Hospital Italiano de Buenos Aires, Buenos Aires, Argentina. Department of Radiology and Medical Imaging, University of Virginia, Charlottesville, Virginia, USA. Klinikum Hochrhein, Waldshut-Tiengen, Germany. Centro Universitario Euro-Americana (UNIEURO), Brasília, DF, Brazil. Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA. Southern District Health Board, Dunedin, New Zealand. Department of Radiology, Houston Methodist, Houston, TX, USA. University of Tennessee medical Center, Knoxville, TN, USA. Mayo Clinic, Department of Radiology, Section of Neuroradiology, Phoenix, AZ, USA. Department of Radiology, Stanford University, Stanford, CA, USA. Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, Maastricht, the Netherlands. Department of Radiology and Medical Imaging, University of Virginia, Charlottesville, VA, USA. Department of Radiology, University of Northwestern, Chicago, IL, USA. Centre Hospitalier de l'Universite de Montreal and Centre de Recherche du CHUM Montreal, Canada. Department of Neuroradiology, MD Anderson Cancer Center, Houston, TX, USA. Departments of Neuroradiology & Biomedical Informatics, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany. Department of Diagnostic and Interventional Radiology, SickKids Hospital, University of Toronto, Canada. Department of Radiology, Baylor College of medicine, Houston, TX, USA. Department of Radiology, New Jersey Medical school, Newark, NJ, USA. Department of Radiology, AZ Monica, Antwerp Area, Belgium. Medicolegal Imaging Experts LLC, Mercer Island, WA, USA. Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA. Memorial Sloan Kettering Cancer Center, New York, NY, USA. MedStar Georgetown University Hospital, Washington, D.C., USA. Department of Radiology, St.Elizabeth's Medical Center, Boston, MA, USA. Walter Reed National Military Medical Center, Bethesda, MD, USA. Keck School of medicine, Los Angeles, CA, USA. Lahey Health Home, Burlington, MA, USA. University of Alabama, Birmingham, AL, USA. Department of Radiology, University of North Carolina School of Medicine, Chapel Hill, NC, USA. Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA. University of Nebraska Medical Center, Omaha, NE, USA. George Washington University, Washington, D.C., USA. Northwell Health, Zucker Hofstra School of Medicine at Northwell, North Shore University Hospital, Hempstead, New York, NY, USA. Cancer Center Amsterdam, Imaging and Biomarkers, Amsterdam, The Netherlands. Ludwig Maximilians University Munich, München, Germany. Faculty of Medicine, Jena University Hospital, Friedrich Schiller University Jena, Jena, Germany. Department of Radiology, Ain Shams University, Cairo, Egypt. Department of Radiology, Ain Shams University, Cairo - Egypt. University of Ioannina School of Medicine, Ioannina, Greece. University of Hamburg, Hamburg, Germany. Radiology Department, Memorial Sloan Kettering Cancer Center, New York City, NY, USA. Iran University of Medical sciences, Tehran, Iran. Columbia University Irving Medical Center, New York, NY, USA. Department of Diagnostic and Interventional Radiology, University Hospital Ulm, Ulm, Germany. Medical School, University of Ioannina, Ioannina, Greece. Department of Radiology, Arad Hospital, Tehran, Iran. Faculty of Medicine, Medical University - Sofia, Sofia, Bulgaria. Functional and Interventional Neuroradiology Unit, Bambino Gesù Children's Hospital, Rome, Italy. Institute of Neuroscience and Medicine (INM-), Research Center Juelich, Juelich, Germany. Mathematical Oncology Laboratory & Department of Mathematics, University of Castilla-La Mancha, Spain. Department of Radiation Oncology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA. Charité-Universitätsmedizin Berlin (Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health), Berlin, Germany. Department of Neuroradiology, Western Lisbon Hospital Centre (CHLO), Portugal. Zagazig University, Zagazig, Egypt. Diagnostic Radiology Department, Wayne State University, Detroit, MI. Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland. Faculty of Medicine, Guilan University of Medical Sciences, Rasht, Iran. Department of Radiology/Division of Neuroradiology, San Diego Veterans Administration Medical Center/UC San Diego Health System, San Diego, CA, USA. Department of Radiology, University of Calgary, Calgary, Canada. EDU Institute of Higher Education, Villa Bighi, Chaplain's House, Kalkara, Malta. Bay Imaging Consultants, Walnut Creek, CA, USA. Ross University School of Medicine, Bridgetown, Barbados. Department of Radiology, Cairo University, Cairo, Egypt. Department of Radiology, Mayo clinic, Rochester, MN, USA. Department of Neurosurgery, Vivantes Klinikum Neukölln, Berlin, Germany. Mercy Catholic Medical Center, Darby, PA, USA. C.M.H. Lahore Medical College, Lahore, Pakistan. Neuroradiology Department, Pedro Hispano Hospital, Matosinhos, Portugal. Department of Radiology, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark. Department of Radiology, Baylor College of Medicine, Houston, TX, USA. Department of Radiology, University of Iowa Hospital and Clinics, Iowa City, IA, USA. National and Kapodistrian University of Athens, School of Medicine, Athens, Greece. Department of Neurosurgery, University Hospital of Ioannina, Ioannina, Greece. Department of Diagnostic and Interventional Radiology, Cairo University, Cairo, Egypt. Department of Radiology, Brigham and Women's Hospital, Massachusetts General Hospital, Boston, MA, USA. Department of Neuroradiology, Universidad Autónoma de Nuevo León, México. Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, USA. Gold Coast University Hospital, Queensland Health, Australia. Department of Radiology Manchester NHS Foundation Trust, North West School of Radiology, Manchester, United Kingdom. Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA. Department of Radiology, MD Anderson Cancer Center, Houston, TX, USA. Corewell Health West, MI, USA. Department of Radiodiagnosis, All India Institute of Medical Sciences Rishikesh, India. Windsor Regional Hospital, Western University, Ontario, Canada. Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA. Department of Radiology, University of Pennsylvania, PA, USA. Department of Radiology, Life Care Hospital, Freetown, Sierra Leone. Department of Medicine and Surgery, Università degli Studi di Perugia, Italy. Department of Neuroradiology, Imperial College Healthcare NHS Trust, London, United Kingdom. Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA. Department of Radiology, Michigan Medicine, Ann Arbor, MI, USA. Department of Radiology, University of Vermont Medical Center, Burlington, VT, USA. Isfahan University of Medical Sciences, Isfahan, Iran. Department of Radiology, The American British Cowdray Medical Center, Mexico City, Mexico. Rush University Medical Center, Chicago, IL, USA. Department of NeuroRadiology, Rockefeller Neuroscience Institute, West Virginia University. Morgantown, WV, USA. Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom. University of Cagliari, School of Medicine and Surgery, Cagliari, Italy. Department of Radiology, University of Arkansas for Medical Sciences, Little Rock, AR, USA. Washington University School of Medicine in St. Louis, St. Louis, MO, USA. Department of Radiology, Children's Hospital of Philadelphia, Philadelphia, PA, USA. MLCommons, San Francisco, CA, USA. Center For Federated Learning in Medicine, Indiana University, Indianapolis, IN, USA. Intel/MLCommons, Hillsboro, OR, USA. Department of Radiology, University of California San Diego, CA, USA.

Abstract summary 

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.

Authors & Co-authors:  Moawad Ahmed W AW Janas Anastasia A Baid Ujjwal U Ramakrishnan Divya D Saluja Rachit R Ashraf Nader N Jekel Leon L Amiruddin Raisa R Adewole Maruf M Albrecht Jake J Anazodo Udunna U Aneja Sanjay S Anwar Syed Muhammad SM Bergquist Timothy T Calabrese Evan E Chiang Veronica V Chung Verena V Conte Gian Marco Marco GMM Dako Farouk F Eddy James J Ezhov Ivan I Familiar Ariana A Farahani Keyvan K Iglesias Juan Eugenio JE Jiang Zhifan Z Johanson Elaine E Kazerooni Anahita Fathi AF Kofler Florian F Krantchev Kiril K LaBella Dominic D Van Leemput Koen K Li Hongwei Bran HB Linguraru Marius George MG Link Katherine E KE Liu Xinyang X Maleki Nazanin N Meier Zeke Z Menze Bjoern H BH Moy Harrison H Osenberg Klara K Piraud Marie M Reitman Zachary Z Shinohara Russel Takeshi RT Tahon Nourel Hoda NH Nada Ayman A Velichko Yuri S YS Wang Chunhao C Wiestler Benedikt B Wiggins Walter W Shafique Umber U Willms Klara K Avesta Arman A Bousabarah Khaled K Chakrabarty Satrajit S Gennaro Nicolo N Holler Wolfgang W Kaur Manpreet M LaMontagne Pamela P Lin MingDe M Lost Jan J Marcus Daniel S DS Maresca Ryan R Merkaj Sarah S Nada Ayaman A Pedersen Gabriel Cassinelli GC von Reppert Marc M Sotiras Aristeidis A Teytelboym Oleg O Tillmans Niklas N Westerhoff Malte M Youssef Ayda A Godfrey Devon D Floyd Scott S Rauschecker Andreas A Villanueva-Meyer Javier J Pflüger Irada I Cho Jaeyoung J Bendszus Martin M Brugnara Gianluca G Cramer Justin J Perez-Carillo Gloria J Guzman GJG Johnson Derek R DR Kam Anthony A Kwan Benjamin Yin Ming BYM Lai Lillian L Lall Neil U NU Memon Fatima F Patro Satya Narayana SN Petrovic Bojan B So Tiffany Y TY Thompson Gerard G Wu Lei L Schrickel E Brooke EB Bansal Anu A Barkhof Frederik F Besada Cristina C Chu Sammy S Druzgal Jason J Dusoi Alexandru A Farage Luciano L Feltrin Fabricio F Fong Amy A Fung Steve H SH Gray R Ian RI Ikuta Ichiro I Iv Michael M Postma Alida A AA Mahajan Amit A Joyner David D Krumpelman Chase C Letourneau-Guillon Laurent L Lincoln Christie M CM Maros Mate E ME Miller Elka E Morón Fanny F Nimchinsky Esther A EA Ozsarlak Ozkan O Patel Uresh U Rohatgi Saurabh S Saha Atin A Sayah Anousheh A Schwartz Eric D ED Shih Robert R Shiroishi Mark S MS Small Juan E JE Tanwar Manoj M Valerie Jewels J Weinberg Brent D BD White Matthew L ML Young Robert R Zohrabian Vahe M VM Azizova Aynur A Brüßeler Melanie Maria Theresa MMT Fehringer Pascal P Ghonim Mohanad M Ghonim Mohamed M Gkampenis Athanasios A Okar Abdullah A Pasquini Luca L Sharifi Yasaman Y Singh Gagandeep G Sollmann Nico N Soumala Theodora T Taherzadeh Mahsa M Yordanov Nikolay N Vollmuth Philipp P Foltyn-Dumitru Martha M Malhotra Ajay A Abayazeed Aly H AH Dellepiane Francesco F Lohmann Philipp P Pérez-García Víctor M VM Elhalawani Hesham H Al-Rubaiey Sanaria S Armindo Rui Duarte RD Ashraf Kholod K Asla Moamen M MM Badawy Mohamed M Bisschop Jeroen J Lomer Nima Broomand NB Bukatz Jan J Chen Jim J Cimflova Petra P Corr Felix F Crawley Alexis A Deptula Lisa L Elakhdar Tasneem T Shawali Islam H IH Faghani Shahriar S Frick Alexandra A Gulati Vaibhav V Haider Muhammad Ammar MA Hierro Fátima F Dahl Rasmus Holmboe RH Jacobs Sarah Maria SM Hsieh Kuang-Chun Jim KJ Kandemirli Sedat G SG Kersting Katharina K Kida Laura L Kollia Sofia S Koukoulithras Ioannis I Li Xiao X Abouelatta Ahmed A Mansour Aya A Maria-Zamfirescu Ruxandra-Catrinel RC Marsiglia Marcela M Mateo-Camacho Yohana Sarahi YS McArthur Mark M McDonnell Olivia O McHugh Maire M Moassefi Mana M Morsi Samah Mostafa SM Muntenu Alexander A Nandolia Khanak K KK Naqvi Syed Raza SR Nikanpour Yalda Y Alnoury Mostafa M Nouh Abdullah Mohamed Aly AMA Pappafava Francesca F Patel Markand D MD Petrucci Samantha S Rawie Eric E Raymond Scott S Roohani Borna B Sabouhi Sadeq S Sanchez-Garcia Laura M LM Shaked Zoe Z Suthar Pokhraj P PP Altes Talissa T Isufi Edvin E Dhermesh Yaseen Y Gass Jaime J Thacker Jonathan J Tarabishy Abdul Rahman AR Turner Benjamin B Vacca Sebastiano S Vilanilam George K GK Warren Daniel D Weiss David D Willms Klara K Worede Fikadu F Yousry Sara S Lerebo Wondwossen W Aristizabal Alejandro A Karargyris Alexandros A Kassem Hasan H Pati Sarthak S Sheller Micah M Bakas Spyridon S Rudie Jeffrey D JD Aboian Mariam M

Study Outcome 

Source Link: Visit source

Statistics
Citations :  Bousabarah Khaled, Ruge Maximilian, Brand Julia-Sarita, Hoevels Mauritius, Rueß Daniel, Borggrefe Jan, Hokamp Nils Große, Visser-Vandewalle Veerle, Maintz David, Treuer Harald, et al. Deep convolutional neural networks for automated segmentation of brain metastases trained on clinical data. Radiation Oncology, 15:1–9, 2020.
Authors :  231
Identifiers
Doi : arXiv:2306.00838v2
SSN : 2331-8422
Study Population
Male,Female
Mesh Terms
Other Terms
Artificial Intelligence;BraTS;BraTS-METS;Brain metastasis;Brain tumor segmentation;Machine learning;Medical image analysis challenge
Study Design
Study Approach
Country of Study
Publication Country
United States