ISLES 2015-A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI
Journal article, 2017
Ischemic stroke is the most common cerebrovascular disease, and its diagnosis, treatment, and study relies on non-invasive imaging. Algorithms for stroke lesion segmentation from magnetic resonance imaging (MRI) volumes are intensely researched, but the reported results are largely incomparable due to different datasets and evaluation schemes. We approached this urgent problem of comparability with the Ischemic Stroke Lesion Segmentation (ISLES) challenge organized in conjunction with the MICCAI 2015 conference. In this paper we propose a common evaluation framework, describe the publicly available datasets, and present the results of the two sub-challenges: Sub-Acute Stroke Lesion Segmentation (SISS) and Stroke Perfusion Estimation (SPES). A total of 16 research groups participated with a wide range of state-of-the-art automatic segmentation algorithms. A thorough analysis of the obtained data enables a critical evaluation of the current state-of-the-art, recommendations for further developments, and the identification of remaining challenges. The segmentation of acute perfusion lesions addressed in SPES was found to be feasible. However, algorithms applied to sub-acute lesion segmentation in SISS still lack accuracy. Overall, no algorithmic characteristic of any method was found to perform superior to the others. Instead, the characteristics of stroke lesion appearances, their evolution, and the observed challenges should be studied in detail. The annotated ISLES image datasets continue to be publicly available through an online evaluation system to serve as an ongoing benchmarking resource (www.isles-challenge.org).
Comparison
MRI
Ischemic stroke
Benchmark
Segmentation
Challenge
Author
O. Maier
Universitaet Zu Lübeck
Bjoern H. Menze
Technical University of Munich
J. von der Gablentz
Universitaet Zu Lübeck
L. Hani
University of Bern
Mattias Heinrich
Universitaet Zu Lübeck
M. Liebrand
Universitaet Zu Lübeck
S. Winzeck
Technical University of Munich
A. Basit
Pakistan Institute of Nuclear Science and Technology
P. Bentley
Imperial College London
L. Chen
Imperial College London
D. Christiaens
KU Leuven– University Hospital Leuven
KU Leuven
F. Dutil
Université de Sherbrooke
K. Egger
Universitats Klinikum Freiburg und Medizinische Fakultat
C. L. Feng
Northeastern University China
B. Glocker
Imperial College London
M. Gotz
German Cancer Research Center (DKFZ)
T. Haeck
KU Leuven
KU Leuven– University Hospital Leuven
H. L. Halme
University of Helsinki
Aalto University
M. Havaei
Université de Sherbrooke
K. M. Iftekharuddin
Old Dominion University
P. M. Jodoin
Université de Sherbrooke
K. Kamnitsas
Imperial College London
E. Kellner
Universitats Klinikum Freiburg und Medizinische Fakultat
A. Korvenoja
University of Helsinki
H. Larochelle
Université de Sherbrooke
C. Ledig
Imperial College London
J. H. Lee
National Taiwan University of Science and Technology
F. Maes
KU Leuven– University Hospital Leuven
KU Leuven
Mahmood Qaiser
Chalmers, Signals and Systems, Signal Processing and Biomedical Engineering
K. H. Maier-Hein
German Cancer Research Center (DKFZ)
R. McKinley
UniversitatsSpital Bern
J. Muschelli
Johns Hopkins University
C. Pal
École Polytechnique de Montréal
L. M. Pei
Old Dominion University
J. R. Rangarajan
KU Leuven
KU Leuven– University Hospital Leuven
S. M. S. Reza
Old Dominion University
D. Robben
KU Leuven– University Hospital Leuven
KU Leuven
D. Rueckert
Imperial College London
E. Salli
University of Helsinki
P. Suetens
KU Leuven– University Hospital Leuven
KU Leuven
C. W. Wang
National Taiwan University of Science and Technology
M. Wilms
Universitaet Zu Lübeck
J. S. Kirschke
Klinikum rechts der Isar
U. M. Kramer
Universitaet Zu Lübeck
T. F. Munte
Universitaet Zu Lübeck
P. Schramme
University Medical Center Lübeck
R. Wiest
UniversitatsSpital Bern
H. Handels
Universitaet Zu Lübeck
M. Reyes
University of Bern
Medical Image Analysis
1361-8415 (ISSN) 13618423 (eISSN)
Vol. 35 250-269Subject Categories
Computer Vision and Robotics (Autonomous Systems)
DOI
10.1016/j.media.2016.07.009
PubMed
27475911