Cross-Modality Augmentation of Brain MR Images Using a Novel Pairwise Generative Adversarial Network for Enhanced Glioma Classification
Paper in proceedings, 2019

Brain Magnetic Resonance Images (MRIs) are commonly used for tumor diagnosis. Machine learning for brain tumor characterization often uses MRIs from many modalities (e.g., T1-MRI, Enhanced-T1-MRI, T2-MRI and FLAIR). This paper tackles two issues that may impact brain tumor characterization performance from deep learning: insufficiently large training dataset, and incomplete collection of MRIs from different modalities. We propose a novel pairwise generative adversarial network (GAN) architecture for generating synthetic brain MRIs in missing modalities by using existing MRIs in other modalities. By improving the training dataset, we aim to mitigate the overfitting and improve the deep learning performance. Main contributions of the paper include: (a) propose a pairwise generative adversarial network (GAN) for brain image augmentation via cross-modality image generation; (b) propose a training strategy to enhance the glioma classification performance, where GAN-augmented images are used for pre-training, followed by refined-training using real brain MRIs; (c) demonstrate the proposed method through tests and comparisons of glioma classifiers that are trained from mixing real and GAN synthetic data, as well as from real data only. Experiments were conducted on an open TCGA dataset, containing 167 subjects for classifying IDH genotypes (mutation or wild-type). Test results from two experimental settings have both provided supports to the proposed method, where glioma classification performance has consistently improved by using mixed real and augmented data (test accuracy 81.03%, with 2.57% improvement). © 2019 IEEE.

cross-modality image augmentation

glioma classification

brain tumor

Pairwise generative adversarial network (GAN)

MR images


Chenjie Ge

Shanghai Jiao Tong University

Chalmers, Electrical Engineering, Signalbehandling och medicinsk teknik, Imaging and Image Analysis

Irene Yu-Hua Gu

Chalmers, Electrical Engineering

Asgeir Store Jakola

University of Gothenburg

Jie Yang

Shanghai Jiao Tong University

Proceedings - International Conference on Image Processing, ICIP

15224880 (ISSN)

Vol. September 559-563 8803808

26th IEEE International Conference on Image Processing, ICIP 2019; Taipei International Convention Center (TICC)
Taipei, Taiwan,

Subject Categories

Other Computer and Information Science

Computer Vision and Robotics (Autonomous Systems)

Medical Image Processing



More information

Latest update