CaloChallenge 2022: a community challenge for fast calorimeter simulation
Review article, 2025
We present the results of the 'Fast Calorimeter Simulation Challenge 2022'-the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including variational autoencoders (VAEs), generative adversarial networks (GANs), normalizing flows, diffusion models, and models based on conditional flow matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in one-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space.Report Numbers: HEPHY-ML-24-05, FERMILAB-PUB-24-0728-CMS, TTK-24-43.
calorimeter
generative AI
machine learning
simulation
CaloChallenge 2022
Author
Claudius Krause
Austrian Academy of Sciences
Heidelberg University
Michele Faucci Giannelli
Chalmers, Microtechnology and Nanoscience (MC2), Quantum Technology
National Institute for Nuclear Physics
Gregor Kasieczka
University of Hamburg
Benjamin Nachman
Lawrence Berkeley National Laboratory
Dalila Salamani
CERN
David Shih
Rutgers University
Anna Zaborowska
CERN
Oz Amram
Fermi National Accelerator Laboratory
Kerstin Borras
Deutsches Elektronen-Synchrotron (DESY)
RWTH Aachen University
Matthew R. Buckley
Rutgers University
Erik Buhmann
University of Hamburg
Thorsten Buss
Deutsches Elektronen-Synchrotron (DESY)
University of Hamburg
Renato Paulo Da Costa Cardoso
CERN
Anthony L. Caterini
Layer 6 AI
Nadezda Chernyavskaya
CERN
Federico A.G. Corchia
University of Bologna
National Institute for Nuclear Physics
Jesse C. Cresswell
Layer 6 AI
Sascha Diefenbacher
Lawrence Berkeley National Laboratory
Etienne Dreyer
Weizmann Institute of Science
Vijay Ekambaram
IBM
Engin Eren
Deutsches Elektronen-Synchrotron (DESY)
Florian Ernst
CERN
Institut für Theoretische Physik Heidelberg
Luigi Favaro
Heidelberg University
Matteo Franchini
University of Bologna
National Institute for Nuclear Physics
Frank Gaede
Deutsches Elektronen-Synchrotron (DESY)
Eilam Gross
Weizmann Institute of Science
Shih Chieh Hsu
University of Washington
Kristina Jaruskova
CERN
Benno Käch
University of Hamburg
Deutsches Elektronen-Synchrotron (DESY)
Jayant Kalagnanam
IBM
Raghav Kansal
Fermi National Accelerator Laboratory
California Institute of Technology (Caltech)
Taewoo Kim
Layer 6 AI
Dmitrii Kobylianskii
Weizmann Institute of Science
Anatolii Korol
Deutsches Elektronen-Synchrotron (DESY)
William Korcari
University of Hamburg
Dirk Krücker
Deutsches Elektronen-Synchrotron (DESY)
Katja Krüger
Deutsches Elektronen-Synchrotron (DESY)
Marco Letizia
University of Genoa
National Institute for Nuclear Physics
Shu Li
Shanghai Jiao Tong University
Qibin Liu
Shanghai Jiao Tong University
Xiulong Liu
University of Washington
Gabriel Loaiza-Ganem
Layer 6 AI
Thandikire Madula
University College London (UCL)
Peter McKeown
Deutsches Elektronen-Synchrotron (DESY)
CERN
Isabell A. Melzer-Pellmann
Deutsches Elektronen-Synchrotron (DESY)
Vinicius Mikuni
Lawrence Berkeley National Laboratory
Nam Nguyen
IBM
Ayodele Ore
Heidelberg University
Sofia Palacios Schweitzer
Heidelberg University
Ian Pang
Rutgers University
Kevin Pedro
Fermi National Accelerator Laboratory
Tilman Plehn
Heidelberg University
Witold Pokorski
CERN
Huilin Qu
CERN
Piyush Raikwar
CERN
John A. Raine
University of Geneva
Humberto Reyes-Gonzalez
RWTH Aachen University
University of Genoa
National Institute for Nuclear Physics
Lorenzo Rinaldi
National Institute for Nuclear Physics
University of Bologna
Brendan Leigh Ross
Layer 6 AI
Moritz A.W. Scham
Deutsches Elektronen-Synchrotron (DESY)
Jülich Research Centre
RWTH Aachen University
Simon Schnake
RWTH Aachen University
Deutsches Elektronen-Synchrotron (DESY)
Chase Shimmin
Yale University
Eli Shlizerman
University of Washington
Nathalie Soybelman
Weizmann Institute of Science
Mudhakar Srivatsa
IBM
Kalliopi Tsolaki
CERN
Sofia Vallecorsa
CERN
Kyongmin Yeo
IBM
Rui Zhang
Nanjing University
University of Wisconsin Madison
Reports on Progress in Physics
0034-4885 (ISSN) 1361-6633 (eISSN)
Vol. 88 11Subject Categories (SSIF 2025)
Computer Systems
DOI
10.1088/1361-6633/ae1304
PubMed
41086821
Related datasets
Fast Calorimeter Simulation Challenge 2022 - Dataset 1 [dataset]
URI: https://zenodo.org/records/8099322