Eva Optimizer: Escaping Low-Curvature Traps in Deep Learning
Paper in proceeding, 2027
Adaptive optimization methods like Adam excel in handling sparse gradients and varying parameter scales, but risk instability in regions of low curvature due to excessive step sizes. We introduce Eva (Escaping Via Adaptive gradients), a family of novel optimizers comprising two variants—Eva+ and Eva-—that dynamically switch between Adam and scaled gradient steps based on local gradient and curvature properties. Our method implements component-wise detection to identify updates where Adam step magnitude exceeds the historical gradient norm. Eva+ employs gradient ascent to escape from degenerate regions, while Eva- provides conservative descent with magnitude damping. We set a threshold for detecting unstable updates, supported by theoretical analysis and empirical validation. Extensive experiments prove that both Eva variants achieve faster convergence and improved generalization compared to popular optimizers.
Gradient-based methods
Low-curvature optimization
Escape mechanisms
Deep learning
Adaptive optimization
Author
Antonio Di Cecco
Università Campus Bio-Medico di Roma
G. d'Annunzio University of Chieti-Pescara
Carlo Metta
Consiglo Nazionale Delle Richerche
Andrea Papini
Chalmers, Mathematical Sciences, Applied Mathematics and Statistics
University of Gothenburg
Marco Fantozzi
University of Parma
Silvia Giulia Galfré
University of Pisa
Michelangelo Vegliò
Università Campus Bio-Medico di Roma
G. d'Annunzio University of Chieti-Pescara
Luigi Amedeo Bianchi
University of Trento
Maurizio Parton
G. d'Annunzio University of Chieti-Pescara
Francesco Morandin
University of Pisa
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
03029743 (ISSN) 16113349 (eISSN)
Vol. 16816 LNCS 112-1279783032316653 (ISBN)
28th International Conference on Pattern Recognition, ICPR 2026
Lyon, France,
Lyon, France,
Subject Categories (SSIF 2025)
Computational Mathematics
Mathematical Analysis
DOI
10.1007/978-3-032-31666-0_8