Eva Optimizer: Escaping Low-Curvature Traps in Deep Learning
Paper i proceeding, 2027
Adaptive optimization methods like Adam excel in handling sparse gradients and varying parameter scales, but risk instability in regions of low curvature due to excessive step sizes. We introduce Eva (Escaping Via Adaptive gradients), a family of novel optimizers comprising two variants—Eva+ and Eva-—that dynamically switch between Adam and scaled gradient steps based on local gradient and curvature properties. Our method implements component-wise detection to identify updates where Adam step magnitude exceeds the historical gradient norm. Eva+ employs gradient ascent to escape from degenerate regions, while Eva- provides conservative descent with magnitude damping. We set a threshold for detecting unstable updates, supported by theoretical analysis and empirical validation. Extensive experiments prove that both Eva variants achieve faster convergence and improved generalization compared to popular optimizers.
Gradient-based methods
Low-curvature optimization
Escape mechanisms
Deep learning
Adaptive optimization
Författare
Antonio Di Cecco
Università Campus Bio-Medico di Roma
Universita degli Studi Gabriele d'annunzio di Chieti-Pescara
Carlo Metta
Consiglo Nazionale Delle Richerche
Andrea Papini
Chalmers, Matematiska vetenskaper, Tillämpad matematik och statistik
Göteborgs universitet
Marco Fantozzi
Universita degli Studi di Parma
Silvia Giulia Galfré
Universita di Pisa
Michelangelo Vegliò
Università Campus Bio-Medico di Roma
Universita degli Studi Gabriele d'annunzio di Chieti-Pescara
Luigi Amedeo Bianchi
Universita degli Studi di Trento
Maurizio Parton
Universita degli Studi Gabriele d'annunzio di Chieti-Pescara
Francesco Morandin
Universita di Pisa
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
03029743 (ISSN) 16113349 (eISSN)
Vol. 16816 LNCS 112-1279783032316653 (ISBN)
28th International Conference on Pattern Recognition, ICPR 2026
Lyon, France,
Lyon, France,
Ämneskategorier (SSIF 2025)
Beräkningsmatematik
Matematisk analys
DOI
10.1007/978-3-032-31666-0_8