Asymptotic Analysis of Machine Learning Models: Comparison Theorems and Universality

Doctoral thesis, 2025

This thesis investigates the asymptotic regime of machine learning models - a regime in which both the number of trainable parameters (model size) and the number of data points grow infinitely at a fixed ratio. Understanding model behavior in this limit provides valuable theoretical insights into model statistics such as training error and generalization error, particularly in high-dimensional settings relevant to contemporary machine learning practice.

The core methodological tools used throughout this work are Gaussian comparison theorems, with a special emphasis on the Convex Gaussian Min-max Theorem (CGMT). These theorems enable the rigorous analysis of complex learning algorithms by comparing them to alternative surrogate problems, which are simpler to analyze. By constructing such asymptotically equivalent optimization problems, we are able to derive characterizations of the models of interest by proxy.

A secondary but significant theme in this thesis is the concept of universality in the asymptotic regime. Universality results demonstrate that many statistical properties of machine learning models are asymptotically governed only by low-order moments (e.g., means and variances) of the data distribution, rather than its full structure. This insight justifies the use of Gaussian surrogate models that match these moments, making them amenable to analysis via Gaussian comparison tools.