To understand the behavior of wide neural networks, prior studies have considered the mean-field (MF) limit of two-layer neural networks (NNs) as the width tends to infinity, establishing theoretical guarantees for its convergence under gradient flow training as well as approximation and generalization capabilities.

In this talk, based on joint work with Joan Bruna and Eric Vanden-Eijnden (https://arxiv.org/abs/2210.16286, JMLR 2026), we will first discuss the infinite-width limit of a type of three-layer NN where the first-layer weights are untrained. To rigorously define the limiting model, we extend the MF theory by lifting the representation of neurons from Euclidean to functional spaces. This allows us to establish the MF training dynamics as a functional gradient flow with a time-varying kernel that remains positive-definite under suitable assumptions, thus proving a linear-rate convergence of its training loss. In addition, we define novel function spaces that contain the solutions obtained through the MF training dynamics and prove Rademacher complexity bounds for these spaces.

If time permits, we will next discuss how to extend this framework to characterize the function space of deeper NNs with all layers trainable, culminating in the Neural Hilbert Ladders theory (https://arxiv.org/abs/2307.01177, JMLR 2024).

Bio: Zhengdao Chen is currently a Research Scientist at Google and previously earned his PhD in Mathematics from the Courant Institute of Mathematical Sciences at New York University. His academic research has largely concerned the mathematics of learning and neural networks.