1. Time: December 22nd (Monday) 10:00
Place: Natural Science Building #744
Speaker: Dr. Jaeyoung Yoon (Technical University of Munich)
Title: Stochastic Modified Equations for SGD in Infinite-Dimensional Hilbert Spaces
Abstract:
Stochastic Gradient Descent (SGD) is a simple and computationally efficient algorithm, but its discrete and non-Gaussian stochastic nature makes a theoretical analysis of its dynamics challenging. To address this difficulty, we formulate a stochastic modified equation (SME) in an infinite-dimensional Hilbert space that approximates SGD in continuous time. We show that the SME accurately captures the expected behavior of SGD for a wide class of functionals, and that non-Gaussian mini-batch noise can be effectively approximated by Gaussian noise in this continuous-time model. Through numerical experiments, we illustrate how the difference between SGD and its continuous-time SME approximation depends on the step size, of which results correspond to the weak convergence analysis. Our work extends Weinan E’s stochastic modified equation theory from finite-dimensional setting ([1]) to infinite-dimensional Hilbert spaces.
[1] Li, Qianxiao, Cheng Tai, and Weinan E. “Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations.” Journal of Machine Learning Research 20 (40): 1–47, 2019.
2. Time: December 22nd (Monday) 11:00
Place: Natural Science Building #744
Speaker: Prof. Junhyeok Byeon (Dalian University of Technology)
Title: A stochastic consensus model for global optimization
Abstract:
We propose a first-order, time-discrete stochastic consensus model for global optimization. The model draws on interaction-based mechanisms to incorporate objective-function information and handles non-convex, non-differentiable, and even discontinuous functions. It is motivated by the Consensus-Based Optimization (CBO) paradigm, which promotes consensus among agents toward a global optimum through simple stochastic dynamics amenable to rigorous mathematical analysis. Despite these promises, the actual behavior of agents in its time-discrete implementation remains largely unknown. We address this issue by the novel observation that the consensus point governs the entire ensemble. We further demonstrate competitive performance across various problems.











