January 29
@
12:00 PM
–
1:00 PM
In contemporary machine learning, realistic models exhibit increasing nonconvexity and overwhelming overparameterization. This nonconvex nature often leads to numerous undesirable or “spurious” local solutions, while overparameterization exacerbates the risk of overfitting. Yet, simple “short-sighted” algorithms, such as gradient descent (GD) or its variants, often find the needle in the haystack: they converge to the correct, low-dimensional solutions even when such structures are neither explicitly encoded in the model nor required by the algorithm. This talk delves into explaining this desirable performance of GD-based algorithms by studying their fine-grained trajectory on over-parameterized models, spanning from low-rank models to deep neural networks.
Bio: Salar Fattahi is an Assistant Professor of Industrial and Operations Engineering at the University of Michigan. He received his Ph.D. from the University of California, Berkeley in 2020. He is the recipient of a National Science Foundation CAREER Award and the Deans’ MLK Spirit Award. His research focuses on optimization and machine learning and has been recognized with multiple nominations and awards, including the INFORMS Junior Faculty Interest Group Best Paper Award, the INFORMS Data Mining Best Paper Award, and the INFORMS Computing Society Best Student Paper Award. He currently serves as Vice Chair for Machine Learning in the INFORMS Optimization Society, as an Associate Editor for the INFORMS Journal on Data Science, and as an Area Chair for several premier conferences, including NeurIPS, ICML, and ICLR.