John Duchi

Career

John Duchi worked as a research assistant with Daphne Koller at Stanford and as a software engineer at Google from 2007 to 2008, then completed a PhD in computer science at UC Berkeley advised by Michael I. Jordan and Martin J. Wainwright. During the doctorate Duchi spent summers from 2009 to 2014 as a research intern and engineer working with Yoram Singer at Google.²

Duchi joined the Stanford faculty in September 2014 as an assistant professor of statistics and electrical engineering and was promoted to associate professor in September 2021.²¹

Research

Duchi's research spans statistical learning, optimization, information theory and computation. The 2011 Adagrad paper with Elad Hazan and Yoram Singer introduced subgradient methods that adapt their step sizes to the geometry of previously observed data, and the method is implemented as an optimizer in PyTorch.¹³⁴

With Jordan and Wainwright, Duchi developed minimax theory for locally private estimation, published with discussion in the Journal of the American Statistical Association in 2018. Duchi's work with Hongseok Namkoong on variance-based regularization with convex objectives received a best paper award at NIPS 2017.²

Teaching and writing

At Stanford, Duchi has taught information theory and statistics, the mathematics of convexity, applied statistics, probability and introductory matrix methods. The CV also lists a book in preparation, Statistics and Information Theory, hosted on the Statistics 311 course site.²

Work and contributions

• Adaptive Subgradient Methods for Online Learning and Stochastic Optimization (Adagrad) Published research

  First author, with Elad Hazan and Yoram Singer, of the 2011 Journal of Machine Learning Research paper presenting subgradient methods that adapt to the geometry of the data seen in earlier iterations; PyTorch's documentation for its Adagrad optimizer refers to this paper for details of the algorithm.³⁴

Timeline

1. 2007Joined Google as a software engineer.²
2. 2008Began doctoral research at UC Berkeley with Michael I. Jordan and Martin J. Wainwright.²
3. 2011Published the Adagrad paper on adaptive subgradient methods in the Journal of Machine Learning Research.³
4. Sep 1, 2014Became an assistant professor of statistics and electrical engineering at Stanford University.²
5. 2018Published "Minimax optimal procedures for locally private estimation" in the Journal of the American Statistical Association.²
6. Sep 1, 2021Became an associate professor of statistics and electrical engineering at Stanford University.²

Other documented roles

Roles the sources document that are not confirmed as current.

• Software Engineer (2007–2008); Research Intern/Engineer, summers 2009–2014, Google²

Honors and recognition

• Best Student Paper Award, ICML 2010 · International Conference on Machine Learning, 2010²
• ACM Doctoral Dissertation Award, Honorable Mention · Association for Computing Machinery, 2014²
• NSF CAREER Award · U.S. National Science Foundation, 2016²
• Sloan Research Fellowship in Mathematics · Alfred P. Sloan Foundation, 2017²
• Best Paper Award, NIPS 2017 · Conference on Neural Information Processing Systems, 2017²
• ONR Young Investigator · Office of Naval Research, 2019²
• SIAM Early Career Prize in Optimization · Society for Industrial and Applied Mathematics, 2020²
• Best Student Paper Award, COLT 2023 · Conference on Learning Theory, 2023²

Where they work

• Stanford University 40 people in the wiki

Related people

• Daphne KollerAlso at Stanford University
• Kunle OlukotunAlso at Stanford University
• Trevor HastieAlso at Stanford University
• Jascha Sohl-DicksteinAlso at Stanford University
• Christopher RéAlso at Stanford University
• Michael I. JordanMachine-learning theory · Training systems
• Martín AbadiTraining systems · Machine-learning theory
• Cho-Jui HsiehTraining systems · Machine-learning theory
• Beren MillidgeTraining systems · Machine-learning theory
• Mathieu BlondelMachine-learning theory