Roy Frostig

Research

Frostig completed a PhD at Stanford, advised by Percy Liang, as part of the statistical machine learning group. Frostig's early papers concerned optimization and learning theory, including stochastic algorithms for empirical risk minimization (2015), principal component projection without principal component analysis (2016), and the expressivity of neural networks at initialization.¹⁷

Frostig later co-authored a study measuring the effects of data parallelism on neural network training (2018) and work on test-set reuse. Later papers turned to the theory of automatic differentiation, such as "Decomposing reverse-mode automatic differentiation" (2021) and "You Only Linearize Once" (2022), which shows how tangents transpose to gradients, as well as modular implicit differentiation. Frostig is also listed with the Gemini Team on the Gemini 1.5 and Gemini 2.5 reports.^1891011

Open-source work

Frostig created JAX with a few colleagues in 2017 and continues to work on it as a core developer. With Matthew Johnson and Chris Leary, Frostig described an early version of the system in "Compiling machine learning programs via high-level tracing" at SysML 2018. JAX's documentation credits Frostig as an author of design proposals for typed PRNG keys and for the jax.extend module, and Frostig's homepage notes small contributions to the Rust programming language in its early stages.¹²⁴⁵³

Work and contributions

• JAX Open-source code

  Co-author and core developer of JAX according to Frostig's homepage; named in the "Citing JAX" author list; co-author of the SysML 2018 paper on an early JAX; author or co-author of JAX design proposals on typed keys and pluggable RNGs (August 2023) and on jax.extend. The repository is released under the Apache License 2.0.^1324512

Timeline

1. Jun 2015Posted "Un-regularizing", on faster stochastic algorithms for empirical risk minimization, as first author.⁷
2. 2017Created JAX with a few colleagues.¹
3. 2018Presented "Compiling machine learning programs via high-level tracing" at SysML 2018, describing early JAX.²¹
4. Nov 2018Co-authored "Measuring the Effects of Data Parallelism on Neural Network Training".⁸
5. May 2021Posted "Decomposing reverse-mode automatic differentiation" as first author.⁹
6. May 2021Co-authored "Efficient and Modular Implicit Differentiation".¹¹
7. Apr 2022Co-authored "You Only Linearize Once, Tangents Transpose to Gradients", later presented at POPL 2023.¹⁰¹

Where they work

• Google DeepMind 64 people in the wiki

Related people

• Razvan PascanuAlso at Google DeepMind
• Mathieu BlondelAlso at Google DeepMind
• Clément FarabetAlso at Google DeepMind
• Shane LeggAlso at Google DeepMind
• Alex GravesAlso at Google DeepMind
• Martín AbadiML frameworks and libraries · Machine-learning theory
• Lorenzo StellaML frameworks and libraries · Machine-learning theory
• Fabian PedregosaML frameworks and libraries · Machine-learning theory
• Alex SmolaMachine-learning theory · ML frameworks and libraries
• Luis LastrasML frameworks and libraries