James Bradbury

Research

Bradbury's early papers are on neural networks for language. Bradbury is first author of "Quasi-Recurrent Neural Networks" (2016) and "Towards Neural Machine Translation with Latent Tree Attention" (2017), and co-authored "Learned in Translation - Contextualized Word Vectors" (2017), the Pointer Sentinel Mixture Models paper and work on non-autoregressive machine translation.⁶⁷⁸

Later papers focus on large-scale training and serving on accelerators. Bradbury co-authored a 2020 study of concurrency limits in ML training on Google TPUs, the PaLM paper (2022) and "Efficiently Scaling Transformer Inference" (2022), and is listed among the authors of the PaLM 2 and Gemini technical reports.⁹¹⁰¹¹⁸

Open-source work

Bradbury is listed among the authors (in alphabetical order) in the citation entry for JAX, the library of composable transformations of Python and NumPy programs, and co-authored the January 2023 JAX design document that proposed shard_map for writing per-device code. Bradbury, listed with Google, is also a co-author of the 2019 PyTorch paper.¹⁴³

Work and contributions

• JAX Open-source code

  Named in the "Citing JAX" author list of the JAX README and credited as a co-author of the shard_map design doc (January 2023) in the JAX documentation. The repository is released under the Apache License 2.0.¹⁴¹²

• PyTorch Published research

  Co-author of "PyTorch: An Imperative Style, High-Performance Deep Learning Library" (2019).²

• Quasi-Recurrent Neural Networks Published research

  First author of "Quasi-Recurrent Neural Networks" (2016), which combines convolutional layers with a minimal recurrent pooling function to model sequences in parallel across timesteps.⁶

Timeline

1. Nov 5, 2016Released "Quasi-Recurrent Neural Networks" as first author, submitted to ICLR 2017.⁶
2. Aug 1, 2017Co-authored "Learned in Translation - Contextualized Word Vectors".⁷
3. 2018JAX, whose citation entry lists Bradbury among its authors, had its open-source release.¹
4. Dec 3, 2019Co-authored the PyTorch paper, listed with Google.²³
5. Nov 7, 2020Co-authored "Exploring the limits of Concurrency in ML Training on Google TPUs".⁹
6. Apr 5, 2022Co-authored "PaLM - Scaling Language Modeling with Pathways".¹⁰
7. Nov 9, 2022Co-authored "Efficiently Scaling Transformer Inference" with Reiner Pope and others.¹¹
8. Jan 2023Co-authored the JAX design document proposing shard_map.⁴

Documented roles

The sources do not confirm a current role; these are the roles they document.

• Listed with Google as an author of the 2019 PyTorch paper, Google³

Companies in their history

• Google 121 people in the wiki

Related people

• Martín AbadiConnected to Google
• Jeff DeanConnected to Google
• Rajat MongaConnected to Google
• Yangqing JiaConnected to Google
• Adam PaszkeConnected to Google
• Pieter NoordhuisTraining systems · ML frameworks and libraries
• Brian Van EssenTraining systems · ML frameworks and libraries
• Jane XuML frameworks and libraries · Training systems
• Luca AntigaML frameworks and libraries · Training systems
• Frédéric BastienML frameworks and libraries · Training systems