Noam Shazeer

Research

While at Google Brain, Noam Shazeer first-authored the 2017 Outrageously Large Neural Networks paper, which introduced the sparsely-gated mixture-of-experts layer, and the same year Shazeer co-authored Attention Is All You Need, whose contribution note credits Shazeer with scaled dot-product attention, multi-head attention and the parameter-free position representation.⁴³

Later single- and first-author papers include Adafactor (2018), an optimizer with adaptive learning rates and sublinear memory cost; multi-query attention (2019), which shares keys and values across attention heads to speed up incremental decoding; and GLU Variants Improve Transformer (2020), which tests gated linear unit variants in Transformer feed-forward layers. Shazeer was also a co-author of Switch Transformers (2021), a simplified sparse mixture-of-experts design.⁶⁷⁸⁹

Open-source work and systems

Shazeer co-authored the 2018 paper describing Tensor2Tensor, a library of deep learning models and datasets, and first-authored Mesh-TensorFlow (2018), a language for specifying how tensor computations are split across processors so that very large models can be trained.⁵¹⁰¹¹

At Google DeepMind, Shazeer is listed among the sponsors of the 2025 Gemma 3 technical report on lightweight open models and is credited as Vice President, Google DeepMind, on the May 2026 Gemini 3.5 announcement.²¹

Work and contributions

• Gemma 3 Open weights

  Listed among the sponsors of the Gemma 3 technical report, which describes Gemma 3 as a multimodal addition to the Gemma family of lightweight open models, ranging from 1 to 27 billion parameters.²

• Transformer (Attention Is All You Need) Published research

  Co-author of the 2017 paper that introduced the Transformer architecture; the paper's contribution note credits Shazeer with proposing scaled dot-product attention, multi-head attention, and the parameter-free position representation.³

• Sparsely-gated mixture-of-experts layer Published research

  First author of Outrageously Large Neural Networks (2017), which uses a sparsely-gated mixture-of-experts layer to increase model capacity through conditional computation.⁴

• Tensor2Tensor Open-source code

  Co-author of the 2018 Tensor2Tensor paper, which describes the Google Brain library of deep learning models that includes the reference implementation of the Transformer.⁵¹⁰

Timeline

1. Jan 2017Posted the sparsely-gated mixture-of-experts layer paper as first author.⁴
2. Jun 2017Co-authored Attention Is All You Need, which introduced the Transformer.³
3. Mar 2018Co-authored the Tensor2Tensor library paper.⁵
4. Apr 2018Posted Adafactor, an optimizer with sublinear memory cost, as first author.⁶
5. Nov 2018Posted Mesh-TensorFlow as first author.¹¹
6. Nov 2019Introduced multi-query attention in a single-author paper on fast Transformer decoding.⁷
7. Feb 2020Posted "GLU Variants Improve Transformer" as sole author.⁸
8. Jan 2021Co-authored Switch Transformers on sparse mixture-of-experts scaling.⁹
9. Mar 2025Listed among the sponsors of the Gemma 3 technical report.²
10. May 19, 2026Credited as Vice President, Google DeepMind, on the Gemini 3.5 announcement.¹

Other documented roles

Roles the sources document that are not confirmed as current.

• Listed with a Google Brain affiliation on 2017–2018 papers, Google Brain³⁴⁵

Where they work

• Google DeepMind 64 people in the wiki

Related people

• Danqi ChenAlso at Google DeepMind
• Clément FarabetAlso at Google DeepMind
• Arthur MenschAlso at Google DeepMind
• Stephen RollerAlso at Google DeepMind
• Arthur ConmyAlso at Google DeepMind
• Yonghui WuLanguage models · Training systems
• Myle OttLanguage models · Training systems
• David GrangierLanguage models · Training systems
• Edward BeechingLanguage models · Training systems
• Akshita BhagiaLanguage models · Training systems