Łukasz Kaiser

Research

Kaiser's early neural network papers took on parsing and algorithm learning. With Oriol Vinyals, Ilya Sutskever, Geoffrey Hinton and others, Kaiser co-wrote "Grammar as a Foreign Language" in 2014, which treated syntactic parsing as translation, and with Sutskever wrote "Neural GPUs Learn Algorithms" in 2015.⁷⁸

Kaiser was a co-author of "Attention Is All You Need" in 2017, the paper that introduced the Transformer, and first-authored "One Model To Learn Them All," a single model trained across several tasks and modalities. Later Transformer work includes Universal Transformers in 2018 and Reformer, an efficient Transformer variant, in 2020.³⁹¹⁰¹¹

Open-source work

Kaiser co-authored the 2016 TensorFlow white paper describing Google's system for large-scale machine learning on heterogeneous hardware. The Transformer paper's contribution note credits Kaiser and Aidan Gomez with designing parts of and implementing Tensor2Tensor, and Kaiser is a corresponding author of the 2018 Tensor2Tensor paper.⁵³⁴¹²

Career

Kaiser's papers from 2017 and 2018 list a Google Brain affiliation, and Kaiser co-authored the 2016 paper on Google's Neural Machine Translation system. Kaiser's GitHub profile now lists OpenAI, and the 2024 OpenAI o1 System Card names Kaiser in its reasoning research leadership.³⁴⁶¹²

Work and contributions

• TensorFlow Open-source code

  Co-author of the TensorFlow white paper, which describes the system built at Google and its open-source release under the Apache 2.0 license.⁵

• Transformer (Attention Is All You Need) Published research

  Co-author of the 2017 paper that introduced the Transformer architecture; the contribution note says Kaiser and Aidan Gomez designed parts of and implemented Tensor2Tensor, replacing the earlier codebase.³

• Tensor2Tensor Open-source code

  One of the two corresponding authors of the 2018 Tensor2Tensor paper; the Apache-2.0 library, developed by the Google Brain team, includes the reference implementation of the Transformer.⁴¹²

• OpenAI o1 Proprietary product

  Listed in the Reasoning Research leadership group in the OpenAI o1 System Card (2024).²

Timeline

1. Dec 23, 2014Co-authored "Grammar as a Foreign Language," applying sequence-to-sequence models to parsing.⁷
2. Nov 25, 2015Released "Neural GPUs Learn Algorithms" with Ilya Sutskever.⁸
3. Mar 14, 2016Co-authored the TensorFlow white paper on large-scale machine learning.⁵
4. Sep 26, 2016Co-authored the paper on Google's Neural Machine Translation system.⁶
5. Jun 12, 2017Co-authored "Attention Is All You Need," which introduced the Transformer.³
6. Jun 16, 2017Released "One Model To Learn Them All" as first author.⁹
7. Mar 16, 2018Released the Tensor2Tensor paper as a corresponding author.⁴
8. Jul 10, 2018Co-authored Universal Transformers.¹⁰
9. Jan 13, 2020Co-authored Reformer, an efficient Transformer architecture.¹¹
10. Dec 21, 2024Named in the reasoning research leadership of the OpenAI o1 System Card.²

Other documented roles

Roles the sources document that are not confirmed as current.

• Listed with a Google Brain affiliation on 2017–2018 papers, Google Brain³⁴

Where they work

• OpenAI 63 people in the wiki

Related people

• Shantanu JainAlso at OpenAI
• Mira MuratiAlso at OpenAI
• Andrej KarpathyAlso at OpenAI
• Collin BurnsAlso at OpenAI
• John SchulmanAlso at OpenAI
• Lysandre DebutML frameworks and libraries · Language models
• Fabian PedregosaML frameworks and libraries · Language models
• Ronan CollobertML frameworks and libraries · Language models
• Yuxin WuML frameworks and libraries · Language models
• Luis LastrasLanguage models · ML frameworks and libraries