Aidan Gomez

Career

Gomez's work on "Attention Is All You Need" was performed while at Google Brain, and the 2018 Tensor2Tensor paper also lists Gomez with a Google Brain affiliation. Gomez is now co-founder and CEO of Cohere, which lists Nick Frosst and Ivan Zhang as its other co-founders, and Gomez's GitHub profile names Cohere as Gomez's organization.²³¹⁴

Research

Gomez is one of eight authors of "Attention Is All You Need" (2017), which introduced the Transformer, a sequence model built entirely on attention; its contribution note credits Gomez and Łukasz Kaiser with designing and implementing parts of Tensor2Tensor, which replaced the team's earlier codebase. The same year Gomez was second author of "One Model To Learn Them All", which trained a single model concurrently on image, translation, captioning, speech, and parsing tasks.²⁵

Gomez is first author of the 2017 RevNet paper, which introduced reversible ResNet-style networks that can be trained without storing most activations in memory, and of "Learning Sparse Networks Using Targeted Dropout" (2019), co-authored with Ivan Zhang and Geoffrey Hinton among others, which trains networks to be robust to later pruning.⁶⁷

Open-source work

Gomez is a co-author of the 2018 paper describing Tensor2Tensor, Google Brain's Apache-2.0 library of deep learning models and datasets that includes the reference implementation of the Transformer.³⁸

Work and contributions

• Transformer (Attention Is All You Need) Published research

  Co-author of the 2017 paper that introduced the Transformer architecture; the contribution note says Gomez and Łukasz Kaiser designed and implemented parts of Tensor2Tensor, which replaced the earlier codebase used for the work.²

• Tensor2Tensor Open-source code

  Co-author of the 2018 Tensor2Tensor paper, which describes the Google Brain library of deep learning models that includes the reference implementation of the Transformer; the repository is published under the Apache 2.0 license.³⁸

Timeline

1. Jun 12, 2017Co-authored "Attention Is All You Need", the paper that introduced the Transformer.²
2. Jun 16, 2017Co-authored "One Model To Learn Them All" as second author.⁵
3. Jul 14, 2017Released the RevNet paper on reversible networks that avoid storing activations, as first author.⁶
4. Mar 16, 2018Co-authored the Tensor2Tensor for Neural Machine Translation paper.³
5. May 31, 2019Released "Learning Sparse Networks Using Targeted Dropout" as first author.⁷

Other documented roles

Roles the sources document that are not confirmed as current.

• Work on the Transformer paper performed while at Google Brain, Google Brain²³

Where they work

• Cohere 5 people in the wiki

Related people

• Jay AlammarAlso at Cohere
• Nils ReimersAlso at Cohere
• Nick FrosstAlso at Cohere
• Joelle PineauAlso at Cohere
• John X. MorrisLanguage models
• Sam McCandlishLanguage models
• Shengjia ZhaoLanguage models
• Luis LastrasLanguage models
• Jacob AndreasLanguage models
• Omar SansevieroLanguage models