Alexei Baevski

Language modeling

Baevski's early papers with Michael Auli were on neural language modeling. "Adaptive Input Representations for Neural Language Modeling" (2018) extended the adaptive softmax to input representations of variable capacity, and Baevski was a co-author of the fairseq toolkit paper in 2019, which describes an open-source PyTorch library for translation, summarization and language modeling.⁹⁵

Speech representation learning

From 2019 Baevski's work turned to self-supervised learning from raw audio. Baevski co-authored the original wav2vec paper and was first author of vq-wav2vec, which learns discrete speech units, and of wav2vec 2.0, which masks speech in a latent space and solves a contrastive task over quantized representations. Follow-up work included XLSR, a cross-lingual version of wav2vec 2.0 pre-trained on many languages, and wav2vec-U (2021), a method for training speech recognition without labeled data.^101111213

Self-supervised learning across modalities

The data2vec paper (2022) applied a single self-supervised objective, predicting latent representations of the full input from a masked view, to speech, images and text. Its successor, data2vec 2.0, made that training more efficient by not encoding masked tokens and by using a fast convolutional decoder. Baevski was also a co-author of the Massively Multilingual Speech paper on extending speech technology to more than 1,000 languages.³¹⁴⁷

Work and contributions

• wav2vec 2.0 Published research

  First author of wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (2020), with Henry Zhou, Abdelrahman Mohamed, and Michael Auli.¹²

• data2vec Published research

  First author of data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language (2022); the paper says models and code are available in the fairseq repository.³⁴

• fairseq Open-source code

  Co-author of fairseq: A Fast, Extensible Toolkit for Sequence Modeling (2019). The fairseq repository is published under the MIT license and lists wav2vec 2.0 (Baevski et al., 2020) among its implemented papers.⁵⁶

Timeline

1. Sep 2018Posted "Adaptive Input Representations for Neural Language Modeling" with Michael Auli as first author.⁹
2. Apr 2019Co-authored the fairseq toolkit paper and the original wav2vec speech pre-training paper.⁵¹⁰
3. Oct 2019Posted vq-wav2vec, on self-supervised learning of discrete speech representations, as first author.¹¹
4. Jun 2020Posted wav2vec 2.0 as first author and co-authored the XLSR cross-lingual speech paper.¹¹²
5. May 2021Posted "Unsupervised Speech Recognition", describing wav2vec-U, as first author.¹³
6. Feb 2022Posted data2vec, a shared self-supervised framework for speech, vision and language, as first author.³
7. Dec 2022Posted data2vec 2.0, a more efficient version of the data2vec objective, as first author.¹⁴
8. May 2023Co-authored "Scaling Speech Technology to 1,000+ Languages", the Massively Multilingual Speech paper.⁷

Other documented roles

Roles the sources document that are not confirmed as current.

• Listed with Facebook AI on the wav2vec 2.0 paper (2020), Facebook AI (Meta)²

Where they work

• Google 121 people in the wiki

Related people

• Pete WardenAlso at Google
• Fei-Fei LiAlso at Google
• Laurence MoroneyAlso at Google
• Oriol VinyalsAlso at Google
• Kai-Fu LeeAlso at Google
• Alec RadfordMultimodal models · Speech and audio
• Michael AuliSpeech and audio · ML frameworks and libraries
• Alexandre DéfossezSpeech and audio · Multimodal models
• Dong YuSpeech and audio · ML frameworks and libraries
• Gabriel SynnaeveSpeech and audio · ML frameworks and libraries