Alexandre Défossez

Career

Alexandre Défossez was a CIFRE PhD student at FAIR and the Sierra team at INRIA, supervised by Léon Bottou and Nicolas Usunier of FAIR and Francis Bach of INRIA. Défossez then spent three years as a research scientist at FAIR, leading the music generation effort that produced MusicGen and co-leading development of the AudioCraft framework.¹

Défossez co-founded Kyutai, an AI lab focused on open-source and open-science work, where Défossez continues to lead research projects, and is listed on Kyutai's team page as Head of Research. Défossez's homepage now also presents Défossez as a co-founder and Chief Science Officer of Gradium, a company that builds voice AI models.¹²

Research

Défossez's research covers generative music and speech models suited to low-latency interactive use across several modalities, as well as reinforcement learning for such models. Earlier work includes the Demucs music source separation models, real-time speech enhancement in the waveform domain, and theoretical papers on stochastic optimization, including a convergence proof for Adam and Adagrad.¹

Défossez is first author of the EnCodec neural audio compression paper (2022) and last author of the MusicGen paper on simple and controllable music generation (2023), both from Meta AI. At Kyutai, Défossez is first author of the 2024 Moshi paper, a speech-text foundation model for real-time, full-duplex spoken dialogue, and has co-authored follow-up work on simultaneous speech-to-speech translation and on retrieval for full-duplex speech models.³⁴⁵⁹⁶¹

Open-source work

Défossez's homepage lists the open-source software Défossez has worked on, including Moshi and its Mimi streaming audio codec, AudioCraft, EnCodec, Demucs, the Denoiser real-time speech enhancement tool, the Julius signal-processing library for PyTorch, and MuScriptor for multi-instrument music transcription. The AudioCraft repository releases its code under the MIT license, and the Moshi repository releases code under MIT and Apache licenses with model weights under CC-BY 4.0.¹⁸⁷

Work and contributions

• AudioCraft (EnCodec and MusicGen) Open-source code

  Co-led development of the AudioCraft framework, per Défossez's homepage. Défossez is first author of the EnCodec paper (2022) and a core-team author of the MusicGen paper (2023); the AudioCraft repository releases its code under the MIT license and its model weights under CC-BY-NC 4.0.¹³⁵⁹⁸

• Moshi Open weights

  First author of the Moshi paper (2024), a speech-text foundation model for real-time dialogue. The Moshi repository provides the code under MIT and Apache licenses and states that the model weights are released under CC-BY 4.0.⁶⁷

Timeline

1. 2015Published a paper with Francis Bach on constant step size least-mean-square at AISTATS 2015.¹
2. 2018Presented SING, a symbol-to-instrument neural generator, at NeurIPS (then NIPS) 2018.¹
3. 2019Posted the Demucs preprints on separating music sources in the waveform domain.¹
4. 2020Published real-time speech enhancement in the waveform domain at Interspeech 2020.¹
5. Oct 24, 2022Posted the EnCodec paper, "High Fidelity Neural Audio Compression," as first author.³
6. Jun 8, 2023Posted the MusicGen paper, "Simple and Controllable Music Generation," later presented at NeurIPS 2023.⁵¹
7. Sep 17, 2024Posted the Moshi paper on a speech-text foundation model for real-time dialogue as first author.⁶
8. 2026Co-authored MoshiRAG and a paper on simultaneous speech-to-speech translation without aligned data at ICML 2026.¹

Other documented roles

Roles the sources document that are not confirmed as current.

• Research scientist at FAIR for three years, leading the music generation effort (MusicGen), Meta (FAIR)¹⁴

Where they work

• Gradium 1 person in the wiki
• Kyutai 2 people in the wiki

Related people

• Xu TanSpeech and audio · Image, video, and 3D generation
• Wei-Ning HsuSpeech and audio · Multimodal models
• Jade CopetImage, video, and 3D generation · Speech and audio
• David HarwathSpeech and audio · Multimodal models
• RJ Skerry-RyanSpeech and audio · Image, video, and 3D generation
• Saining XieImage, video, and 3D generation · Multimodal models
• Heiga ZenSpeech and audio · Image, video, and 3D generation
• Sanyuan ChenSpeech and audio · Image, video, and 3D generation
• Jiaming SongImage, video, and 3D generation · Multimodal models
• Alexei BaevskiSpeech and audio · Multimodal models