Victor Sanh

Research at Hugging Face

Victor Sanh's work at Hugging Face includes efficient language models and open-source tooling. Sanh was first author of DistilBERT (2019), a distilled version of BERT that is smaller and faster, and a co-author of the 2019 paper describing Hugging Face's open-source Transformers library. With Thomas Wolf and Alexander M. Rush, Sanh also wrote "Movement Pruning" (2020), on adaptive sparsity during fine-tuning.²⁷³⁸

Sanh was first author of "Multitask Prompted Training Enables Zero-Shot Task Generalization" (2021), the BigScience paper that introduced the T0 family of models trained on many tasks phrased as natural-language prompts. The T0pp model card on Hugging Face documents one of the released checkpoints.⁴⁹

Vision-language models

Sanh later worked on open multimodal models released under Hugging Face's HuggingFaceM4 organization. Sanh was a senior contributor to OBELICS (2023), an open web-scale dataset of interleaved image-text documents, and co-authored "What matters when building vision-language models?" (2024), the paper that released Idefics2, an 8-billion-parameter vision-language model.⁵¹⁰⁶¹¹

In August 2024 Sanh co-authored a further paper with Hugo Laurençon, Andrés Marafioti and Léo Tronchon on insights and future directions in building vision-language models. Sanh's GitHub profile now describes Sanh as an AI builder previously at Hugging Face.¹²¹

Work and contributions

• Transformers Open-source code

  Third-listed author of "Transformers: State-of-the-Art Natural Language Processing" (2019), the paper describing the open-source library, written with a Hugging Face affiliation.³

• DistilBERT Open weights

  First author of the DistilBERT paper (2019), which pre-trains a smaller general-purpose model through knowledge distillation and tests it on device; the distilbert-base-uncased model card lists the Apache 2.0 license.²⁷

• T0 Open weights

  First author of "Multitask Prompted Training Enables Zero-Shot Task Generalization" (2021), which fine-tunes an encoder-decoder model on a prompted multitask mixture and releases the trained models; the T0pp model card lists the Apache 2.0 license.⁴⁹

• OBELICS, IDEFICS and Idefics2 Open weights

  Marked as an equal and senior contributor on the OBELICS paper (2023), which releases an open interleaved image-text dataset and the IDEFICS models, and last-listed author of the 2024 study that develops and releases the Idefics2 8B vision-language model, whose model card lists the Apache 2.0 license.⁵¹⁰⁶¹¹

Timeline

1. Oct 2, 2019First author of DistilBERT, a smaller, faster distilled version of BERT.²
2. Oct 9, 2019Co-authored the paper describing Hugging Face's Transformers library.³
3. May 15, 2020First author of "Movement Pruning: Adaptive Sparsity by Fine-Tuning".⁸
4. Oct 15, 2021First author of the multitask prompted training paper that introduced the T0 models.⁴
5. Jun 21, 2023Senior contributor to OBELICS, an open dataset of interleaved image-text web documents.⁵
6. May 3, 2024Co-authored the paper releasing Idefics2, "What matters when building vision-language models?".⁶
7. Aug 22, 2024Co-authored a paper on insights and future directions for building vision-language models.¹²

Other documented roles

Roles the sources document that are not confirmed as current.

• Listed with a Hugging Face affiliation on the DistilBERT (2019) and Idefics2 (2024) papers, Hugging Face¹²³⁶

Where they work

• Rilix 1 person in the wiki

Related people

• Ali FarhadiMultimodal models · Language models
• Sathwik Tejaswi MadhusudhanLanguage models · Multimodal models
• Alec RadfordLanguage models · Multimodal models
• Douwe KielaLanguage models · Multimodal models
• Armen AghajanyanMultimodal models · Language models
• Junyang LinLanguage models · Multimodal models
• Sasha ShengLanguage models · Multimodal models
• Ilya SutskeverLanguage models · Multimodal models
• Sahil DuaLanguage models · Multimodal models
• Xiaodong HeMultimodal models · Language models