Stas Bekman

Career

Stas Bekman has written software since 1994, has taught at technical conferences and user groups, and has published several books, including Practical mod_perl. Around 2020 Bekman helped develop fastai, then worked at Hugging Face from 2020 to 2023, contributing to the Transformers library and to projects such as DeepSpeed and PyTorch.¹

After Hugging Face, Bekman worked at Contextual AI on a distributed training framework and on training and serving retrieval-augmented generation models. Bekman now works at Snowflake AI Research, developing efficient post-training software with a team that includes DeepSpeed creators.¹²

Large-scale training

Bekman's site names the BLOOM training run, on 384 A100 GPUs, as a major accomplishment and describes Bekman as its lead engineer; Bekman is among the co-authors of the 2022 BLOOM paper and of a companion study on choosing what language model to train with a fixed compute budget. In 2023 Bekman's team trained the IDEFICS-80b multimodal model, described in the OBELICS paper.¹⁶⁷⁵

At Snowflake, Bekman is first author of the 2025 Arctic Long Sequence Training paper, which describes memory optimizations for training on multi-million-token sequences and is open-sourced through DeepSpeed. Bekman also co-authored the 2024 Universal Checkpointing paper on distributed checkpoints that can be reloaded under different parallelism settings.³⁸

Open-source work and writing

Bekman maintains the Machine Learning Engineering Open Book, a CC BY-SA 4.0 collection of methods and tools for training, fine-tuning, and serving large models, and an early work in progress called The Art of Debugging. Bekman is also a co-author of the 2021 paper on the Hugging Face Datasets library.⁴¹⁹

Work and contributions

• Arctic Long Sequence Training (ALST) Open-source code

  First author of the 2025 paper on ALST, a set of memory optimizations for training on very long sequences with Hugging Face models; the paper says ALST is open-sourced via DeepSpeed and Snowflake's Arctic Training.³

• Machine Learning Engineering Open Book Published research

  Author and maintainer of an open collection of methods, tools and step-by-step instructions for training, fine-tuning and serving large language and multimodal models, distributed under CC BY-SA 4.0.⁴¹

• Universal Checkpointing Published research

  Co-author of the 2024 paper on Universal Checkpointing, a distributed checkpointing system for large-scale training with reconfigurable parallelism.⁸

• OBELICS and IDEFICS Open data

  Equal-contribution author, with a Hugging Face affiliation, of the 2023 OBELICS paper, which releases an open interleaved image-text dataset and trains the IDEFICS models.⁵

• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model Open weights

  Co-author of the 2022 BLOOM paper; Bekman's site describes serving as lead engineer for the model's training run.⁶¹

Timeline

1. 1994Began writing software, work that later spanned several technical domains.¹
2. 2020Began work at Hugging Face, contributing to the Transformers library.¹
3. Sep 7, 2021Co-authored the paper Datasets, a community library for natural language processing.⁹
4. Oct 27, 2022Co-authored a study on what language model to train with one million GPU hours.⁷
5. Nov 9, 2022Co-authored the BLOOM paper describing the 176B-parameter open-access multilingual model.⁶
6. Jun 21, 2023Co-authored the OBELICS paper, which releases an interleaved image-text dataset and IDEFICS models.⁵
7. Jun 27, 2024Co-authored the Universal Checkpointing paper on flexible distributed checkpointing.⁸
8. Jun 16, 2025Published Arctic Long Sequence Training as first author, open-sourced via DeepSpeed.³

Other documented roles

Roles the sources document that are not confirmed as current.

• Worked on a distributed training framework and on training and inference of RAG LLMs, Contextual AI¹
• Worked at Hugging Face (2020–2023); lead engineer on the BLOOM training, Hugging Face¹⁵

Honors and recognition

• PyTorch Contributor Award, PyTorch Ambassador · PyTorch, 2023¹⁰¹

Where they work

• Snowflake 5 people in the wiki

Related people

• Hao ZhangAlso at Snowflake
• Sridhar RamaswamyAlso at Snowflake
• Benoit DagevilleAlso at Snowflake
• Thierry CruanesAlso at Snowflake
• Torsten ScholakLanguage models · Training systems
• Naveen RaoTraining systems · Language models
• Weizhu ChenLanguage models · Training systems
• Sami JaghouarTraining systems · Language models
• Clément FarabetLanguage models · Training systems
• Lucas AtkinsLanguage models · Training systems