Edward Beeching

Research

Beeching completed a PhD at INSA Lyon with the INRIA CHROMA team, studying deep reinforcement learning for planning, navigation and embodied agents. Papers from that period, written with Christian Wolf, Jilles Dibangoye and Olivier Simonin, include EgoMap, a structured egocentric memory for reinforcement learning agents, and work on learning to plan with uncertain topological maps, presented as a spotlight at ECCV 2020.¹

At Hugging Face, Beeching's research turned to post-training and reasoning for language models. Beeching was co-first author, with Lewis Tunstall, of Zephyr, which distills supervised and preference signals to align a smaller open chat model, and later co-wrote papers on meta reinforcement fine-tuning for test-time compute and on QED-Nano, a small open model for Olympiad-level proofs.¹⁴

Open-source work

Beeching is listed among the authors of TRL, Hugging Face's library for post-training with supervised fine-tuning, preference optimization and reinforcement learning, and has co-written Hugging Face blog posts on its use, including a 2023 guide to RLHF fine-tuning of 20B models on a consumer GPU and the StackLLaMA tutorial. Other open projects on Beeching's site include Open R1, an open reproduction of reasoning-model training recipes, NuminaMath, a set of open models and competition-mathematics data, and JAT, a multi-purpose Transformer agent.³¹

Beeching also co-authored Godot Reinforcement Learning Agents, an open interface between the Godot game engine and common reinforcement-learning frameworks, and is listed among the authors of the 2025 SmolLM3 release.¹⁵

Work and contributions

• TRL (Transformers Reinforcement Learning) Open-source code

  Listed among the authors in TRL's CITATION.cff, which describes the Apache-2.0 library as an open-source toolkit for post-training with SFT, reward modeling, DPO, and GRPO. Beeching's site also lists co-authored Hugging Face blog posts on TRL, such as "Fine-Tuning 20B LLMs with RLHF on a 24GB Consumer GPU" (2023) and a 2026 post on delta weight sync in TRL.³¹

• SmolLM3 Open weights

  Listed among the authors of the Hugging Face blog post and citation for SmolLM3, which the post describes as a fully open 3B multilingual, long-context reasoning model released with its checkpoint and full training recipe.⁵¹

• Zephyr: Direct Distillation of LM Alignment Published research

  Co-first author with Lewis Tunstall of the 2023 paper, which applies distilled direct preference optimization with AI feedback to align a smaller open language model to user intent.⁴¹

Timeline

1. 2020Presented work on learning to plan with uncertain topological maps as a spotlight at ECCV 2020.¹
2. 2021Presented Godot Reinforcement Learning Agents at the AAAI Workshop on Reinforcement Learning in Games.¹
3. Mar 28, 2022Co-wrote the Hugging Face blog post introducing Decision Transformers on the Hub.¹
4. Mar 9, 2023Co-wrote a Hugging Face guide to fine-tuning 20B language models with RLHF on a 24GB consumer GPU.¹
5. Oct 25, 2023Released the Zephyr paper on direct distillation of language model alignment as co-first author.⁴
6. Jul 11, 2024Co-wrote the Hugging Face post on how NuminaMath won the first AIMO Progress Prize.¹
7. Jul 8, 2025Co-authored the SmolLM3 release on the Hugging Face blog.⁵
8. Aug 5, 2025Co-wrote the OpenAI Cookbook guide to fine-tuning gpt-oss with Hugging Face Transformers.¹
9. May 27, 2026Co-wrote a Hugging Face post on delta weight sync in TRL.¹

Where they work

• Hugging Face 34 people in the wiki

Related people

• Lewis TunstallAlso at Hugging Face
• Younes BelkadaAlso at Hugging Face
• Stas BekmanAlso at Hugging Face
• Leandro von WerraAlso at Hugging Face
• Colin RaffelAlso at Hugging Face
• Sami JaghouarTraining systems · Reinforcement learning
• Mika SenghaasReinforcement learning · Training systems
• Weizhu ChenLanguage models · Training systems
• Jakub PachockiLanguage models · Reinforcement learning
• Johannes HagemannTraining systems · Reinforcement learning