Beren Millidge

Career

Millidge completed a PhD in machine learning and computational neuroscience at the University of Edinburgh (2017 to 2021) and then worked as a postdoctoral researcher at the University of Oxford with Rafal Bogacz on predictive coding and learning in the brain. Millidge was later a senior research scientist at Verses Research Lab and Head of Research at Conjecture, where the CV describes technical alignment research focused on interpretability of large language models.²¹

In 2023 Millidge was one of four co-founders of Apollo Research before leaving to start Zyphra, where Millidge has served as co-founder and Chief Scientist since July 2023.²

Research at Zyphra

At Zyphra, Millidge is the last-listed author on the Zamba (2024) and Zamba2 (2024) technical reports, which describe hybrid models that pair Mamba state-space blocks with shared attention layers. Millidge is also the senior author of the 2026 Zamba2-VL report, which extends the Zamba2 backbone to vision-language tasks.³⁴⁵

Millidge's CV also lists Zyphra work on BlackMamba, a mixture-of-experts state-space model, and on Tree Attention, a topology-aware decoding method for long-context attention on GPU clusters.²

Work and contributions

• Zamba Open weights

  Senior author of the Zamba technical report, which describes a 7B model combining a Mamba backbone with a single shared attention module and releases its weights and checkpoints.³⁶

• Zamba2-7B Open weights

  Senior author of the Zamba2 suite technical report, which covers the 1.2B, 2.7B and 7.4B hybrid Mamba2-transformer models and their open weights.⁴

• Zamba2-VL-7B Open weights

  Senior author of the Zamba2-VL technical report and release post for Zyphra's vision-language models built on the Zamba2 backbone at 1.2B, 2.7B and 7B parameters.⁵⁷

Timeline

1. 2017Began a PhD in machine learning and computational neuroscience at the University of Edinburgh.²
2. Apr 2021Joined the University of Oxford as a postdoctoral researcher working with Rafal Bogacz.²
3. Sep 2022Became Head of Research at Conjecture.²
4. May 2023Co-founded Apollo Research as one of four co-founders.²
5. Jul 2023Co-founded Zyphra and became its Chief Scientist.²
6. May 2024Released the Zamba technical report as senior author.³
7. Nov 2024Released the Zamba2 suite technical report as senior author.⁴
8. May 2026Released the Zamba2-VL technical report as senior author.⁵

Other documented roles

Roles the sources document that are not confirmed as current.

• Co-founder (May 2023 to July 2023), Apollo Research²
• Head of Research (September 2022 to May 2023), Conjecture²
• Postdoctoral researcher in computational neuroscience (April 2021 to September 2022), University of Oxford²¹

Where they work

• Zyphra 4 people in the wiki

Related people

• Paolo GloriosoAlso at Zyphra
• Krithik PuthalathAlso at Zyphra
• Hassan ShapourianAlso at Zyphra
• Stephen RollerLanguage models · Training systems
• Sid BlackTraining systems · Language models
• Danqi ChenLanguage models · Training systems
• Jakub PachockiLanguage models · Training systems
• Colin RaffelLanguage models · Training systems
• Bryan CatanzaroLanguage models · Training systems
• Lucas AtkinsLanguage models · Training systems