Jan Leike

Career

Jan Leike completed a PhD in reinforcement learning theory at the Australian National University, with a 2016 thesis on nonparametric general reinforcement learning supervised by Marcus Hutter. Leike then worked as an alignment researcher at DeepMind, where Leike prototyped reinforcement learning from human feedback.¹

At OpenAI, Leike co-led the Superalignment team, developed the company's approach to alignment research, co-authored the team's research roadmap, and was involved in InstructGPT, ChatGPT, and the alignment of GPT-4. Leike now leads the Alignment Science team at Anthropic, which works on aligning an automated alignment researcher, scalable oversight, weak-to-strong generalization, and robustness to jailbreaks.¹

Research

Leike's research asks how AI systems can be trained to follow human intent on tasks that are hard for people to evaluate directly. Leike was second author of the 2017 paper on deep reinforcement learning from human preferences and first author of AI Safety Gridworlds (2017) and of the 2018 research agenda on scalable agent alignment via reward modeling.¹²⁴³

Leike co-authored the March 2022 InstructGPT paper on training language models to follow instructions with human feedback, the December 2023 paper on weak-to-strong generalization, and the June 2024 paper "LLM Critics Help Catch LLM Bugs," which trains critic models that help people evaluate model-written code more accurately.⁵⁶⁷

Work and contributions

• Deep reinforcement learning from human preferences Published research

  Second-listed author of the 2017 paper that trains agents from human preferences between pairs of trajectory segments.²

• Scalable agent alignment via reward modeling Published research

  First author of the 2018 research-direction paper on aligning agents through reward modeling.³

• InstructGPT and weak-to-strong generalization Published research

  Co-author of "Training language models to follow instructions with human feedback" (2022) and of "Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision" (2023).⁵⁶

• AI Safety Gridworlds Published research

  First author of a 2017 suite of reinforcement learning environments illustrating safety properties such as safe interruptibility, avoiding side effects, reward gaming, and safe exploration.⁴

Timeline

1. 2016Completed a PhD thesis on nonparametric general reinforcement learning, supervised by Marcus Hutter.¹
2. Jun 12, 2017Co-authored "Deep reinforcement learning from human preferences."²
3. Nov 27, 2017Released AI Safety Gridworlds as first author.⁴
4. Nov 19, 2018First author of "Scalable agent alignment via reward modeling, a research direction."³
5. Mar 4, 2022Co-authored the InstructGPT paper on following instructions with human feedback.⁵
6. Dec 14, 2023Co-authored "Weak-to-Strong Generalization, Eliciting Strong Capabilities With Weak Supervision."⁶
7. Jun 28, 2024Co-authored "LLM Critics Help Catch LLM Bugs."⁷

Other documented roles

Roles the sources document that are not confirmed as current.

• Co-led the Superalignment team, OpenAI¹⁵
• Alignment researcher, DeepMind¹

Where they work

• Anthropic 34 people in the wiki

Related people

• Chris OlahAlso at Anthropic
• Collin BurnsAlso at Anthropic
• Ben MannAlso at Anthropic
• Samuel R. BowmanAlso at Anthropic
• Amanda AskellAlso at Anthropic
• Shengjia ZhaoReinforcement learning · Safety and alignment
• Adam GleaveSafety and alignment · Reinforcement learning
• Paul ChristianoSafety and alignment · Reinforcement learning
• Wojciech ZarembaSafety and alignment · Reinforcement learning
• Sami JaghouarReinforcement learning