Paul Christiano

Career

Paul Christiano received a PhD in statistical learning theory from UC Berkeley and later ran the language model alignment team at OpenAI. Christiano then founded the Alignment Research Center (ARC), a nonprofit developing methods for white-box analysis of neural networks, and is its executive director.¹

In April 2024 U.S. Secretary of Commerce Gina Raimondo named Christiano head of AI safety at the U.S. AI Safety Institute, housed at the National Institute of Standards and Technology. Christiano's site says Christiano previously served as Head of Safety at the Center for AI Standards and Innovation and the AI Safety Institute and is now a technical advisor at the Center for AI Standards and Innovation within NIST.²¹

Research

Christiano co-authored "Concrete Problems in AI Safety" (2016) and was first author of "Deep reinforcement learning from human preferences" (2017), which trains agents on goals defined by human comparisons between pairs of behavior clips. Christiano also co-authored the 2020 paper "Learning to summarize from human feedback", which applied this kind of preference learning to language models.⁴³⁵

With Buck Shlegeris and Dario Amodei, Christiano wrote "Supervising strong learners by amplifying weak experts" (2018), which proposes iterated amplification for training systems on tasks too hard for humans to evaluate directly. Christiano later co-authored "Formalizing the presumption of independence" (2022) with Eric Neyman and Mark Xu, on formalizing heuristic reasoning that relies on default independence assumptions.⁶⁷

Work and contributions

• Deep reinforcement learning from human preferences Published research

  First author of the 2017 paper, which trains reinforcement learning agents on goals defined by non-expert human preferences between pairs of trajectory segments, solving Atari games and simulated robot locomotion tasks without access to the reward function.³

Timeline

1. Jun 21, 2016Co-authored "Concrete Problems in AI Safety".⁴
2. Jun 12, 2017First author of "Deep reinforcement learning from human preferences".³
3. Oct 19, 2018First author of "Supervising strong learners by amplifying weak experts", on iterated amplification.⁶
4. Sep 2, 2020Co-authored "Learning to summarize from human feedback".⁵
5. Nov 12, 2022First author of "Formalizing the presumption of independence" with Eric Neyman and Mark Xu.⁷
6. Apr 16, 2024Named head of AI safety at the U.S. AI Safety Institute at NIST.²

Other documented roles

Roles the sources document that are not confirmed as current.

• Head of AI Safety, NIST (U.S. AI Safety Institute and Center for AI Standards and Innovation)²¹
• Ran the language model alignment team, OpenAI¹

Where they work

• Alignment Research Center 1 person in the wiki
• NIST 3 people in the wiki

Related people

• Wojciech ZarembaSafety and alignment · Reinforcement learning
• Adam GleaveSafety and alignment · Reinforcement learning
• Jan LeikeSafety and alignment · Reinforcement learning
• Shengjia ZhaoReinforcement learning · Safety and alignment
• Shixiang Shane GuReinforcement learning
• Pieter AbbeelReinforcement learning
• Lilian WengReinforcement learning
• Neel NandaSafety and alignment
• Jack ClarkSafety and alignment
• Saadia GabrielSafety and alignment