Christopher Ré

Career

Christopher Ré was an assistant professor at the University of Wisconsin–Madison from 2009 to 2013 before joining the Stanford computer science faculty. At Stanford, Ré is a professor in the Stanford AI Lab and is affiliated with the Machine Learning Group and the Center for Research on Foundation Models.⁸²¹

Ré's homepage describes the lab's main goal as helping students become professors, entrepreneurs, and researchers, and notes that more than a dozen group members have gone on to professorships. With students and collaborators, Ré has co-founded several companies, and Together AI lists Ré among its founders.¹³

Research

Ré's research has spanned database theory, database systems, and machine learning. The MacArthur Foundation highlighted DeepDive, an inference engine that extracts facts from text, images, and other hard-to-process data into a knowledge base. HOGWILD!, a 2011 lock-free approach to parallel stochastic gradient descent written with Feng Niu, Benjamin Recht, and Stephen J. Wright, later received the NeurIPS 2020 test-of-time award.²⁸⁹¹

More recent work aims at the foundations of foundation models. The group's papers include FlashAttention, a faster and more memory-efficient exact attention algorithm, and Hyena, a convolution-based alternative to attention that the group also applied to DNA in HyenaDNA. Ré also co-authored the HELM evaluation and the RedPajama open dataset.⁴⁵¹⁶⁷

The lab's current systems work includes ThunderKittens for building AI kernels, along with ParallelKittens for multiple GPUs and HipKittens for AMD hardware, and an Intelligence per Watt project that measures the efficiency of AI models.¹⁰¹

Work and contributions

• RedPajama Open data

  Co-author of "RedPajama: an Open Dataset for Training Large Language Models" (2024); Ré's homepage links to the RedPajama-Data repository, which is Apache-2.0.⁷¹¹¹

• HELM (Holistic Evaluation of Language Models) Published research

  Co-author of "Holistic Evaluation of Language Models" (2022).⁶

• FlashAttention Published research

  Co-author, with Tri Dao, Daniel Y. Fu, Stefano Ermon and Atri Rudra, of "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness" (2022).⁴

• Hyena Published research

  Co-author of "Hyena Hierarchy: Towards Larger Convolutional Language Models" (2023); Ré's homepage lists Hyena and its application HyenaDNA among the lab's work.⁵¹

• ThunderKittens Open-source code

  Co-author, with Benjamin F. Spector, Simran Arora, Aaryan Singhal, and Daniel Y. Fu, of the 2024 paper on ThunderKittens, a framework for writing fast AI kernels; the lab has since extended it to multi-GPU and AMD hardware.¹⁰¹

Timeline

1. 2009Became an assistant professor at the University of Wisconsin–Madison.⁸
2. Jun 28, 2011Released the HOGWILD! paper on lock-free parallel stochastic gradient descent.⁹
3. 2015Named a MacArthur Fellow by the MacArthur Foundation.⁸
4. May 27, 2022Co-authored the FlashAttention paper on fast, memory-efficient exact attention.⁴
5. Nov 16, 2022Co-authored the HELM paper on holistic evaluation of language models.⁶
6. Feb 21, 2023Co-authored the Hyena paper on convolutional language models.⁵
7. Oct 27, 2024Released the ThunderKittens paper on building AI kernels.¹⁰
8. Nov 19, 2024Co-authored the RedPajama paper on open datasets for training language models.⁷

Other documented roles

Roles the sources document that are not confirmed as current.

• Assistant professor (2009–2013), University of Wisconsin–Madison⁸

Honors and recognition

• MacArthur Fellowship · John D. and Catherine T. MacArthur Foundation, 2015⁸²
• PODS Best Paper Award · ACM Symposium on Principles of Database Systems (PODS), 2012²
• SIGMOD Best Paper Award · ACM SIGMOD Conference, 2014²
• ICML Best Paper Award · International Conference on Machine Learning (ICML), 2016²
• NeurIPS Test of Time Award (for HOGWILD!) · Conference on Neural Information Processing Systems (NeurIPS), 2020²¹
• PODS Test of Time Award · ACM Symposium on Principles of Database Systems (PODS), 2022²

Where they work

• Stanford University 40 people in the wiki
• Together AI 5 people in the wiki

Related people

• Ce ZhangAlso at Stanford University
• Rohan TaoriAlso at Stanford University
• Christopher ManningAlso at Stanford University
• Fei-Fei LiAlso at Stanford University
• Azalia MirhoseiniAlso at Stanford University
• Sara HookerLanguage models · Datasets and data
• Matei ZahariaTraining systems · Datasets and data
• Zach NussbaumLanguage models · Datasets and data
• Lucas AtkinsLanguage models · Training systems
• Sam McCandlishTraining systems · Language models