Tim Dettmers

Career

Tim Dettmers completed a PhD at the University of Washington advised by Luke Zettlemoyer. Dettmers is now an assistant professor at Carnegie Mellon University and a research scientist at the Allen Institute for AI, with the stated goal of making AI accessible enough that anyone can tinker with it, learn from it and use it in their own work.¹⁴⁵

Research

Dettmers's research holds that computationally efficient methods accelerate progress in deep learning. First-author papers include 8-bit optimizers via block-wise quantization (2021), LLM.int8() for 8-bit matrix multiplication in large Transformers (2022), QLoRA for fine-tuning 4-bit quantized models through low-rank adapters (2023), and SpQR, a sparse-quantized format for near-lossless weight compression (2023).¹⁶⁷³⁸

Dettmers also co-authored Petals (2022), a system for collaborative inference and fine-tuning of large models over the internet. Current interests listed by Dettmers include open-source agents, on-device mixture-of-experts models, hierarchical LLM architectures and scientific automation.⁹¹

Open-source work

Dettmers created and maintains bitsandbytes, a PyTorch library that brings these quantization methods to practitioners. Its README describes k-bit quantization support including 8-bit optimizers, LLM.int8() inference and 4-bit quantization for QLoRA fine-tuning.¹²

Work and contributions

• bitsandbytes

  Created the library and maintains it; its README cites Dettmers's papers for the 8-bit optimizers, LLM.int8(), and QLoRA features.¹⁴²

• QLoRA

  First author of the paper introducing QLoRA, a method for fine-tuning 4-bit quantized language models through low-rank adapters.³

Timeline

1. Oct 2021Posted "8-bit Optimizers via Block-wise Quantization" as first author.⁶
2. Aug 2022Posted LLM.int8(), 8-bit matrix multiplication for Transformers at scale, as first author.⁷
3. Sep 2022Co-authored Petals, a system for collaborative inference and fine-tuning of large models.⁹
4. May 2023Posted QLoRA, efficient fine-tuning of quantized language models, as first author.³
5. Jun 2023Posted SpQR, a sparse-quantized representation for LLM weight compression, as first author.⁸

Honors and recognition

• Google Open Source Award · Google, 2023¹
• PyTorch Foundation Award · PyTorch Foundation, 2023¹
• Madrona Prize · Madrona Venture Group, 2023¹
• AI2050 Early Career Fellow · Schmidt Sciences, 2024¹
• Google ML and Systems Junior Faculty Award · Google, 2025¹

Where they work

• Carnegie Mellon University 24 people in the wiki
• Ai2 23 people in the wiki

Related people

• Carlos GuestrinAlso at Carnegie Mellon University
• Tianqi ChenAlso at Carnegie Mellon University
• Zhilin YangAlso at Carnegie Mellon University
• Maarten SapAlso at Carnegie Mellon University
• Christopher ManningAlso at Carnegie Mellon University
• Daniel HanTraining systems · Inference and serving
• Robert ShawInference and serving · Training systems
• Hao ZhangInference and serving · Training systems
• Byron HsuTraining systems · Inference and serving
• Michael IntratorTraining systems · Inference and serving