Song Han

Career

Song Han received a PhD from Stanford University advised by Bill Dally. Han is now an associate professor with tenure in MIT's Department of Electrical Engineering and Computer Science, where Han leads the MIT HAN Lab.¹²

Han is also a research director at NVIDIA, where Han leads the Efficient AI team. NVIDIA's profile notes that Han's team has recently worked on language model quantization, sparse attention, long-context inference and efficient visual generation.³

Research

Han's research focuses on efficient AI computing. Deep Compression (2015), written with Huizi Mao and Bill Dally, combined pruning, trained quantization and Huffman coding to shrink neural networks, and the Efficient Inference Engine (EIE, 2016) brought weight sparsity to AI accelerator design. Han has received best paper awards at ICLR 2016, FPGA 2017 and MLSys 2024.¹⁹¹⁰

Han's group extended this work to hardware-aware neural architecture search with Once-for-All (2019) and to TinyML with MCUNet (2020), which runs deep learning on microcontrollers. For large language models, the lab developed SmoothQuant (2022), AWQ (2023), which received the MLSys 2024 best paper award, and StreamingLLM (2023), and Han's site notes that these methods were adopted by NVIDIA TensorRT-LLM. Han also co-authored SANA (2024), a linear diffusion transformer for efficient high-resolution image synthesis.^1112647113

Teaching

Han developed EfficientML.ai, an open lecture series and the course site for MIT 6.5940, TinyML and Efficient AI Computing, to share advances in efficient machine learning research.¹⁸

Work and contributions

• AWQ (Activation-aware Weight Quantization) and TinyChat Open-source code

  Co-author of the AWQ paper (2023) with Ji Lin, Jiaming Tang and others, which also presents TinyChat, an inference framework for 4-bit on-device language and vision-language models. The MIT-licensed llm-awq repository describes TinyChat as a turn-key solution for on-device inference on resource-constrained edge platforms and shows it running on NVIDIA Jetson Orin, and Han's MIT page notes the paper's Best Paper Award at MLSys 2024.⁴⁵¹⁴¹

• SmoothQuant Open-source code

  Co-author of SmoothQuant (2022), a training-free 8-bit post-training quantization method for large language models; the code is published by MIT HAN Lab under the MIT license.⁶¹⁵¹

• StreamingLLM Open-source code

  Co-author with Guangxuan Xiao, Yuandong Tian, Beidi Chen and Mike Lewis of "Efficient Streaming Language Models with Attention Sinks" (2023), which keeps attention-sink tokens in the KV cache so a model can generate long text with a fixed memory budget; the code is MIT-licensed.⁷¹⁶¹

• TinyML and Efficient AI Computing (MIT 6.5940, EfficientML.ai) Published research

  Developed the open lecture series EfficientML.ai, the course site for MIT 6.5940 TinyML and Efficient AI Computing, to share advances in efficient machine learning research.¹⁸

Timeline

1. Oct 1, 2015Co-authored Deep Compression, on pruning, trained quantization and Huffman coding of neural networks.⁹
2. Feb 4, 2016First author of EIE, an efficient inference engine for compressed deep neural networks.¹⁰
3. Aug 26, 2019Co-authored Once-for-All, on training one network and specializing it for efficient deployment.¹¹
4. 2020Received the NSF CAREER Award.¹
5. Jul 20, 2020Co-authored MCUNet, on tiny deep learning for IoT devices.¹²
6. Nov 18, 2022Co-authored SmoothQuant, a training-free 8-bit quantization method for large language models.⁶
7. 2023Received a Sloan Research Fellowship.¹
8. Jun 1, 2023Co-authored AWQ, activation-aware weight quantization for compressing and accelerating language models.⁴
9. Sep 29, 2023Co-authored StreamingLLM, on efficient streaming language models with attention sinks.⁷
10. Jun 13, 2024Received the MLSys 2024 Best Paper Award for AWQ.¹
11. Oct 14, 2024Co-authored SANA, efficient high-resolution image synthesis with linear diffusion transformers.¹³

Honors and recognition

• NSF CAREER Award · U.S. National Science Foundation, 2020¹
• Sloan Research Fellowship · Alfred P. Sloan Foundation, 2023¹
• ICLR Best Paper Award · International Conference on Learning Representations, 2016¹
• FPGA Best Paper Award · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2017¹
• MLSys Best Paper Award (AWQ) · Conference on Machine Learning and Systems, 2024¹

Where they work

• MIT 26 people in the wiki
• NVIDIA 43 people in the wiki

Related people

• Bill DallyAlso at MIT
• Mathias LechnerAlso at MIT
• Saadia GabrielAlso at MIT
• Jonathan FrankleAlso at MIT
• Karen LivescuAlso at MIT
• Azalia MirhoseiniChips and hardware · Language models
• Arjun DesaiLanguage models · Inference and serving
• Max KrasnyanskyInference and serving · Chips and hardware
• Jeff BolzInference and serving · Chips and hardware
• Johannes GäßlerInference and serving · Chips and hardware