Bill Dally

Career

Dally's research career began at the California Institute of Technology, where from 1983 to 1986 Dally designed the MOSSIM Simulation Engine and the Torus Routing chip, which introduced wormhole routing and virtual-channel flow control. From 1986 to 1997 Dally was at the Massachusetts Institute of Technology, where Dally's team built the J-Machine and M-Machine experimental parallel computers.²

Dally then spent 12 years at Stanford University, including as chair of the computer science department, with a team that developed architecture, routing, signaling and synchronization technology for large parallel computers. Dally joined NVIDIA in January 2009 as chief scientist and also leads NVIDIA Research as senior vice president. NVIDIA's profile notes that Dally co-founded Velio Communications and Stream Processors and is an author of four textbooks.²¹

Research

NVIDIA Research lists Dally's areas as circuits and VLSI design, computer architecture, high performance computing, networking, programming systems, and artificial intelligence and machine learning. Recent publications on the profile include work on low-energy transceivers for die-to-die interfaces.²

With Song Han and colleagues, Dally co-authored a series of papers on efficient neural networks. "Learning both Weights and Connections for Efficient Neural Networks" (2015) pruned unneeded connections, Deep Compression (2015) combined pruning, trained quantization and Huffman coding, and EIE (2016) described an inference engine that runs directly on compressed networks.⁵³⁶

Work and contributions

• Deep Compression Published research

  Co-author, with Song Han and Huizi Mao, of "Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding" (2015), which reports shrinking AlexNet and VGG-16 storage without loss of accuracy.³

• J-Machine and M-Machine

  At MIT from 1986 to 1997, Dally and team built the J-Machine and M-Machine experimental parallel computer systems, which separated mechanism from programming models and showed low-overhead synchronization and communication.²

Timeline

1. 1983Began work at Caltech, where Dally designed the MOSSIM Simulation Engine and the Torus Routing chip.²
2. 1986Joined MIT, where Dally's team built the J-Machine and M-Machine parallel computers.²
3. Jan 2009Joined NVIDIA as chief scientist after 12 years at Stanford University.²
4. Jun 2015Co-authored "Learning both Weights and Connections for Efficient Neural Networks".⁵
5. Oct 2015Co-authored Deep Compression with Song Han and Huizi Mao.³
6. Feb 2016Co-authored EIE, an efficient inference engine for compressed deep neural networks.⁶

Other documented roles

Roles the sources document that are not confirmed as current.

• Chairman of the computer science department and Willard R. and Inez Kerr Bell Professor of Engineering, Stanford University¹
• Led the team that built the J-Machine and M-Machine (1986–1997), Massachusetts Institute of Technology²

Honors and recognition

• IEEE Seymour Cray Computer Engineering Award · IEEE Computer Society, 2004⁷
• ACM Eckert-Mauchly Award · ACM and IEEE Computer Society²
• ACM Maurice Wilkes Award · Association for Computing Machinery²
• Member, National Academy of Engineering · National Academy of Engineering²
• Fellow, American Academy of Arts and Sciences · American Academy of Arts and Sciences²
• IEEE Fellow · Institute of Electrical and Electronics Engineers²
• ACM Fellow · Association for Computing Machinery²

Where they work

• NVIDIA 43 people in the wiki

Related people

• Song HanAlso at NVIDIA
• Jeff BolzAlso at NVIDIA
• Ian BuckAlso at NVIDIA
• Natalia GimelsheinAlso at NVIDIA
• Tianqi ChenAlso at NVIDIA
• Neo ZhangInference and serving · Chips and hardware
• Johannes GäßlerInference and serving · Chips and hardware
• Jeff JohnsonChips and hardware · Inference and serving
• Aaron TeoInference and serving · Chips and hardware
• Max KrasnyanskyInference and serving · Chips and hardware