Bryan Catanzaro

Career

Bryan Catanzaro earned a PhD in Electrical Engineering and Computer Sciences at the University of California, Berkeley, focusing on parallel computing, machine learning, and programming models. NVIDIA's blogs say Catanzaro's research at NVIDIA led to the creation of the cuDNN library. The developer blog adds that, prior to NVIDIA, Catanzaro worked at Baidu on systems for training and deploying end-to-end deep learning speech recognition.¹²

As Vice President of Applied Deep Learning Research at NVIDIA, Catanzaro leads a team applying deep learning to areas from language understanding and video games to computer graphics and chip design, and helped lead the team that invented DLSS 2.0. NVIDIA's GTC session listings for October 2026 include a talk by Catanzaro on building open models that developers can inspect, adapt, and deploy.²¹³

Research

Catanzaro co-authored the 2014 cuDNN paper, which describes a library of optimized deep learning primitives for GPUs, and the 2015 Deep Speech 2 paper on end-to-end speech recognition in English and Mandarin.⁴⁷

Catanzaro's language model work includes the 2019 Megatron-LM paper on intra-layer model parallelism and the 2021 paper on efficient large-scale training on GPU clusters with Megatron-LM. Catanzaro also wrote NVIDIA technical blog posts on Megatron, including one on scaling training to a trillion parameters, and a 2023 post introducing NeMo Data Curator for building trillion-token datasets.⁵⁸⁹²

Open models

In 2025 Catanzaro wrote a series of NVIDIA blog posts on the company's open models and data, including the September post on how Nemotron models, datasets, and techniques are shared, an October post on open models for language, biology, and robotics, and a December post on open model development announced at NeurIPS. Catanzaro is also listed in the leadership credits of the December 2025 Nemotron 3 white paper.¹⁶

Work and contributions

• Megatron-LM Open-source code

  Co-author, with Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley and Jared Casper, of the 2019 Megatron-LM paper on intra-layer model parallelism for training transformer language models; the NVIDIA/Megatron-LM repository README asks users to cite this paper.⁵⁸

• NVIDIA Nemotron Open weights

  Listed in the leadership credits and the NVFP4 contributor group of the NVIDIA Nemotron 3 white paper (December 2025), which states that NVIDIA will release the model weights, training datasets and recipes; also wrote NVIDIA's September 2025 post "Open Secret: How NVIDIA Nemotron Models, Datasets and Techniques Fuel AI Development".⁶¹

• cuDNN Published research

  Co-author of "cuDNN: Efficient Primitives for Deep Learning" (2014), which describes a BLAS-like library of optimized deep learning routines for GPUs; NVIDIA's developer blog says Catanzaro's research at NVIDIA led to the creation of cuDNN.⁴²

• Deep Speech 2 Published research

  Co-author of "Deep Speech 2: End-to-End Speech Recognition in English and Mandarin" (2015), from the period when Catanzaro worked at Baidu on end-to-end deep learning speech recognition.⁷²

• Nemotron-CC Open data

  Co-author, listed last, of the 2024 paper introducing Nemotron-CC, an English pretraining dataset built from Common Crawl.¹⁰

Timeline

1. Oct 3, 2014Co-authored "cuDNN, Efficient Primitives for Deep Learning."⁴
2. Dec 8, 2015Co-authored the Deep Speech 2 paper on end-to-end speech recognition.⁷
3. Sep 17, 2019Co-authored the Megatron-LM paper on training multi-billion-parameter language models.⁵
4. Apr 9, 2021Co-authored "Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM."⁹
5. Aug 8, 2023Wrote the NVIDIA technical blog post introducing NeMo Data Curator.²
6. Oct 28, 2025Wrote NVIDIA's post on open models and data for language, biology, and robotics.¹
7. Dec 2025Listed in the leadership credits of the NVIDIA Nemotron 3 white paper.⁶
8. Dec 1, 2025Wrote NVIDIA's post on open model development for digital and physical AI at NeurIPS.¹

Other documented roles

Roles the sources document that are not confirmed as current.

• Worked on systems for training and deploying end-to-end deep learning speech recognition, Baidu²

Where they work

• NVIDIA 43 people in the wiki

Related people

• Clément FarabetAlso at NVIDIA
• Jiaxuan YouAlso at NVIDIA
• Song HanAlso at NVIDIA
• Yejin ChoiAlso at NVIDIA
• Jonathan CohenAlso at NVIDIA
• Stas BekmanTraining systems · Language models
• Liliang RenLanguage models · Training systems
• Jakub PachockiLanguage models · Training systems
• Colin RaffelLanguage models · Training systems
• Tom BrownTraining systems · Language models