Patrick Esser

Research

Esser's early papers, with Björn Ommer and colleagues, include work on conditional image generation, such as "A Variational U-Net for Conditional Appearance and Shape Generation" (2018) as first author. With Robin Rombach, Esser then introduced VQGAN in "Taming Transformers for High-Resolution Image Synthesis" (CVPR 2021), which pairs a convolutional codebook with a Transformer to generate high-resolution images.⁸¹²

Esser co-authored the latent diffusion paper (CVPR 2022), which runs diffusion in the latent space of a pretrained autoencoder, and was first author of "Structure and Content-Guided Video Synthesis with Diffusion Models" (2023). In 2024 Esser was first author of the rectified flow transformer scaling paper and co-authored latent adversarial diffusion distillation for fast image synthesis.⁵⁴⁹⁷¹⁰

Later work

Esser is listed among the authors of Black Forest Labs' 2025 FLUX.1 Kontext paper on flow matching for in-context image generation and editing, and co-authored "Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis" (2026) with Robin Rombach and others. Code for Taming Transformers and latent diffusion is published in the CompVis GitHub organization under MIT-style licenses.¹¹¹²³⁶

Work and contributions

• Taming Transformers (VQGAN) Open-source code

  Co-first author with Robin Rombach, with Björn Ommer, of the CVPR 2021 paper; the CompVis repository's license names Esser, Rombach, and Ommer as copyright holders under MIT-style terms.¹²³

• Latent Diffusion Models Open-source code

  Co-author, with Robin Rombach, Andreas Blattmann, Dominik Lorenz, and Björn Ommer, of "High-Resolution Image Synthesis with Latent Diffusion Models" (CVPR 2022), listed with Runway ML in the proceedings version; the CompVis/latent-diffusion repository is released under the MIT License.⁵⁴⁶

• Scaling Rectified Flow Transformers Published research

  First author of "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis" (2024) with Sumith Kulal, Andreas Blattmann, Robin Rombach, and others.⁷

Timeline

1. Apr 12, 2018Released "A Variational U-Net for Conditional Appearance and Shape Generation" as first author.⁸
2. Dec 17, 2020Released "Taming Transformers for High-Resolution Image Synthesis" (VQGAN) as co-first author with Robin Rombach.¹²
3. Dec 20, 2021Co-authored "High-Resolution Image Synthesis with Latent Diffusion Models".⁵
4. Feb 6, 2023Released "Structure and Content-Guided Video Synthesis with Diffusion Models" as first author.⁹
5. Mar 5, 2024Released "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis" as first author.⁷
6. Mar 18, 2024Co-authored "Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation".¹⁰
7. Jun 17, 2025Listed among the authors of the FLUX.1 Kontext paper from Black Forest Labs.¹¹
8. Mar 6, 2026Co-authored "Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis".¹²

Documented roles

The sources do not confirm a current role; these are the roles they document.

• Listed with Runway ML as an author of the Latent Diffusion Models paper (CVPR 2022), Runway ML⁴

Companies in their history

• Runway 2 people in the wiki

Related people

• Yuval AlalufConnected to Runway
• Michael J. BlackComputer vision · Image, video, and 3D generation
• Christoph LassnerComputer vision · Image, video, and 3D generation
• Richard ZhangComputer vision · Image, video, and 3D generation
• Miika AittalaImage, video, and 3D generation · Computer vision
• Jonathan HoImage, video, and 3D generation · Computer vision
• Amir HertzImage, video, and 3D generation · Computer vision
• Jun-Yan ZhuImage, video, and 3D generation · Computer vision
• Samuli LaineImage, video, and 3D generation · Computer vision
• Kaiming HeComputer vision · Image, video, and 3D generation