João Carreira

Career

Carreira completed a PhD at the University of Bonn in 2012 on object segmentation, advised by Cristian Sminchisescu, and then worked as a postdoctoral researcher with Jitendra Malik at Berkeley on 3D and human pose estimation. Carreira's 2010 paper with Adrian Ion and Sminchisescu proposed a mid-level image segmentation framework built on ranked figure-ground hypotheses, and a 2015 first-author paper with Malik and others introduced iterative error feedback for human pose estimation.¹²⁷⁸

At DeepMind, now Google DeepMind, Carreira has focused on learning from video. Carreira's 2026 ICIST keynote described the team's recent work on 4D reconstruction, multitask vision models, city-scale perception and modeling the future for acting and robotics.¹²

Video datasets and benchmarks

Carreira is one of the core creators of the Kinetics human action video dataset, described in a May 2017 paper, and was first author with Andrew Zisserman of the I3D paper "Quo Vadis, Action Recognition?", which introduced inflated 3D convolutional networks trained on Kinetics. Carreira was also first author of follow-up notes extending the dataset, including Kinetics-700 in 2019.⁹³⁴¹⁰

Carreira is a co-author of the Perception Test (2023), a diagnostic benchmark for multimodal video models released with data and baselines in a Google DeepMind repository.¹¹⁵

Models

Carreira co-authored the Perceiver (2021), a general architecture that uses iterative attention to handle many kinds of input. Later first-author papers include "Learning from One Continuous Video Stream" (2023), which studies learning online from a single stream of video, and "Scaling 4D Representations" (2024), on scaling self-supervised video models.⁶¹²¹³

Work and contributions

• I3D and the Kinetics dataset Published research

  First author, with Andrew Zisserman, of "Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset" (2017); Google DeepMind's kinetics-i3d repository publishes the I3D models trained on Kinetics that the paper reports.³⁴

• Perception Test Open data

  Listed among the authors in the citation entry of the Perception Test, a benchmark whose repository provides data and annotation downloads, baselines, and evaluation scripts.⁵

• Perceiver Published research

  Co-author of "Perceiver: General Perception with Iterative Attention" (2021).⁶

Timeline

1. Sep 2010Posted an image segmentation paper with Adrian Ion and Cristian Sminchisescu as first author.⁷
2. 2012Completed a PhD at the University of Bonn on object segmentation.¹
3. Jul 2015Posted "Human Pose Estimation with Iterative Error Feedback" as first author.⁸
4. May 2017Co-authored the paper describing the DeepMind Kinetics human action video dataset.⁹
5. May 2017Posted the I3D paper "Quo Vadis, Action Recognition?" with Andrew Zisserman as first author.³
6. Jul 2019Posted a note describing the Kinetics-700 human action dataset as first author.¹⁰
7. Mar 2021Co-authored "Perceiver" on general perception with iterative attention.⁶
8. May 2023Co-authored the Perception Test benchmark for multimodal video models.¹¹
9. Dec 2023Posted "Learning from One Continuous Video Stream" as first author.¹²
10. Dec 2024Posted "Scaling 4D Representations" as first author.¹³

Where they work

• Google DeepMind 64 people in the wiki

Related people

• Saining XieAlso at Google DeepMind
• Douwe KielaAlso at Google DeepMind
• Jascha Sohl-DicksteinAlso at Google DeepMind
• Nicholas CarliniAlso at Google DeepMind
• David StutzAlso at Google DeepMind
• Richard ZhangComputer vision · Evaluation and benchmarks
• Matt DeitkeMultimodal models · Computer vision
• Hassan ShapourianMultimodal models · Computer vision
• Trevor DarrellComputer vision · Multimodal models
• Christoph FeichtenhoferComputer vision · Multimodal models