Xinlei Chen

Career

Xinlei Chen received a PhD from the Language Technologies Institute at Carnegie Mellon University while working at the Robotics Institute, where Chen's projects included the Never Ending Image Learner, web-scale ConvNet training, and object and sense discovery.¹

Chen then became a research scientist at Meta FAIR, working on pre-training and understanding visual representations, and later a Member of Technical Staff at xAI working on multimodal understanding and generation. Chen's site says Chen is now at OpenAI, contributing to its multimodal work.¹

Research

Chen was first author of "Microsoft COCO Captions: Data Collection and Evaluation Server" (2015), which describes the caption annotations and evaluation server for the COCO dataset. Chen's work with Kaiming He on self-supervised learning includes MoCo v2 (2020), which improved baselines for momentum contrastive learning; SimSiam (2020), a simple Siamese method that learns representations without negative pairs; and a 2021 empirical study of training self-supervised Vision Transformers that benchmarks MoCo v3.⁵⁶²³

Chen co-authored Masked Autoencoders (2021), which pre-train Vision Transformers by reconstructing masked image patches, and was first author of "Deconstructing Denoising Diffusion Models for Self-Supervised Learning" (2024), which studies the representation learning abilities of diffusion models.⁴⁷

Work and contributions

• SimSiam Published research

  First author, with Kaiming He, of "Exploring Simple Siamese Representation Learning" (2020).²

• Self-supervised Vision Transformers (MoCo v3 study) Published research

  First author, with Saining Xie and Kaiming He, of "An Empirical Study of Training Self-Supervised Vision Transformers" (2021), which benchmarks ViT results in MoCo v3 and other self-supervised frameworks.³

Timeline

1. Apr 1, 2015First author of "Microsoft COCO Captions - Data Collection and Evaluation Server".⁵
2. Mar 9, 2020First author of "Improved Baselines with Momentum Contrastive Learning" (MoCo v2).⁶
3. Nov 20, 2020First author with Kaiming He of SimSiam, "Exploring Simple Siamese Representation Learning".²
4. Apr 5, 2021First author of "An Empirical Study of Training Self-Supervised Vision Transformers", which benchmarks MoCo v3.³
5. Nov 11, 2021Co-authored "Masked Autoencoders Are Scalable Vision Learners".⁴
6. Jan 25, 2024First author of "Deconstructing Denoising Diffusion Models for Self-Supervised Learning".⁷

Other documented roles

Roles the sources document that are not confirmed as current.

• Member of Technical Staff, xAI¹
• Research Scientist, FAIR, Meta (FAIR)¹

Where they work

• OpenAI 63 people in the wiki

Related people

• Aditya RameshAlso at OpenAI
• Alexander KirillovAlso at OpenAI
• Aravind SrinivasAlso at OpenAI
• Alex NicholAlso at OpenAI
• Jim FanAlso at OpenAI
• Hassan ShapourianMultimodal models · Computer vision
• Fei-Fei LiComputer vision · Multimodal models
• Cordelia SchmidComputer vision · Multimodal models
• Trevor DarrellComputer vision · Multimodal models
• Ali FarhadiComputer vision · Multimodal models