Zhilin Yang

Research

Yang was first author of HotpotQA in 2018, a dataset for diverse, explainable multi-hop question answering. In 2019 Yang and Zihang Dai were co-first authors of Transformer-XL, a language model that attends beyond a fixed-length context, and of XLNet, a generalized autoregressive pretraining method built on ideas from Transformer-XL.⁶⁴²

Yang's later co-authored work includes GLM, a general language model pretrained with autoregressive blank infilling, and P-Tuning v2, which showed prompt tuning can match fine-tuning across model scales and tasks; both appeared at ACL 2022. Yang's homepage also lists TLM, a framework for NLP without large-scale pretraining at ICML 2022, and the September 2022 release of CodeGeeX, a multilingual code generation model.⁷⁸¹

Open-source work

The Transformer-XL and XLNet repositories publish code and pretrained models under the Apache 2.0 license. Yang's homepage describes the current work as a startup, following research positions at Google Brain with Quoc V. Le and at Meta AI with Jason Weston.⁵⁹³¹⁰¹

Work and contributions

• XLNet Open-source code

  Co-first author with Zihang Dai of "XLNet: Generalized Autoregressive Pretraining for Language Understanding" (2019). The Apache-2.0 repository released the code with XLNet-Large and XLNet-Base pretrained models.²³¹⁰

• Transformer-XL Open-source code

  Co-first author with Zihang Dai of "Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context" (2019). The Apache-2.0 repository under Yang's GitHub account provides PyTorch and TensorFlow code and pretrained TensorFlow models.⁴⁵⁹¹¹

Timeline

1. Sep 25, 2018Released HotpotQA, a dataset for explainable multi-hop question answering, as first author.⁶
2. 2019Completed a PhD at Carnegie Mellon's School of Computer Science.¹
3. Jan 9, 2019Released Transformer-XL as co-first author with Zihang Dai.⁴
4. Jun 19, 2019Released XLNet, a generalized autoregressive pretraining method, as co-first author.²
5. Mar 18, 2021Co-authored GLM, general language model pretraining with autoregressive blank infilling.⁷
6. Oct 14, 2021Co-authored P-Tuning v2 on prompt tuning across model scales and tasks.⁸
7. Sep 2022Released CodeGeeX, a multilingual code generation model, with collaborators.¹

Documented roles

The sources do not confirm a current role; these are the roles they document.

• Worked at Google Brain with Quoc V. Le, Google Brain¹
• Worked at Meta AI with Jason Weston, Meta AI¹
• PhD student, School of Computer Science (PhD 2019, advised by Ruslan Salakhutdinov and William W. Cohen), Carnegie Mellon University¹

Companies in their history

• Google 121 people in the wiki
• Meta 106 people in the wiki
• Carnegie Mellon University 24 people in the wiki

Related people

• David GrangierConnected to Google
• Azalia MirhoseiniConnected to Google
• Hyung Won ChungConnected to Google
• Nanyun (Violet) PengConnected to Google
• Jason WeiConnected to Google
• Dani YogatamaLanguage models
• Zach NussbaumLanguage models
• Stephen RollerLanguage models
• Tri DaoLanguage models
• Craig FederighiLanguage models