Graham Neubig

Research

Neubig's research statement at Carnegie Mellon describes a long-term goal of breaking down barriers in human-human and human-machine communication through natural language processing, including machine translation and natural language understanding. The work spans advances in core NLP methods and models of language connected to other modalities such as speech, and the faculty page lists applications of large language models to tasks such as software development.¹

Neubig's papers include a 2017 single-author tutorial on neural machine translation and sequence-to-sequence models, the XTREME benchmark (2020) for evaluating cross-lingual generalization across many languages, and a 2021 survey that organized research on prompting methods in natural language processing.⁷⁸⁹

Agents and open-source work

In 2017 Neubig was first author of the paper describing DyNet, a toolkit for neural networks whose computation graphs are built dynamically. More recent work centers on language agents. Neubig co-authored WebArena (2023), a realistic web environment for building and testing autonomous agents, and SOTOPIA, which evaluates social intelligence in language agents.¹⁰¹¹⁵

Neubig is a co-author of the 2024 paper introducing OpenHands, an open platform for AI software developers released under the MIT license, and the OpenHands team page lists Neubig as Chief Scientist. The CMU faculty page notes that Neubig shares papers, course materials, video lectures and open-source software openly.³⁴²¹

Work and contributions

• OpenHands Open-source code

  Listed among the authors of the 2024 paper introducing OpenHands as an open platform for AI software developers as generalist agents; the OpenHands repository is released under the MIT license.³⁴

• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents Published research

  Co-author of the paper presenting SOTOPIA, an open-ended environment in which language agents role-play social scenarios and are scored with the SOTOPIA-Eval framework.⁵

Timeline

1. Jan 2017Posted the DyNet dynamic neural network toolkit paper as first author.¹⁰
2. Mar 2017Posted a tutorial on neural machine translation and sequence-to-sequence models as sole author.⁷
3. Mar 2020Co-authored XTREME, a multilingual multi-task benchmark for cross-lingual generalization.⁸
4. Jul 2021Co-authored a systematic survey of prompting methods in natural language processing.⁹
5. Jul 2023Co-authored WebArena, a realistic web environment for building autonomous agents.¹¹
6. Oct 2023Co-authored SOTOPIA, an environment for evaluating social intelligence in language agents.⁵
7. Jul 2024Co-authored the paper introducing OpenHands, an open platform for AI software developers.³

Where they work

• Carnegie Mellon University 24 people in the wiki
• All Hands AI 3 people in the wiki

Related people

• Zhilin YangAlso at Carnegie Mellon University
• Christopher ManningAlso at Carnegie Mellon University
• Karen LivescuAlso at Carnegie Mellon University
• Ruslan SalakhutdinovAlso at Carnegie Mellon University
• Maarten SapAlso at Carnegie Mellon University
• Omar KhattabDeveloper tools · Language models
• Binyuan HuiAgents · Language models
• Michele CatastaAgents · Language models
• Harrison ChaseAgents · Developer tools
• Winston WeinbergAgents · Language models