AI People Wiki

Builders & open source

Shinji Watanabe

Associate Professor, Carnegie Mellon University

Shinji Watanabe is an Associate Professor at Carnegie Mellon University, working on automatic speech recognition, speech enhancement, spoken language understanding, and machine learning for speech and language processing. Before CMU, Watanabe was a research scientist at NTT Communication Science Laboratories, a senior principal research scientist at Mitsubishi Electric Research Laboratories, and an associate research professor at Johns Hopkins University. Watanabe's homepage lists the ESPnet end-to-end speech processing toolkit, the VERSA evaluation toolkit, and Open Whisper-style Speech Models (OWSM) as software; Watanabe is first author of the ESPnet paper and last author of the VERSA paper, and both repositories use the Apache-2.0 license.12345

At a glance

Current role
Associate Professor
Sector
University
Education
Carnegie Mellon, Georgia Tech, Johns Hopkins
Work listed
2
Honors
3
Sources cited
7
Profile reviewed

Career

Shinji Watanabe was a research scientist at NTT Communication Science Laboratories from 2001 to 2011, with a visiting scholarship at the Georgia Institute of Technology in 2009. Watanabe was then a senior principal research scientist at Mitsubishi Electric Research Laboratories from 2012 to 2017 and an associate research professor at Johns Hopkins University from 2017 to 2020.1

Watanabe is now an associate professor at Carnegie Mellon University, and is listed as the faculty member of the WAVLab speech research group. Watanabe has chaired the IEEE Signal Processing Society's Speech and Language Technical Committee.16

Open-source work

Watanabe is first author of the 2018 paper introducing ESPnet, an Apache-2.0 end-to-end speech processing toolkit, and is last author of the 2024 paper on VERSA, a toolkit for versatile evaluation of speech and audio.2345

Watanabe's group also develops Open Whisper-style Speech Models (OWSM). The 2023 paper introducing OWSM, with Watanabe as last author, reproduces Whisper-style multilingual speech recognition and translation training using an open-source toolkit and publicly available data.17

Work and contributions

  • ESPnet Open-source code

    First author of ESPnet: End-to-End Speech Processing Toolkit (2018); Watanabe's homepage lists ESPnet as software, and the repository is published under the Apache-2.0 license.213

  • VERSA Open-source code

    Last author of VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music (2024), listed on Watanabe's homepage as Versatile Evaluation of Speech and Audio; the repository is published under the Apache-2.0 license.415

Timeline

  1. Joined NTT Communication Science Laboratories as a research scientist.1
  2. Joined Mitsubishi Electric Research Laboratories as a senior principal research scientist.1
  3. Became an associate research professor at Johns Hopkins University.1
  4. Posted the ESPnet end-to-end speech processing toolkit paper as first author.2
  5. Co-authored the paper introducing Open Whisper-style Speech Models.7
  6. Received the ISCA Interspeech best paper award.1
  7. Co-authored the VERSA speech and audio evaluation toolkit paper.4

Other documented roles

Roles the sources document that are not confirmed as current.

  • Associate research professor (2017–2020), Johns Hopkins University1

Honors and recognition

  • IEEE Fellow · Institute of Electrical and Electronics Engineers1
  • ISCA Fellow · International Speech Communication Association1
  • Best Paper Award, Interspeech 2024 · International Speech Communication Association, 20241

Where they work

Suggest a correction or removal

Profiles cover public professional work only, and each statement cites a source. If something is out of date, missing or wrong, tell us and link a source. People can also ask for their profile to be corrected or removed.

References

7 sources · profile reviewed Oct 21, 2026 · data updated Oct 21, 2026

  1. Shinji Watanabe — homepage (short bio and software)Shinji Watanabe · Official page · accessed Oct 21, 2026
  2. ESPnet: End-to-End Speech Processing Toolkit (arXiv 1804.00015)arXiv · Paper · published Mar 30, 2018 · accessed Oct 21, 2026
  3. espnet/espnet repository (Apache-2.0 license)ESPnet (GitHub repository) · Repository · accessed Oct 21, 2026
  4. VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music (arXiv 2412.17667)arXiv · Paper · published Dec 23, 2024 · accessed Oct 21, 2026
  5. wavlab-speech/versa repository (Apache-2.0 license)WAVLab (GitHub repository) · Repository · accessed Oct 21, 2026
  6. WAVLab (Carnegie Mellon University) — Members and alumniWAVLab, Carnegie Mellon University · Official page · accessed Oct 21, 2026
  7. Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data (arXiv 2309.13876)arXiv · Paper · published Sep 25, 2023 · accessed Oct 21, 2026