Jiatong Shi

Career

Jiatong Shi was a PhD student in WAVLab, the Audio and Voice Lab led by Shinji Watanabe at Carnegie Mellon University's Language Technologies Institute, from September 2019 to December 2025 according to the lab's member list. Shi's GitHub profile now lists Anuttacon as Shi's organization.³²¹

Research

Shi's research covers speech and music processing, with a focus on toolkits and benchmarks. In 2022 Shi was first author of Muskits, an open-source toolkit for end-to-end singing voice synthesis that allows reproducible comparison of published methods. In 2023 Shi was first author of ML-SUPERB, which extends the SUPERB benchmark for self-supervised speech models to 143 languages for speech recognition and language identification.⁶⁷

ML-SUPERB 2.0, accepted at Interspeech 2024, broadened that benchmark across modeling constraints, languages, and datasets. Shi is also a co-author of "Towards Robust Speech Representation Learning for Thousands of Languages" (EMNLP 2024) and of EFFUSE, a method for fusing self-supervised features for multilingual and low-resource speech recognition.⁸⁹¹⁰

Open-source work

Shi is first author of VERSA, a toolkit that gathers evaluation metrics for speech, audio, and music in one place, released in the WAVLab GitHub organization under the Apache-2.0 license. The VERSA paper lists Shinji Watanabe as last author.⁴⁵

Work and contributions

• VERSA Open-source code

  First author of VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music (2024), with Shinji Watanabe as last author; the WAVLab repository is published under the Apache-2.0 license.⁴⁵

• Muskits Open-source code

  First author of the 2022 paper introducing Muskits, an open-source end-to-end music processing toolkit focused on singing voice synthesis, modeled on the ESPnet and Kaldi recipe style.⁶

• ML-SUPERB Published research

  First author of ML-SUPERB (2023), a multilingual benchmark for self-supervised speech models covering 143 languages, and of its follow-up ML-SUPERB 2.0 (2024).⁷⁸

Timeline

1. Sep 2019Began PhD work in WAVLab at Carnegie Mellon University's Language Technologies Institute.³
2. May 9, 2022Released the Muskits paper on an end-to-end singing voice synthesis toolkit.⁶
3. May 18, 2023Released ML-SUPERB, a multilingual benchmark for self-supervised speech models covering 143 languages.⁷
4. Jun 12, 2024Released ML-SUPERB 2.0, later accepted at Interspeech 2024.⁸
5. Jun 30, 2024Co-authored "Towards Robust Speech Representation Learning for Thousands of Languages."⁹
6. Dec 23, 2024Released the VERSA evaluation toolkit paper for speech, audio, and music.⁴
7. Dec 2025Concluded the PhD period in WAVLab listed on the lab's member page.³

Other documented roles

Roles the sources document that are not confirmed as current.

• PhD student in WAVLab (2019–2025), Carnegie Mellon University (WAVLab)³

Honors and recognition

• EMNLP 2024 Best Paper Award ("Towards Robust Speech Representation Learning for Thousands of Languages", co-author) · EMNLP 2024 (Association for Computational Linguistics), 2024¹¹¹²
• Interspeech 2024 Best Paper Award (EFFUSE, co-author) · Interspeech 2024 (International Speech Communication Association), 2024¹⁰

Where they work

• Anuttacon 1 person in the wiki

Related people

• Shinji WatanabeSpeech and audio · Evaluation and benchmarks
• Emmanuel DupouxSpeech and audio · Evaluation and benchmarks
• Karen LivescuSpeech and audio · Evaluation and benchmarks
• Ju-Chieh ChouSpeech and audio
• Swabha SwayamdiptaEvaluation and benchmarks
• Gabriel SynnaeveSpeech and audio
• Collin BurnsEvaluation and benchmarks
• Ronan CollobertSpeech and audio
• Rama AkkirajuSpeech and audio
• Yulia TsvetkovEvaluation and benchmarks