Ying Sheng works on systems for serving and running large language models. Sheng is first author of FlexGen, a 2023 system that runs large language models with limited GPU memory by spreading weights and cache across GPU, CPU memory, and disk and compressing them to 4 bits. Sheng is an author of the SGLang paper and a codeowner of the SGLang repository, whose maintenance guide names Sheng and Lianmin Zheng as the people who start nominations for new maintainers. Sheng is also an officer of LMSYS, the nonprofit that hosts SGLang.1234567
At a glance
- Current roles
- Officer +1 more
- Companies
- LMSYS, SGLang project
- Fields
- Inference and serving
- Education
- Stanford
- Work listed
- 2
- Honors
- 4
- Sources cited
- 11
- Profile reviewed
Career
Ying Sheng completed a PhD in computer science at Stanford University in the Centaur group, advised by Clark Barrett. Along the way Sheng was a research intern at Microsoft Research Asia (2015), a quantitative software engineer at Two Sigma (2018), a research intern at Facebook Novi working on smart contract verification (2021), and spent 2022 at X, the moonshot factory, working on AI for code.1
Sheng was a visiting researcher with Ion Stoica and Joseph E. Gonzalez at UC Berkeley's Sky lab (2023–2024) and then a member of technical staff at xAI from October 2024 to August 2025, where Sheng co-led the inference team. Sheng's homepage notes that it is no longer kept up to date.1
Research
Sheng's early research was in formal methods and automated reasoning: "Politeness for the Theory of Algebraic Datatypes" received a best paper award at IJCAR 2020, and Sheng is a co-author of cvc5, an open-source SMT solver whose paper received the TACAS 2022 best tool paper award.8
Sheng's language model systems work includes FlexGen (2023), for high-throughput inference with limited GPU memory; S-LoRA (2023), for serving thousands of LoRA adapters concurrently; and SGLang (2023), a language and runtime for structured language model programs. Sheng also co-authored "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" (2023) and contributed to the FastChat platform and the Vicuna chatbot.293101
Open-source work
Sheng is an officer of LMSYS, a nonprofit founded to incubate early-stage open-source projects, and a codeowner of the SGLang repository. SGLang's maintenance guide names Sheng and Lianmin Zheng as the contacts who start nomination and review for new maintainers and CI on-call volunteers.61457
Work and contributions
SGLang
Listed as an author of the SGLang paper (arXiv 2312.07104) and as a codeowner for core areas of the repository, including the scheduler managers, memory cache, model executor, LoRA, and speculative decoding code.341
FlexGen (FlexLLMGen)
First author of the FlexGen paper on high-throughput inference of large language models with a single GPU. The project repository, now archived, describes running large models on a single GPU for throughput-oriented work.211
Timeline
- Received the IJCAR 2020 best paper award for work on politeness for the theory of algebraic datatypes.8
- Co-authored the cvc5 SMT solver paper, which won the TACAS 2022 best tool paper award.8
- Posted FlexGen as first author, on running large language models with a single GPU.2
- Co-authored "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena".10
- Posted S-LoRA as first author, on serving thousands of concurrent LoRA adapters.9
- Co-authored the SGLang paper on efficient execution of structured language model programs.3
- Joined xAI as a member of technical staff and co-led the inference team until August 2025.1
Other documented roles
Roles the sources document that are not confirmed as current.
Honors and recognition
- Best Paper Award, IJCAR 2020 · International Joint Conference on Automated Reasoning, 202081
- Best Tool Paper Award, TACAS 2022 (cvc5) · International Conference on Tools and Algorithms for the Construction and Analysis of Systems, 202281
- HAI Graduate Fellowship · Stanford Institute for Human-Centered Artificial Intelligence, 20221
- ML and Systems Rising Stars · MLCommons, 20231
Where they work
- LMSYS 3 people in the wiki
- SGLang project 2 people in the wiki
Related people
- Ion StoicaAlso at LMSYS
- Joseph E. GonzalezAlso at LMSYS
- Yagil BurowskiInference and serving
- Song HanInference and serving
- Matthew PrinceInference and serving
- Chris LattnerInference and serving
- Sid ShethInference and serving
- Tim DettmersInference and serving
- Radoslav GerganovInference and serving
- Davide EynardInference and serving
Suggest a correction or removal
Profiles cover public professional work only, and each statement cites a source. If something is out of date, missing or wrong, tell us and link a source. People can also ask for their profile to be corrected or removed.
References
11 sources · profile reviewed Oct 21, 2026 · data updated Oct 21, 2026
- Ying Sheng — homepage (marked by its author as outdated)
- FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU (arXiv 2303.06865)
- SGLang: Efficient Execution of Structured Language Model Programs (arXiv 2312.07104)
- sgl-project/sglang CODEOWNERS
- SGLang Code Maintenance Model (MAINTAINER.md)
- About — LMSYS
- sgl-project/sglang README
- Ying Sheng — Publications
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters (arXiv 2311.03285)
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv 2306.05685)
- FMInference/FlexLLMGen README (archived repository)