Hoifung Poon

Research

Poon's early work with Pedro Domingos introduced sum-product networks, a deep probabilistic architecture that grew out of asking when the partition function of a graphical model is tractable. The paper showed that essentially all tractable graphical models can be cast as sum-product networks.⁴

Later research turned to biomedicine. Poon co-authored PubMedBERT, which showed the value of pretraining language models from scratch on biomedical text, and BioGPT, a generative transformer for biomedical text generation and mining. Multimodal follow-ups included BiomedCLIP, trained on image-text pairs from scientific articles, the LLaVA-Med assistant and BiomedParse for parsing biomedical images.⁵⁶⁷²⁸

In a February 2025 Microsoft Research Forum keynote, Poon set out an agenda for precision health that uses generative AI to learn patient embeddings from multimodal, longitudinal patient records. The talk described GigaPath, a digital pathology foundation model developed with Providence Health System and the University of Washington.¹

Work and contributions

• LLaVA-Med Published research

  Co-author of the 2023 paper "LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day", which fine-tunes a general-domain vision-language model on instruction data generated from PubMed Central figure captions.²

• Medprompt Published research

  Co-author of "Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine" (2023), which introduces Medprompt, a composition of prompting strategies evaluated on medical benchmark datasets.³

• GigaPath Published research

  Presented GigaPath in a Microsoft Research Forum keynote as a digital pathology foundation model created by Poon's team with Providence Health System and the University of Washington, adapting dilated attention to whole-slide images, and published in Nature.¹

Timeline

1. Feb 2012Posted "Sum-Product Networks, A New Deep Architecture" with Pedro Domingos as first author.⁴
2. Jul 2020Co-authored PubMedBERT, on domain-specific language model pretraining for biomedical NLP.⁵
3. Oct 2022Co-authored BioGPT, a generative pre-trained transformer for biomedical text.⁶
4. Mar 2023Co-authored BiomedCLIP, a multimodal biomedical foundation model.⁷
5. Jun 2023Co-authored LLaVA-Med, a language-and-vision assistant for biomedicine.²
6. Nov 2023Co-authored the Medprompt study of generalist foundation models in medicine.³
7. May 2024Co-authored BiomedParse, a biomedical foundation model for image parsing.⁸
8. Feb 2025Gave a Microsoft Research Forum keynote on multimodal generative AI for precision health.¹

Documented roles

The sources do not confirm a current role; these are the roles they document.

• General Manager, Microsoft Health Futures (as introduced in February 2025), Microsoft Health Futures (Microsoft Research)¹

Companies in their history

• Microsoft 75 people in the wiki

Related people

• Xiaodong HeConnected to Microsoft
• Jyoti AnejaConnected to Microsoft
• Jianfeng GaoConnected to Microsoft
• David BauConnected to Microsoft
• Su Lin BlodgettConnected to Microsoft
• Sasha ShengLanguage models · Multimodal models
• Junyang LinLanguage models · Multimodal models
• Ali FarhadiMultimodal models · Language models
• Alex RivesAI for science · Language models
• Sergey EdunovLanguage models · AI for science