~ whoami

Zhihao Zhang

PhD student at CMU Catalyst, advised by Zhihao Jia.
I build systems that make large language models faster to train and serve.

// selected papers

2026

OSDI

Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing Tensor Programs

Xinhao Cheng*, Zhihao Zhang*, Yu Zhou*, Jianan Ji*, Jinchen Jiang, Zepeng Zhao, Ziruo Xiao, Zihao Ye, Yingyi Huang, Ruihang Lai, Hongyi Jin, Bohan Hou, Mengdi Wu, Yixin Dong, Anthony Yip, Zihao Ye, Songting Wang, Wenqin Yang, Xupeng Miao, Tianqi Chen, Zhihao Jia

2025

ICLR

TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention

Lijie Yang*, Zhihao Zhang*, Zhuofu Chen, Zikun Li, Zhihao Jia

2024

ICML

Accelerating Retrieval-Augmented Language Model Serving with Speculation

Zhihao Zhang, Alan Zhu, Lijie Yang, Yihua Xu, Lanting Li, Phitchaya Mangpo Phothilimthana, Zhihao Jia

2024

ASPLOS

SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification

Xupeng Miao*, Gabriele Oliaro*, Zhihao Zhang*, Xinhao Cheng*, Zeyu Wang, Rae Ying Yee Wong, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, Zhihao Jia

2021

ICLR

GradSign: Model Performance Inference with Theoretical Insights

Zhihao Zhang, Zhihao Jia