Exploring CXL-based KV Cache Storage for LLM Serving

Yupeng Tang, Runxiang Cheng, Ping Zhou, Tongping Liu, Fei Liu, Wei Tang, Kyoungryun Bae, Jianjun Chen, Wu Xiang, Rui Shi
October 2024
Publication
To appear (NIPS 2024 ML for Systems Workshop)
Yupeng Tang
Yupeng Tang
Research Scientist @ Meta

My research interests include distributed systems, software-hardware co-design and hardware accelerators.