aiqbench analysis
The event centred on LLM inference, specifically how KV Cache is stored and managed across CXL memory, HBM, and serving software. For AI readers, the practical question is memory capacity and bandwidth: KV Cache grows with context length and concurrency, so pooling memory via CXL or offloading it to shared storage can change serving costs and throughput. The recap is a summary of talks rather than benchmark results, so the performance claims would need to be checked against the speakers' published data.
KVCache Meetup Korea 2026: Event Recap
On the evening of Sep 30, 2026, PyTorchKR and XCENA co-hosted KVCache Meetup Korea. PyTorchKR is a major AI/ML community in Korea, and XCENA is a startup that focuses on... The post KVCache Meetup Korea 2026: Event Recap appeared first on PyTorch.
On the evening of Sep 30, 2026, PyTorchKR and XCENA co-hosted KVCache Meetup Korea. PyTorchKR is a major AI/ML community in Korea, and XCENA is a startup that focuses on CXL (Compute Express Link). Tensormesh, the company behind LMCache, also sponsored the event. Speakers from XCENA, SK hynix, FuriosaAI, and LMCache shared their work on large language model (LLM) inference and KV Cache optimization. LLM inference runs a language model to generate output. KV Cache stores attention data for reuse during inference. Attendees at KVCache Meetup Korea 2026. Talks covered CXL memory expansion, GPU and high-bandwidth memory (HBM) bottlenecks, request scheduling on RNGD, and LMCache MP Mode. The sessions showed how hardware and serving software handle KV Cache in different ways. Serving software processes requests that run a model. The program combined technical talks with opportunities for attendees to connect. Attendees who wanted to…
Excerpt shown under the publisher's RSS terms · 144 words · full text and figures are on the publisher's site