Raj Uppala | Sr Director, Marketing & Partnerships
Rambus

Future of Memory and Storage - Day 3 @ 08:50

Al Inference needs a mix-and-match memory strategy

AI inference spans diverse workloads, from low‑latency chat to long‑context reasoning and large‑scale recommendations—making single, monolithic accelerator and memory designs increasingly inefficient. This talk explains how inference naturally splits into prefill and decode stages with fundamentally different bottlenecks: prefill is compute‑bound, while decode is dominated by memory bandwidth and latency. By matching memory technologies to each stage, using cost‑efficient GDDR or LPDDR for prefill and reserving premium HBM for decode, with pooled memory for KV offload, operators can significantly reduce cost per token without sacrificing latency. The session outlines emerging disaggregated architectures for AI inference workloads.

Raj Uppala, Sr Director, Marketing & Partnerships, Rambus

×

Raj Uppala / Al Inference needs a mix-and-match memory strategy

Raj Uppala, Sr Director, Marketing & Partnerships,
Rambus

Raj Uppala is the Sr. Director of Marketing & Partnerships at Rambus for the Silicon IP business unit. Prior to Rambus, Raj held several roles at Western Digital in product management, product marketing, and ecosystem partnerships, for the Hard Disk Drive (HDD) product line and a Smart Video...

CloseMore