Haerang Choi | Team Leader
SK hynix
Haerang Choi received a Ph.D. in Computer Science and Engineering from Seoul National University in 2021. Since then he has been involved in the Processing-in-Memory (PIM) development project at SK hynix. Currently serving as Team leader of the AI Memory Solution Hardware Team, he leads the developement of memory-centric AI accelerator technologies.
Appearances:
Future of Memory and Storage - Day 3 @ 13:05
PIM/PNM-based Attention Disaggregation System
We are developing technologies to eliminate the memory bottleneck in AI inference on AI‑DC infrastructure. The problem is that even when a large remote memory pool solves storage and cost issues, the data must still be moved to local GPUs, leaving a persistent bottleneck. Demand for remote‑transfer performance keeps rising.In practice, much of AI‑serving cost and latency comes from the GPU‑memory‑network path rather than from computation. In large, highly concurrent services, remote data movement becomes the primary performance and cost bottleneck.Our solution is a new memory system that fundamentally cuts data movement. By computing where data resides and transmitting only the results, we dramatically reduce transfers using Processing‑in‑Memory (PIM) and Processing‑Near‑Memory (PNM). We have built a prototype large‑language‑model (LLM) inference system on PIM/PNM and are expanding it into a next‑generation memory platform. We plan to share our early results.
Created by
Terrapinn is proud to be a member of isla.
Working together to build sustainable events