Samsung’s LPDDR5X-PIM: Revolutionizing AI Inference at the Edge

At the Hot Chips 2026 conference, Samsung Semiconductor unveiled a transformative evolution in memory architecture: the industry’s first LPDDR5X-PIM (Processing-in-Memory). By integrating logic directly into the memory fabric, Samsung is addressing the most critical bottleneck in modern artificial intelligence—the "memory wall." This technology promises to redefine how AI models run on everything from power-constrained mobile devices to enterprise-grade accelerators, effectively turning passive storage into an active participant in computation.

The Core Concept: Moving Logic to Data

For decades, the von Neumann architecture has dominated computing, requiring data to travel constantly between the processor (CPU/GPU) and the memory (DRAM). In the era of massive AI models like Llama 3.1, this "data movement" has become the primary source of latency and power consumption.

Samsung’s PIM technology fundamentally challenges this status quo. By placing specialized logic units directly alongside DRAM cells, the memory can execute basic arithmetic operations—such as Multiply-Accumulate (MAC) operations—locally. This eliminates the need to shuttle vast quantities of data across the system bus.

During its presentation at Hot Chips 2026, Samsung demonstrated how this approach enables LPDDR5X to perform inference tasks up to 2.28x faster than standard, non-PIM counterparts. Furthermore, when measured in terms of throughput, the technology yielded a staggering 3.01x increase in tokens per second (TPS) during Llama 3.1 (8B parameter) testing.

A Chronology of Innovation

Samsung’s journey toward PIM began long before this week’s announcement. The company first signaled its intent in 2021, focusing initially on High Bandwidth Memory (HBM). By modifying AMD’s MI100 accelerators with HBM-PIM, Samsung proved that in-memory processing could provide immediate performance dividends.

• 2021: Initial public demonstrations of HBM-PIM integration.
• 2023: Samsung formally introduced the concept of LPDDR5X-PIM, setting the stage for mobile-first AI integration.
• 2026 (Hot Chips): The debut of a validated, functional LPDDR5X-PIM product, alongside the announcement that the company is actively working with JEDEC to standardize LPDDR6X-PIM specifications within the year.

This progression marks a shift from experimental research to commercially viable hardware, signaling that Samsung intends to bring the performance benefits of PIM to the consumer market, not just the high-end server farm.

Technical Deep-Dive: How LPDDR5X-PIM Works

The architecture of LPDDR5X-PIM is a masterclass in spatial efficiency. Unlike earlier HBM-PIM implementations, where Samsung had to sacrifice memory banks to accommodate the added logic, the LPDDR5X-PIM design integrates a PIM unit for every single memory bank.

The Mechanism

1. Dual-Mode Operation: The memory operates in two distinct modes: standard DRAM (Single-Bank) mode for traditional tasks, and PIM (Multi-Bank) mode for AI acceleration. The system controller manages this transition, using standard read/write commands to toggle functionality.
2. Address Align Mode (AAM): One of the biggest challenges in memory design is reordering instructions. Samsung’s proprietary Address Align Mode resolves this by mapping DRAM addresses directly to MAC instructions. This ensures that the Vector Register File (VRF) and Source Register File (SRF) are synchronized with the Request Address (RA) and Column Address (CA), bypassing the need for a complex instruction register file.
3. Data Flow: In a typical MAC operation, 512 bytes of FP8 activation data are broken into 16, 256-bit packets and stored across the banks. The PIM logic reads the weight data locally, performs the multiplication, and writes the result back into the VRF. Finally, the host reads the output, completing the cycle without the data ever leaving the memory chip’s immediate vicinity.

Performance Metrics and Benchmarks

The preliminary benchmarks presented by Samsung at Hot Chips are significant. Using an edge AI accelerator—potentially a precursor to the rumored Gaia SoC—Samsung tested the memory against a standard 8-billion parameter Llama 3.1 model.

The results were stark:

• Model Run Time: 2.28x reduction in latency.
• Throughput: 3.01x increase in tokens per second.
• Bandwidth: While standard LPDDR5X-9600 offers a peak bandwidth of 76.8 GB/s, the PIM-enabled version effectively elevates this to 614 GB/s in local computation scenarios.

While some attendees at the conference noted discrepancies in the model output accuracy during these early tests, Samsung confirmed that they are currently in the optimization phase. They remain confident that these performance gains will remain consistent as the software stack matures.

The Power Paradox: Why PIM Saves Energy

A common concern with adding logic to memory is the potential for increased power consumption. However, Samsung argues that LPDDR5X-PIM is actually more power-efficient than conventional DRAM.

"We’re not having a significant power increase," stated Samsung’s Karam Hwang. The logic here is straightforward: while the actual calculation inside the memory chip does draw a "burst" of power, it is dwarfed by the energy required to move data across the high-speed bus to the main processor. By reducing the number of times data must travel, the total system energy draw is significantly lowered, making it an ideal candidate for battery-operated devices like smartphones and laptops.

Strategic Implications for the Industry

The significance of this development cannot be overstated. Micron recently warned that the "silicon gap" between HBM and DDR5 is widening, with HBM wafers becoming increasingly expensive and difficult to source. By bringing PIM to the LPDDR5X standard, Samsung is democratizing high-performance AI.

Lowering the Barrier to Entry

HBM is essentially a luxury for enterprise AI. By contrast, LPDDR5X is ubiquitous. Its use in modules like CAMM2 (used by Nvidia’s Vera CPU) and its integration into Intel’s "Crescent Island" accelerators suggests that PIM is intended for a massive, scalable market.

The Future of Edge AI

Mobile devices and edge gateways are currently limited by their reliance on the host CPU/GPU for AI inference. With LPDDR5X-PIM, a standard smartphone could, in theory, run large language models locally with significantly less battery drain and much higher speed. This is the "holy grail" for AI developers who want to maintain privacy and responsiveness by keeping computation on-device.

Standardization and Competition

Samsung’s push to have LPDDR6X-PIM standardized by JEDEC this year indicates an aggressive roadmap. They are not merely launching a product; they are attempting to set the standard for the next decade of memory architecture. As other memory manufacturers scramble to catch up, the industry is entering a new phase where the line between "memory" and "processor" is permanently blurred.

Conclusion: A New Era for Computing

Samsung’s presentation at Hot Chips 2026 serves as a definitive roadmap for the future of hardware. By tackling the data movement bottleneck at the source, the company has provided a viable, power-efficient, and high-performance solution to the most pressing problem in the AI era.

While the technology is currently in its validation phase, the integration of PIM into LPDDR5X is a clear signal that the future of AI will not just be faster—it will be smarter, smaller, and more efficient. As these modules reach the hands of developers and hardware integrators, we can expect a rapid shift in how AI models are deployed across the entire computing ecosystem, from the palm of our hands to the massive server clusters driving the global economy.