The memory wall is the growing performance gap between the processing speed of CPUs and GPUs and the rate at which memory can supply them with data. While compute power has surged dramatically over the past decade, memory bandwidth has not kept pace — creating a bottleneck that starves processors of the data they need to run efficiently.

For enterprise AI, this bottleneck is acutely damaging. AI inference workloads are estimated to be 70% memory-bound and only 30% compute-bound. Large context windows, high user concurrency, and growing KV cache demands all intensify pressure on memory systems. The result: slower response times, reduced throughput, inflated infrastructure costs, and underutilized GPU clusters — even when powered by the latest hardware like NVIDIA H100 or B300.

As organizations move from AI pilots to full production deployments, the challenge only compounds. Scalability limitations force organizations to procure additional hardware and build increasingly complex infrastructure just to keep inference performance stable under load.

Compute Express Link (CXL) is an industry-open standard protocol that fundamentally redefines how servers manage memory and compute resources. By enabling high-speed, low-latency connections between CPUs or GPUs and memory over PCIe infrastructure, CXL eliminates traditional data processing bottlenecks and unlocks new levels of scalability for AI-heavy workloads.

Penguin Solutions' new family of CXL-based Add-In Cards (AICs) — available in 4-DIMM and 8-DIMM configurations — are the industry's first high-density DIMM AICs to adopt the CXL protocol. Supporting industry-standard DDR5 DIMMs, they allow data center architects to add up to 4TB of memory per server quickly using a familiar, easy-to-deploy form factor. Servers can reach up to 1TB of memory per CPU using cost-effective 64GB RDIMMs — without requiring costly infrastructure overhauls.

CXL is also more pin-efficient than conventional DIMM-based parallel bus approaches, meaning more memory can be added without running into the physical pin limitations of CPUs. This opens the door to scalable, future-proof infrastructure that can grow alongside increasingly demanding AI workloads.

Penguin Solutions is hosting a dedicated webinar — Enterprise-Scale AI Inference is Memory Bound: How to Overcome the Memory Wall — on June 9, 2026, from 5–6 PM BST, in partnership with Technology Magazine. The session is designed for IT leaders, infrastructure architects, and platform engineers seeking practical guidance to scale generative AI inference workloads without sacrificing budget, performance, or long-term architectural freedom.

The webinar will be led by Andy Mills, VP of Advanced Product Development at Penguin Solutions. A pioneer in next-generation hardware acceleration and enterprise computing, Andy specialises in architecting high-performance infrastructure solutions that overcome complex engineering constraints, with a particular focus on massive, data-heavy AI workloads.

Attendees will learn how disaggregated memory architecture can eliminate operational silos, bridge the gap between compute power and memory capacity, and transform AI infrastructure from an expensive cost center into a highly scalable, profitable engine for long-term growth.

Key Takeaways

• →AI inference workloads are 70% memory-bound, meaning memory architecture — not compute power — is the primary constraint on enterprise AI performance and scalability.
• →The memory wall — caused by processors outpacing memory bandwidth — creates inference latency, reduced throughput, and inflated infrastructure costs for organizations deploying large language models at scale.
• →CXL technology provides a high-speed, low-latency interconnect standard that enables memory expansion of up to 4TB per server and 1TB per CPU, bypassing traditional DIMM pin-count limitations.
• →Penguin Solutions launched the industry's first production-ready CXL-based KV Cache Server — MemoryAI™ — in March 2026, delivering lower inference latency, higher throughput, and improved GPU cluster efficiency.
• →Penguin Solutions co-developed the CXL standard alongside Alibaba, Cisco, Dell EMC, Google, HPE, Intel, and Microsoft, bringing 30 years of advanced memory engineering to the AI infrastructure challenge.
• →IT leaders, infrastructure architects, and platform engineers can learn how to overcome the memory wall at Penguin Solutions' free webinar on June 9, 2026, hosted in partnership with Technology Magazine.