CXL memory pooling is the practice of attaching DRAM to a server through PCIe lanes rather than DDR channels, so that capacity can be added, removed and re-assigned without rebuilding the machine. The protocol — Compute Express Link, an open standard built on the PCIe electrical layer — gives a processor byte-addressable, cache-coherent load/store access to memory sitting on a card. The operating system sees that memory as an extra NUMA node with no CPU behind it. Nothing about the media changes: a CXL module is filled with the same DDR5 that goes into a DIMM. What changes is the path to it, and therefore three things a buyer cares about — how much you can fit, how fast you can reach it, and when you are allowed to change your mind.

What is the difference between CXL pooling and CXL sharing?

This is the distinction that most vendor decks blur, and it decides what your scheduler can do. The CXL Consortium's own timeline is precise about it. CXL 2.0, released in November 2020 on the PCIe 5.0 physical layer, introduced switching and memory pooling: devices can be partitioned into Logical Devices, each assigned to one host at a time, with a fabric manager re-assigning capacity as demand moves. Practitioners describe the consequence bluntly — pooling means one owner at a time. Host A cannot read the region while Host B is using it; what you get is a fast, reboot-free way to move memory between machines. CXL 3.0, released in August 2022, moved to PCIe 6.0 (64 GT/s, double the link rate), added multi-level switching and fabric capabilities, and introduced true memory sharing: multiple hosts holding hardware-coherent copies of the same region simultaneously. CXL 3.1 (November 2023) extended sharing between accelerators without going through a host and added port-based routing; CXL 3.2 (December 2024) added the CXL Hot-Page Monitoring Unit, which lets software find the pages worth keeping in the fast tier; CXL 4.0 (November 2025) doubled the link rate again to 128 GT/s.

The practical read: if a vendor says "shared memory pool" while shipping CXL 2.0 silicon, they mean re-assignable, not simultaneously shared. That difference is not semantic — a re-assignable pool needs a policy that decides who gets the memory and when, which means downtime windows or quiesce points, whereas a shared pool needs coherence hardware and a fabric manager that can enforce it. Budget for the first, and treat the second as a 3.x roadmap item until you see the switch on the bill of materials.

What does one CXL module actually add?

Use the parts that are in volume. Samsung's CMM-D line lists two generations: the MD220 at CXL 2.0 / PCIe 5.0 with 256 GB or 128 GB of DDR5 at up to 6.4 Gbps in an E3.S 2T form factor, and the MD310 at CXL 3.2 / PCIe 6.0 with 256 GB and up to 72 GB/s. Intel's Xeon 6 product brief states up to 64 lanes of CXL 2.0 at 32 GT/s per lane, supporting Type 1, 2 and 3 devices, alongside up to 12 channels of DDR5-6400 — and AMD's EPYC 9005 generation documents CXL 2.0 support as well. Put those side by side and the shape of the tier appears.

TierCapacity per unitBandwidthLoad latencyChangeable after build?
Local DDR5 (Xeon 6, 12 ch × DDR5-6400)Up to 12 DIMMs (2 DPC drops the data rate)≈614 GB/s per socket (our arithmetic: 12 × 51.2 GB/s)97–112 ns measuredNo — population fixed at boot
CXL 2.0 Type 3 module, x8 (Samsung MD220)256 GB (128 GB option), E3.S 2T≈32 GB/s per direction; vendor names 36 GB/s device performance (2R1W)219–255 ns measuredYes — hot-plug, re-assignable
8 × CXL 2.0 modules (all 64 lanes)2 TB≈250 GB/s theoretical aggregateSame per access, plus fabric queueingYes
NVMe (for scale)TB per drive6–14 GB/s per drive≈10,000 nsYes

Two notes on the table, because both numbers look wrong at first glance. The 36 GB/s in Samsung's datasheet exceeds the ~32 GB/s a PCIe 5.0 x8 link can carry in one direction; the figure is a device-level 2-read/1-write mix, and split that way (roughly 24 GB/s of reads and 12 GB/s of writes) it sits comfortably inside the link budget — the same arithmetic makes the 72 GB/s CXL 3.2 number consistent with a x8 PCIe 6.0 link. And the 250 GB/s aggregate is theoretical: it assumes you spend every one of the socket's 64 CXL lanes on memory, which no real server does, because NICs, NVMe and GPU host links draw from the same budget.

Why is the bandwidth so much worse than the capacity?

It is not, per pin — which is why the usual explanation is wrong. A DDR5 channel costs roughly 300 package pins for about 51 GB/s, while a x16 CXL link costs roughly 80 pins for about 63 GB/s: serial lanes are more bandwidth-efficient per pin, not less. The reason CXL feels slow is where the bandwidth comes from and how much of it a single device gets. DDR bandwidth arrives with the socket; CXL bandwidth is carved out of the PCIe lane budget the rest of the machine also needs. And a single x8 device gets about 32 GB/s, or 5% of the socket's DDR bandwidth, so any workload that streams through one module sequentially — a large memcpy, a sequential scan, a model load — runs at one-twentieth of the speed it expects.

Latency is the more interesting constraint because it is now well characterised rather than argued about. Independent measurements on production CXL 2.0 Type 3 devices converge: Zhong et al. measured 97 ns for on-board DDR5 and 219 ns for a CXL device on Intel Sapphire Rapids, with per-core bandwidth topping out at 48 GB/s on a x16 PCIe 5.0 link; Unal et al. (HotOS 2025) report 112 ns versus 237 ns with bandwidth just under 50 GB/s; Weisgut et al. recorded a median 255 ns load latency on AMD Genoa servers with 46 GB/s of sustained copy per device, scaling to 92 GB/s across four cards. A 2026 survey by Sharma et al. explains the composition: two full traversals of the CXL port stack at roughly 21–25 ns each plus ~15 ns of flight time comes to about 57 ns of protocol latency per access, with queueing added under load — and about 6–9% of link bandwidth disappears into flit headers, CRC/FEC and flow control. The rule of thumb that falls out of three independent measurements: CXL memory costs you roughly double the latency of local DRAM, and lands in the same range as reaching the other socket of a two-socket server.

That last sentence is the useful one for procurement. Software that already survives two-socket NUMA — most databases, most JVMs, most inference servers with a memory policy — can survive CXL. Software written on the assumption that all memory is equidistant will not, and no amount of bandwidth fixes a stall that happens once per cache line.

What does CXL cost you per token?

Here is where the tier earns or loses its place, and it needs arithmetic rather than adjectives. Take a 70B-class model with grouped-query attention: 80 layers, 8 KV heads, head dimension 128, FP16. Its KV cache is 2 × 80 × 8 × 128 × 2 bytes = 327,680 bytes, or 320 KB per token. An 8K-token conversation therefore holds 2.62 GB of cache, and a 2 TB CXL pool holds about 780 of those sessions. Capacity is not the problem — that is the whole point of the tier.

Throughput is. Decoding one token requires reading that sequence's KV cache once. At 32 GB/s from a single module, 2.62 GB takes 82 ms, which caps a single 8K stream at roughly 12 tokens per second before any model compute is counted. Aggregate eight modules and the ceiling rises to about 95 tokens per second; put the same cache in an H100's 3.35 TB/s of HBM and the theoretical ceiling is over 1,280. That is why KV cache lives in HBM and why every serious serving stack fights to keep it there. The conclusion is not that CXL is useless for inference — it is that CXL is for the KV cache you are not decoding right now: the prefix caches that get re-used across requests, the sessions that are idle between turns, the checkpoints of a conversation that may never resume.

Samsung's own whitepaper tests exactly that pattern and reports the optimistic version: using a CXL switch-based memory pool as the backend for LMCache, a KV-cache offload layer, it measured performance comparable to conventional DRAM, at about 92% of DRAM performance in a multi-GPU tensor-parallel setup. Read the qualifier carefully — that is an end-to-end service-quality number for a cache that mostly hits, not a per-token streaming number, and it is the vendor's own benchmark on the vendor's own hardware. It does not contradict the 12-tokens-per-second arithmetic above; it describes a different workload. Both statements can be true at once, and the difference between them is the entire discipline of memory tiering: what fraction of your accesses land in the fast tier. Research on that policy question is unusually consistent, per a 2026 survey of tiered-memory work: transparent page placement (TPP) keeps a tiered system within about 1% of an all-DRAM baseline, and Intel's hardware-managed Flat Memory Mode with the Memstrata allocator kept 82% of 115 Azure traces within 5% of local DRAM while cutting the worst slowdown from 34% to below 6%. Both results depend on a migration policy doing real work, not on the memory being fast.

Where does pooling actually pay?

When the alternative is a new server. DIMM population is fixed at build time: once the 12 channels are full, the only ways to add capacity are a larger DIMM (if the vendor qualifies it), a second socket, or a new machine. CXL capacity is hot-pluggable and re-assignable, which turns a capital decision into an operating one. That optionality is the product, and it is worth paying a bandwidth tax for in workloads whose memory footprint is spiky rather than sustained — analytics jobs, build fleets, virtualisation hosts with a few memory-hungry tenants, inference fleets with a long tail of idle sessions.

When the software already tiers. Intel's Xeon 6 brief describes Flat Memory Mode as "a hardware-controlled tiering mode that is completely independent of O/S version & data-tiering capabilities," and explicitly frames it as a TCO tool "when using lower-cost memory, such as DDR4 with CXL 2.0" — the memory controller, not the kernel, decides what sits where. That is the least glamorous and most deployable version of CXL in 2026: you do not need a new scheduler, you need a BIOS setting and a module population rule. The kernel-side alternative — NUMA balancing, DAMON, TPP-style demotion — gets better numbers but requires the policy work described above.

When lanes are cheaper than slots. GPUs are hungrier for PCIe lanes than for DIMM slots on an inference node, and a CXL switch placed behind a PCIe switch lets a memory pool be attached without consuming CPU-attached PCIe slots — the configuration Samsung's whitepaper calls out, because it leaves the host's own slots free for accelerators. If your constraint is "no more DIMM slots and no more rack units," this is the argument that wins.

Not when you are hoping for cheaper gigabytes. We could find no vendor publishing a lower cost per gigabyte for a CXL module than for an RDIMM of the same capacity, and the bill of materials argues against it: a CXL controller, a retimer and a PCB are added cost on top of identical DRAM. CXL memory also inherits the DRAM shortage rather than escaping it — the modules are filled with the same DDR5 whose contract prices rose 93–98% in a single quarter of 2026. Anyone selling CXL as a way around the memory bill is selling the wrong product. What it actually sells is the ability to stop over-provisioning: capacity that follows demand instead of capacity that is welded to a chassis for five years.

What breaks first?

The fabric manager, not the silicon. Pooling requires software that knows which host owns which Logical Device, that can quiesce and re-assign without corrupting a running workload, and that fails safely when a link drops. This layer is the least standardised part of the stack and the most likely to be the reason a pilot stays a pilot. Then the switch supply. CXL 2.0 switches only began shipping in 2024, and CXL 3.x fabric switches are rarer still; a design that needs multi-level switching in 2026 is a design waiting on silicon. Then the GPU's reach. CXL memory is CPU-side. A GPU does not magically gain a terabyte of framebuffer; it reaches pooled memory over PCIe — or, more often, the CPU stages data for it. Every "CXL expands GPU memory" claim should be read as "CXL expands the buffer the CPU can stage from," which is a real benefit for offload layers like LMCache and a non-benefit for anything that assumed unified addressing.

And the strategic risk deserves a sentence, because it is the one buyers underweight: HBM capacity per accelerator keeps rising. Every generation that doubles accelerator memory shrinks the population of workloads that need an external tier, and the tier's economics get worse as the fast tier gets bigger. CXL pooling is therefore a bet on spiky, heterogeneous, long-tail memory demand — the demand a hyperscaler with mixed tenants actually has — rather than on frontier training, where the answer has been and remains "buy more HBM."

What should a buyer ask before adding a CXL tier?

1. Which CXL generation is on the bill of materials, and does the word "shared" mean what you think? CXL 2.0 pools; CXL 3.0 shares. If the answer is pooling, ask who or what re-assigns capacity and how long that takes. 2. What is the per-device bandwidth, and how many devices will you actually populate? One module is 5% of socket bandwidth; eight are about 40%. Buying one big module for a bandwidth problem is the classic misconfiguration. 3. What fraction of accesses will hit the fast tier? Ask for the vendor's hit-rate curve, not their latency slide — a tiered system's performance is a hit-rate story with a latency footnote. 4. Does the workload's tail latency live in memory at all? If p99 is set by GPU compute or by the network, an extra memory tier changes concurrency and cost, not speed, and that is a perfectly good reason to buy it — as long as nobody promised a latency win.

Our view, stated as a view: CXL pooling in 2026 is a capacity instrument with a latency tax, and the correct way to price it is not dollars per gigabyte but dollars per gigabyte of capacity you would otherwise have bought as a whole server. On that basis it wins in mixed-tenant and long-tail inference fleets, loses in single-workload training clusters, and is roughly neutral anywhere the memory sits idle most of the day — which, given what we know about utilisation rates across this industry, is a larger category than the vendors admit.