An Nvidia Vera Rubin rack holds 72 graphics chips. Each one carries eight stacks of memory. That's 576 stacks in a single rack. Each stack is reported to be a tower of 12 thin memory chips.

Do the math and one rack holds roughly 6,900 memory chips before anyone counts a processor. Multiply that by the racks being ordered, and you have the reason AI is running short of HBM4.

What is HBM4 memory?

HBM4 is the newest type of high bandwidth memory. It stacks memory chips into a short tower and sets that tower beside the AI processor.

Regular computer memory, called DRAM, sits on separate sticks a few inches from the processor. Data travels to it over a fairly narrow path. HBM puts the memory in the same package as the processor and connects it through thousands of tiny wires. Picture a 2,048-lane highway next to a two-lane road.

The "4" means fourth generation. The standards group JEDEC finalized it in April 2025. According to Tom's Hardware's summary of the standard, HBM4 moves data at up to 8 gigabits per second per lane across a 2,048-bit interface. That adds up to about 2 terabytes per second from each stack. A stack can be 4 to 16 chips tall and hold up to 64 GB.

HBM4 vs HBM3E

HBM4 doubles the width of the connection compared with earlier generations. SK hynix said its HBM4 uses 2,048 input and output terminals, doubling bandwidth, with power efficiency more than 40% better than the prior generation. The new standard also works with existing HBM3 memory controllers, so chip designers don't have to start over.

Why do AI chips need so much of it?

An AI model has to sit in memory while it runs. Bigger models need more memory, and faster chips need memory that can keep them fed.

A graphics chip that waits on memory is an expensive chip doing nothing. Here's how much HBM4 the two leading rack designs for 2026 carry:

HBM4 in the two leading AI rack designs
Rack GPUs per rack HBM4 per GPU HBM4 per rack
Nvidia Vera Rubin NVL72 72 288 GB in 8 stacks 20.7 TB
AMD Helios 72 Up to 432 GB 31 TB

Sources: Tom's Hardware on Nvidia's Vera Rubin launch and AMD's Helios announcement. These are company specifications, not measured shipments.

The stack count is our arithmetic. Seventy-two chips times eight stacks is 576. The Korea Herald reported that Nvidia was expected to use eight of SK hynix's 12-layer HBM4 stacks in each Rubin chip. If every stack is 12 layers, that's 6,912 memory chips per rack. AMD hasn't said how many stacks Helios uses, so we haven't counted them.

Why is HBM4 in short supply?

Each AI rack needs hundreds of stacks, and each stack uses far more silicon than ordinary memory. Only a handful of companies can make them.

The makers are SK hynix, Samsung and Micron. HBM starts from the same kind of memory wafer as regular DRAM, so every wafer is a choice between the two.

That choice is lopsided. A Micron executive spoke at the Hot Chips conference in August. He said HBM takes roughly three times the wafer area of DDR5 for the same amount of memory. He said the gap is "definitely not getting better" with newer generations.

Demand is already spoken for. SK hynix sold out its 2026 memory production. In July, its CEO said 2027 will be the "worst year" for the memory shortage and that the crunch could last until 2030.

That pressure is changing deals. Our earlier piece on AMD's Helios memory needs covers reports that Samsung wanted AMD's chip manufacturing orders in return for more HBM4.