Tebibyte per second to Gigabyte per second

AMD's MI300X stacks 8 HBM3 memory modules and multiple compute chiplets on a single package using advanced 2.5D packaging with silicon interposers. The short physical distance between compute and memory dies — millimeters instead of centimeters — dramatically reduces signal latency and power per bit. This allows a 5.3 TB/s aggregate bandwidth that would be physically impossible with traditional socketed memory. The trend toward chiplet packaging is how the industry keeps scaling bandwidth despite hitting limits in single-die manufacturing.

Significantly. When provisioning an AI training cluster with hundreds of GPUs, a 10% bandwidth miscalculation cascades through the entire system design — buffer sizes, interconnect capacity, cooling, and power. Getting the units wrong could mean the difference between a training run finishing in 30 days vs 33 days.

Training large language models (100B+ parameters), molecular dynamics simulations, weather modeling, and fluid dynamics at scale. These workloads move enormous matrices through memory billions of times. The TiB/s memory bandwidth of modern GPUs is what makes training models like GPT-4 possible in months rather than decades.

Memory bandwidth dwarfs network bandwidth. Each H100 GPU has 3.35 TiB/s of internal memory bandwidth but connects to the network at only 0.05 TiB/s (400 Gbps InfiniBand). This 60:1 ratio is why AI chip designers obsess over keeping computations local to each GPU and minimising network communication.

Not in the same way. Quantum computers process information through qubits that exist in superposition, so they do not shuttle classical data around at TiB/s. However, the classical control systems that manage quantum processors and process measurement results do need high bandwidth — current quantum-classical interfaces operate at modest Gbps rates.

CPUs constantly shuttle data between RAM and their caches. DDR5-6000 provides about 96 GB/s of bandwidth in dual-channel mode. In games, insufficient RAM bandwidth causes frame drops during complex scenes. In productivity tasks like video encoding, it directly limits how fast the CPU can process data.

Thunderbolt 4 runs at 40 Gbps, which is 5 GB/s. Thunderbolt 5, released in 2024, doubles this to 80 Gbps (10 GB/s) with a burst mode up to 120 Gbps (15 GB/s). This is fast enough to run an external NVMe SSD at near-internal speeds.

Both, depending on generation. A PCIe 3.0 ×4 interface caps at ~3.5 GB/s, bottlenecking modern NAND. PCIe 4.0 ×4 raises this to ~7 GB/s, and PCIe 5.0 ×4 to ~14 GB/s. The drive's NAND flash and controller also have limits — the fastest SSDs and the fastest interfaces are in a constant leapfrog.

GPUs use wide memory buses (256–384 bits) with very fast HBM or GDDR6X memory running at high clock speeds. An RTX 4090 has a 384-bit bus with GDDR6X at 21 Gbps per pin, totalling 1,008 GB/s. HBM3 in data center GPUs achieves 3,000+ GB/s through stacked memory with 4096-bit buses.

At multi-GB/s rates, CPU processing speed, software efficiency, and thermal throttling become bottlenecks. A 14 GB/s PCIe 5.0 SSD can deliver data faster than most applications can consume it. Decompression, parsing, and memory allocation in software often cannot keep up with raw storage bandwidth.