Tebibyte per second to Byte per second

AMD's MI300X stacks 8 HBM3 memory modules and multiple compute chiplets on a single package using advanced 2.5D packaging with silicon interposers. The short physical distance between compute and memory dies — millimeters instead of centimeters — dramatically reduces signal latency and power per bit. This allows a 5.3 TB/s aggregate bandwidth that would be physically impossible with traditional socketed memory. The trend toward chiplet packaging is how the industry keeps scaling bandwidth despite hitting limits in single-die manufacturing.

Significantly. When provisioning an AI training cluster with hundreds of GPUs, a 10% bandwidth miscalculation cascades through the entire system design — buffer sizes, interconnect capacity, cooling, and power. Getting the units wrong could mean the difference between a training run finishing in 30 days vs 33 days.

Training large language models (100B+ parameters), molecular dynamics simulations, weather modeling, and fluid dynamics at scale. These workloads move enormous matrices through memory billions of times. The TiB/s memory bandwidth of modern GPUs is what makes training models like GPT-4 possible in months rather than decades.

Memory bandwidth dwarfs network bandwidth. Each H100 GPU has 3.35 TiB/s of internal memory bandwidth but connects to the network at only 0.05 TiB/s (400 Gbps InfiniBand). This 60:1 ratio is why AI chip designers obsess over keeping computations local to each GPU and minimising network communication.

Not in the same way. Quantum computers process information through qubits that exist in superposition, so they do not shuttle classical data around at TiB/s. However, the classical control systems that manage quantum processors and process measurement results do need high bandwidth — current quantum-classical interfaces operate at modest Gbps rates.

Files are stored in bytes because CPUs address memory in byte-sized (8-bit) chunks — the smallest unit a program can read or write. Networks measure in bits because physical signals on a wire or fiber are serial: one bit at a time, clocked at a specific frequency. A 1 GHz signal produces 1 Gbps, not 1 GBps. The two worlds evolved independently and neither adopted the other's convention, leaving users to divide by 8 forever.

In modern computing, yes — a byte is universally 8 bits. Historically, some architectures used 6, 7, or 9-bit bytes, which is why the unambiguous term "octet" exists in networking standards. But for all practical bandwidth conversions today, 1 byte = 8 bits.

Network protocols add overhead — TCP headers, encryption (TLS), error correction, and packet framing all consume bandwidth without contributing to file data. A 100 Mbps connection might deliver 11 MB/s instead of the theoretical 12.5 MB/s because 10–15% goes to protocol overhead.

USB 3.0 has a theoretical maximum of 625 MB/s (5 Gbps ÷ 8), but real-world sustained transfers hit 300–400 MB/s due to protocol overhead and controller limitations. USB 3.2 Gen 2 doubles this to about 700–900 MB/s in practice.

The bit came first, coined by Claude Shannon in 1948. The byte was introduced at IBM in the mid-1950s by Werner Buchholz to describe the smallest addressable group of bits in the IBM Stretch computer. Originally it could be any size; the 8-bit byte became standard with the IBM System/360 in 1964.