Terabyte per second to Kibibyte per second

Large language models have billions of parameters that must be read from memory for every inference pass. An LLM with 70 billion parameters at 16-bit precision needs 140 GB of data read per forward pass. At 3 TB/s, the H100 can perform roughly 20 inference passes per second — bandwidth directly determines tokens-per-second output.

During LLM inference each token requires reading all model weights from memory. A 70-billion-parameter model at 16-bit precision means 140 GB read per forward pass. At 30 tokens per second, that is 4.2 TB/s of memory reads — right at the limit of an H100's HBM3. This is why AI inference is "memory-bound": the GPU's compute cores sit idle waiting for data. Quantising weights to 8-bit or 4-bit halves or quarters the bandwidth demand, directly increasing tokens per second.

The NVIDIA B200 GPU with HBM3e achieves approximately 8 TB/s of memory bandwidth as of 2025. Each generation roughly doubles bandwidth — from 2 TB/s (A100) to 3.35 TB/s (H100) to 4.8 TB/s (H200) to 8 TB/s (B200). The trajectory suggests 16+ TB/s within a few years.

About 16.7 minutes. A petabyte is 1,000 terabytes, so at 1 TB/s, the math is simple division. For context, the Library of Congress contains roughly 10–20 petabytes of data. Transferring it all at 1 TB/s would take about 3–6 hours.

Yes — petabytes per second (PB/s). Experimental optical interconnects and photonic computing architectures are pushing toward PB/s-class bandwidth. Some supercomputer storage systems already aggregate into the PB/s range when all nodes operate simultaneously. It is the next frontier for AI training clusters.

Rsync follows IEC binary conventions because it deals with file sizes that are measured in binary units by the filesystem. Since files occupy whole filesystem blocks (typically 4 KiB), reporting transfer speed in KiB/s makes the math consistent with actual data moved on disk.

1 KiB/s (1,024 bytes/second) is 2.4% faster than 1 kB/s (1,000 bytes/second). The difference is tiny at this scale but matters when you are designing buffer sizes for embedded systems where every byte of RAM counts.

Microsoft started using binary units more consistently in Windows 10 after years of ambiguity where "KB" sometimes meant 1,000 and sometimes 1,024 bytes. The shift toward KiB follows IEC recommendations and reduces confusion, though the transition is still incomplete across all Windows tools.

A 3.5-inch floppy drive transferred data at about 31–62 KiB/s (250–500 kbps). Copying a full 1.44 MB floppy took roughly 25–50 seconds. For comparison, a modern NVMe SSD is about 100,000 times faster.

In embedded systems with tight memory constraints, confusing 1,024 with 1,000 can overflow a buffer. In network protocols, a spec written in KiB/s being implemented as kB/s means transmitting 2.4% less data than expected per second — enough to cause timing violations in real-time systems.