Terabyte per second to Kibibit per second

Large language models have billions of parameters that must be read from memory for every inference pass. An LLM with 70 billion parameters at 16-bit precision needs 140 GB of data read per forward pass. At 3 TB/s, the H100 can perform roughly 20 inference passes per second — bandwidth directly determines tokens-per-second output.

During LLM inference each token requires reading all model weights from memory. A 70-billion-parameter model at 16-bit precision means 140 GB read per forward pass. At 30 tokens per second, that is 4.2 TB/s of memory reads — right at the limit of an H100's HBM3. This is why AI inference is "memory-bound": the GPU's compute cores sit idle waiting for data. Quantising weights to 8-bit or 4-bit halves or quarters the bandwidth demand, directly increasing tokens per second.

The NVIDIA B200 GPU with HBM3e achieves approximately 8 TB/s of memory bandwidth as of 2025. Each generation roughly doubles bandwidth — from 2 TB/s (A100) to 3.35 TB/s (H100) to 4.8 TB/s (H200) to 8 TB/s (B200). The trajectory suggests 16+ TB/s within a few years.

About 16.7 minutes. A petabyte is 1,000 terabytes, so at 1 TB/s, the math is simple division. For context, the Library of Congress contains roughly 10–20 petabytes of data. Transferring it all at 1 TB/s would take about 3–6 hours.

Yes — petabytes per second (PB/s). Experimental optical interconnects and photonic computing architectures are pushing toward PB/s-class bandwidth. Some supercomputer storage systems already aggregate into the PB/s range when all nodes operate simultaneously. It is the next frontier for AI training clusters.

Because "kilo" was used to mean both 1,000 and 1,024 depending on context, causing real confusion. RAM manufacturers used 1,024 (binary) while network engineers used 1,000 (decimal). The IEC created kibi (Ki) in 1998 to unambiguously mean 1,024, leaving kilo for exactly 1,000.

Very few people outside of standards bodies and kernel developers. Linux kernel networking code sometimes uses binary units internally, and some IEC-compliant technical documents use Kibps. But consumer networking has fully standardized on decimal kilobits (kbps), making kibibits a niche pedantic distinction.

At the kibi/kilo level, only 2.4%. But the gap compounds — mebi vs mega is 4.86%, gibi vs giga is 7.37%, and tebi vs tera is 9.95%. A "1 TB" hard drive holds only 931 GiB in binary terms, which is why your new drive looks smaller than advertised in Windows.

Hard drives are built from sectors of arbitrary size, so decimal marketing (1 TB = 1,000 GB) is natural and makes drives look bigger. RAM is addressed in powers of 2 because of how binary memory chips work, so binary units (GiB) reflect actual hardware architecture. Neither side wants to change.

Almost certainly not. Networking adopted decimal (×1000) from the beginning because serial link speeds are clock-derived and have nothing to do with powers of 2. Ethernet has always been 10/100/1000 Mbps. Binary prefixes solve a storage problem that networking never had.