Terabyte per second to Mebibit per second

Large language models have billions of parameters that must be read from memory for every inference pass. An LLM with 70 billion parameters at 16-bit precision needs 140 GB of data read per forward pass. At 3 TB/s, the H100 can perform roughly 20 inference passes per second — bandwidth directly determines tokens-per-second output.

During LLM inference each token requires reading all model weights from memory. A 70-billion-parameter model at 16-bit precision means 140 GB read per forward pass. At 30 tokens per second, that is 4.2 TB/s of memory reads — right at the limit of an H100's HBM3. This is why AI inference is "memory-bound": the GPU's compute cores sit idle waiting for data. Quantising weights to 8-bit or 4-bit halves or quarters the bandwidth demand, directly increasing tokens per second.

The NVIDIA B200 GPU with HBM3e achieves approximately 8 TB/s of memory bandwidth as of 2025. Each generation roughly doubles bandwidth — from 2 TB/s (A100) to 3.35 TB/s (H100) to 4.8 TB/s (H200) to 8 TB/s (B200). The trajectory suggests 16+ TB/s within a few years.

About 16.7 minutes. A petabyte is 1,000 terabytes, so at 1 TB/s, the math is simple division. For context, the Library of Congress contains roughly 10–20 petabytes of data. Transferring it all at 1 TB/s would take about 3–6 hours.

Yes — petabytes per second (PB/s). Experimental optical interconnects and photonic computing architectures are pushing toward PB/s-class bandwidth. Some supercomputer storage systems already aggregate into the PB/s range when all nodes operate simultaneously. It is the next frontier for AI training clusters.

Mainly in Linux system tools, IEC-compliant technical specifications, and some enterprise storage documentation. The iperf3 network testing tool can report in Mibps if configured to use binary units. Most consumer-facing software and ISPs use megabits exclusively.

Multiply by 1.048576. So 100 Mibps = 104.86 Mbps. To go from Mbps to Mibps, divide by 1.048576. At small values the difference is negligible, but at gigabit scales it can mean a meaningful amount of data.

Linux kernel developers historically followed IEC recommendations to use binary prefixes where applicable. Some tools like dd and rsync default to binary (MiB/s) for disk operations. However, network-facing tools like ethtool and ip still use decimal Mbps because that is what the hardware reports.

For casual use, no. For capacity planning and SLA compliance, yes. If a contract guarantees 100 Mibps and the provider measures in Mbps, the customer might get 100 Mbps (only 95.4 Mibps) and technically be short-changed. Data center SLAs should specify which unit system applies.

No — ISPs legitimately use decimal megabits because Ethernet and fiber standards are decimal. A "100 Mbps" plan genuinely delivers 100,000,000 bits per second. The confusion arises only when comparing with binary-unit tools. ISPs are not hiding anything; the two systems just coexist awkwardly.