Terabyte per second to Bit per second

Large language models have billions of parameters that must be read from memory for every inference pass. An LLM with 70 billion parameters at 16-bit precision needs 140 GB of data read per forward pass. At 3 TB/s, the H100 can perform roughly 20 inference passes per second — bandwidth directly determines tokens-per-second output.

During LLM inference each token requires reading all model weights from memory. A 70-billion-parameter model at 16-bit precision means 140 GB read per forward pass. At 30 tokens per second, that is 4.2 TB/s of memory reads — right at the limit of an H100's HBM3. This is why AI inference is "memory-bound": the GPU's compute cores sit idle waiting for data. Quantising weights to 8-bit or 4-bit halves or quarters the bandwidth demand, directly increasing tokens per second.

The NVIDIA B200 GPU with HBM3e achieves approximately 8 TB/s of memory bandwidth as of 2025. Each generation roughly doubles bandwidth — from 2 TB/s (A100) to 3.35 TB/s (H100) to 4.8 TB/s (H200) to 8 TB/s (B200). The trajectory suggests 16+ TB/s within a few years.

About 16.7 minutes. A petabyte is 1,000 terabytes, so at 1 TB/s, the math is simple division. For context, the Library of Congress contains roughly 10–20 petabytes of data. Transferring it all at 1 TB/s would take about 3–6 hours.

Yes — petabytes per second (PB/s). Experimental optical interconnects and photonic computing architectures are pushing toward PB/s-class bandwidth. Some supercomputer storage systems already aggregate into the PB/s range when all nodes operate simultaneously. It is the next frontier for AI training clusters.

A bit represents a single binary choice — 0 or 1 — which is the fundamental quantum of digital information. Every larger unit (byte, kilobit, megabit) is just a multiple of bits. You cannot meaningfully subdivide a binary digit, so bps is the floor of data rate measurement.

LoRaWAN IoT sensors, some RFID readers, and legacy serial ports (RS-232 at 300–9600 baud) still deal in raw bps ranges. Satellites communicating with deep-space probes also use very low bps — NASA's Voyager 1 transmits at about 160 bps from interstellar space.

Not exactly. Baud measures symbol changes per second, while bps measures bits per second. If each symbol encodes one bit, they are equal. But modern modems encode multiple bits per symbol — a 2400-baud modem using 16-QAM transmits 9600 bps because each symbol carries 4 bits.

Research suggests human speech carries about 39 bits per second of actual information content, regardless of language. Italian speakers talk faster but convey less information per syllable than Japanese speakers, balancing out to roughly the same bps across all studied languages.

The 56 kbps limit came from the Shannon-Hartley theorem applied to analogue phone lines. The 3.1 kHz bandwidth of a voice telephone channel, combined with its signal-to-noise ratio, creates a theoretical ceiling near 56 kbps. FCC power regulations further capped actual downstream to 53.3 kbps.