Terabyte per second to Gigabit per second

Large language models have billions of parameters that must be read from memory for every inference pass. An LLM with 70 billion parameters at 16-bit precision needs 140 GB of data read per forward pass. At 3 TB/s, the H100 can perform roughly 20 inference passes per second — bandwidth directly determines tokens-per-second output.

During LLM inference each token requires reading all model weights from memory. A 70-billion-parameter model at 16-bit precision means 140 GB read per forward pass. At 30 tokens per second, that is 4.2 TB/s of memory reads — right at the limit of an H100's HBM3. This is why AI inference is "memory-bound": the GPU's compute cores sit idle waiting for data. Quantising weights to 8-bit or 4-bit halves or quarters the bandwidth demand, directly increasing tokens per second.

The NVIDIA B200 GPU with HBM3e achieves approximately 8 TB/s of memory bandwidth as of 2025. Each generation roughly doubles bandwidth — from 2 TB/s (A100) to 3.35 TB/s (H100) to 4.8 TB/s (H200) to 8 TB/s (B200). The trajectory suggests 16+ TB/s within a few years.

About 16.7 minutes. A petabyte is 1,000 terabytes, so at 1 TB/s, the math is simple division. For context, the Library of Congress contains roughly 10–20 petabytes of data. Transferring it all at 1 TB/s would take about 3–6 hours.

Yes — petabytes per second (PB/s). Experimental optical interconnects and photonic computing architectures are pushing toward PB/s-class bandwidth. Some supercomputer storage systems already aggregate into the PB/s range when all nodes operate simultaneously. It is the next frontier for AI training clusters.

For most households, no. A family of four streaming 4K, gaming, and video-calling simultaneously uses about 100–150 Mbps. Gigabit becomes worthwhile if you regularly transfer large files, run a home server, or have 15+ connected devices all active at once. The real benefit is future-proofing.

Dedicated bandwidth means your 1 Gbps line is yours alone — common in business fiber (leased lines). Residential fiber is shared: a 10 Gbps trunk splits across 32–128 homes via a passive optical splitter (GPON). During peak evening hours, your "gigabit" plan might deliver 300–600 Mbps because neighbors are all streaming. This is why business fiber costs 5–10× more for the same headline speed — you are paying for a guarantee, not just capacity.

As of 2026, several ISPs offer 10 Gbps residential plans in select cities — Google Fiber, AT&T, and some European providers. South Korea and Japan have had multi-gigabit home connections since the early 2020s. The bottleneck is usually the home network equipment, not the ISP connection.

Data centers connect racks of servers with 25–100 Gbps links to handle millions of simultaneous user requests. A single popular website might serve hundreds of Gbps of traffic during peak hours. Spine-leaf network architectures aggregate these links to provide non-blocking Tbps-class switching capacity.

A traditional spinning hard drive writes at about 1–1.5 Gbps (125–180 MB/s), so it can just barely keep up with a 1 Gbps connection. An NVMe SSD at 3–7 Gbps handles it easily. If you have gigabit internet but an old HDD, your disk is the bottleneck, not your connection.