Terabyte per second to Mebibyte per second

Large language models have billions of parameters that must be read from memory for every inference pass. An LLM with 70 billion parameters at 16-bit precision needs 140 GB of data read per forward pass. At 3 TB/s, the H100 can perform roughly 20 inference passes per second — bandwidth directly determines tokens-per-second output.

During LLM inference each token requires reading all model weights from memory. A 70-billion-parameter model at 16-bit precision means 140 GB read per forward pass. At 30 tokens per second, that is 4.2 TB/s of memory reads — right at the limit of an H100's HBM3. This is why AI inference is "memory-bound": the GPU's compute cores sit idle waiting for data. Quantising weights to 8-bit or 4-bit halves or quarters the bandwidth demand, directly increasing tokens per second.

The NVIDIA B200 GPU with HBM3e achieves approximately 8 TB/s of memory bandwidth as of 2025. Each generation roughly doubles bandwidth — from 2 TB/s (A100) to 3.35 TB/s (H100) to 4.8 TB/s (H200) to 8 TB/s (B200). The trajectory suggests 16+ TB/s within a few years.

About 16.7 minutes. A petabyte is 1,000 terabytes, so at 1 TB/s, the math is simple division. For context, the Library of Congress contains roughly 10–20 petabytes of data. Transferring it all at 1 TB/s would take about 3–6 hours.

Yes — petabytes per second (PB/s). Experimental optical interconnects and photonic computing architectures are pushing toward PB/s-class bandwidth. Some supercomputer storage systems already aggregate into the PB/s range when all nodes operate simultaneously. It is the next frontier for AI training clusters.

dd uses binary units because Linux filesystems work in binary block sizes (4 KiB, etc.). Drive manufacturers use decimal MB/s because it makes speeds look about 5% higher and aligns with their decimal capacity marketing. A "550 MB/s" SSD shows roughly 524 MiB/s in dd.

Run "dd if=/dev/zero of=testfile bs=1M count=1024 oflag=direct" and it will report write speed in MiB/s. For read speed, use "dd if=testfile of=/dev/null bs=1M". The oflag=direct flag bypasses filesystem cache to measure actual disk performance.

No — 550 MiB/s is about 577 MB/s, and 550 MB/s is about 524 MiB/s. The ~5% difference means an SSD advertised at 550 MB/s will show around 524 MiB/s in Linux tools. It is not a defect or false advertising, just different unit systems measuring the same physical speed.

A RAID 0 stripe of two SATA SSDs gives roughly 1,000–1,100 MiB/s sequential reads. Four NVMe SSDs in RAID 0 can hit 12,000–14,000 MiB/s. RAID 5/6 arrays sacrifice some write speed for redundancy — expect 70–90% of raw stripe performance on writes.

Sequential reads let the drive stream data from contiguous locations, maximising throughput. Random I/O forces the controller to seek different addresses, adding latency per operation. An NVMe SSD might do 7,000 MiB/s sequential but only 50–80 MiB/s random (at 4 KiB block size), because the bottleneck shifts from bandwidth to IOPS.