Deep research · High · 24 sources · 2026-08-24

NVFP4 is a memory win before it is a speed win

Four-bit floating point on Blackwell halves the weight footprint of a 70B model, but only recovers its throughput advantage above batch 8.

2.1× Memory saved vs BF16, 70B weights
0.7 pp MMLU lost measured, not claimed [3]
8 Break-even batch

What the format actually is

Element bitsE2M1 · 4
Block size16
Scale formatE4M3 [1]
Hopper supportnone
NVFP41,842 tok/s
BF16861 tok/s
FormatBitsMMLU ΔHardware
NVFP44.25−0.7Blackwell
MXFP44.25−1.9Blackwell, MI355
INT4 AWQ4.13−2.4any
The sources disagree. NVIDIA reports −0.7 pp on MMLU [3]; an independent reproduction measured −2.1 pp on the same checkpoint [7]. The reproduction used a different calibration set, which plausibly accounts for the gap.
Verdict

Adopt NVFP4 when the constraint is memory and the batch is small; keep BF16 where the last point of accuracy is worth twice the VRAM.

Blackwell only
Accuracy recovery is a calibration problem, not a format problem. NVIDIA TensorRT-LLM release notes [3]
2024-03Blackwell announced with NVFP4 [1]
2025-01TensorRT-LLM ships the first kernels [3]