2.1×
Memory saved
vs BF16, 70B weights
0.7 pp
MMLU lost
measured, not claimed [3]
8
Break-even batch
Element bitsE2M1 · 4
Block size16
Scale formatE4M3 [1]
Hopper supportnone
| Format | Bits | MMLU Δ | Hardware |
| NVFP4 | 4.25 | −0.7 | Blackwell |
| MXFP4 | 4.25 | −1.9 | Blackwell, MI355 |
| INT4 AWQ | 4.13 | −2.4 | any |
The sources disagree. NVIDIA reports −0.7 pp on MMLU [3];
an independent reproduction measured −2.1 pp on the same checkpoint [7].
The reproduction used a different calibration set, which plausibly accounts for the gap.
Verdict
Adopt NVFP4 when the constraint is memory and the batch is small; keep BF16 where
the last point of accuracy is worth twice the VRAM.
Blackwell only
Accuracy recovery is a calibration problem, not a format problem.
NVIDIA TensorRT-LLM release notes [3]
2024-03Blackwell announced with NVFP4 [1]
2025-01TensorRT-LLM ships the first kernels [3]