Nvidia has demonstrated that its NVFP4 4-bit floating point format — originally intended for inference — can be used for stable, large-scale training of LLMs with minimal accuracy loss, offering significant gains in compute and memory efficiency over both FP8 and BF16.
