It's 4 bit floating point, at twice the speed of 8 bit floating point. There's also FP6, doesn't offer faster compute than FP8 but manages to take advantage of

It's 4 bit floating point, at twice the speed of 8 bit floating point. There's also FP6, doesn't offer faster compute than FP8 but manages to take advantage of the better memory bandwith and cache use of the 6 bit format. Apparently some people are drawing connections to this paper [1] on 4 bit LLMs, which has one NVIDIA employee among its contributors 1: https://arxiv.org/pdf/2310.16836.pdf https://arxiv.org/pdf/2310.16836.pdf