4 ms·
It's two bits, DAZ and FTZ. (seems like "denormals are zero" and "flush to zero") Bfloat16 obviously can handle denormals. The encoding is possible. There woul
by burfog 7y ago
It's two bits, DAZ and FTZ. (seems like "denormals are zero" and "flush to zero")
Bfloat16 obviously can handle denormals. The encoding is possible. There would be no need to handle the issue if the encoding did not exist.
As hex, these would be denormal: 0x0001 to 0x0080, and 0x8001 to 0x8080. It's the same as plain old 32-bit IEEE, with half the bits lopped off.
ML is all about difficulties with reproducibility. I don't see a reason to get upset about denormals when a 3D-printed turtle can be confused with a rifle.
- Veedrac 7y ago> It's two bits, DAZ and FTZ. (seems like "denormals are zero" and "flush to zero") You mean the standard ones for normal floats? You certainly wouldn't want to reuse that for bfloat16s. > ML is all about difficulties with reproducibility. I don't see a reason to get upset about denormals when a 3D-printed turtle can be confused with a rifle. These are different things, despite the similarity in terminology.