4 ms·
FP8 or even a Frankenstein bf8 should be even better
by villgax 3y ago
FP8 or even a Frankenstein bf8 should be even better
- lostmsu 3y agoYeah, I'd expect the most value to come out of 2 bit mantissa 3 bit exponent 1 bit sign setting. Actually could probably try it on its own: a lookup table for binary ops over 6 bit arguments is just 6/8 of 4kb.
- eth-mld 3y agohttps://github.com/NVIDIA/TransformerEngine https://github.com/NVIDIA/TransformerEngine - mixed precision FP8 is already here and provides similar accuracy to FP16/BF16
- villgax 3y agoYeah and for some reason limited to only hardware support of H100. Even the cost of doing it in software is outweighed by the speed & storage gains from it