3 ms·
Even below FP8 works: https://arxiv.org/abs/2310.10537 https://arxiv.org/abs/2310.10537 (6 bit and lower)
by buildbot 3y ago
Even below FP8 works: https://arxiv.org/abs/2310.10537 https://arxiv.org/abs/2310.10537
(6 bit and lower)
- pclmulqdq 3y agoI have seen FP4 as a proposed format, as well as batch floating point with FP8 numbers (batch floating point means n mantissas for every exponent - an old DSP trick), resulting in ~4-5 bits per number. I'm just disappointed that Google isn't taking quantization very seriously. Edit: As the commenter below points out, "block floating point" is the common name, not "batch floating point."
- buildbot 3y agoThe Microscaling paper and the following MX OCP spec is along the lines of what you call batch floating point, though I believe the original 1963(?) work on it called it "blocked" floating point. Datatypes are really tricky. Hardware designers tend to be conservative in my experience, and don't want to waste die space on things that might not be useful. Edit - The original: https://www.abebooks.com/first-edition/Rounding-Errors-Algebraic-Processes-Prentice-Hall-Series/31429549335/bd https://www.abebooks.com/first-edition/Rounding-Errors-Algeb...