4 ms·
I wonder how much progress (if any) is being done on floating point formats other than IEEE floats; on serious adoption in hardware in particular. Stuff like po
by andrepd 1y ago
I wonder how much progress (if any) is being done on floating point formats other than IEEE floats; on serious adoption in hardware in particular. Stuff like posits [1] for instance look very promising.
[1] https://posithub.org/docs/posit_standard-2.pdf https://posithub.org/docs/posit_standard-2.pdf
- adgjlsfhk1 1y agothe problem with posits is that they aren't enough better to be worth a switch. switching the industry over would cost billions in software rewrites and there are benefits, but they are fairly marginal.
- hedgehog 1y agoFor deep learning workloads the software for posits isn't the issue, it's doing anything that's not NVIDIA if you want to do it as a standalone product. For NVIDIA it's likely the penalty of not being able to share logic with standard size IEEE floats. If adopting posits allowed significantly smaller data types then NVIDIA would likely have adopted already.
- andrepd 1y agoBut 8-bit posits are actually a very nice alternative for deep learning, especially when using the quire for dot products!
- hedgehog 1y agoI don't disagree, it's just that the advantage hasn't yet been shown to be big enough to justify dedicating die area in a mainstream chip. There's potential there, if I were designing an accelerator today I would look hard at posits and variations of blocked representations especially around four bits. A few years back I got to have coffee with John Gustafson which was pretty neat and got me more excited about the idea.
- nibles_and_bits 1y agoThere is actual hardware available for posits. [1][2] [1] https://youtu.be/vzVlQhaAZtQ?si=DJRmwOoyYGdq6mUQ https://youtu.be/vzVlQhaAZtQ?si=DJRmwOoyYGdq6mUQ [2] https://calligotech.com/uttunga/ https://calligotech.com/uttunga/