4 ms·
Half the reason why they move back up to 8/16 bit is that current hardware doesn't properly support 4 bit floats, and you get better performance from the conver
by creshal 3y ago
Half the reason why they move back up to 8/16 bit is that current hardware doesn't properly support 4 bit floats, and you get better performance from the conversion. I think once this hardware hits, most of the computation will shift to native 4 bit just for efficency's sake.
...assuming the recent 1.58b paper doesn't render the entire float quantization approach obsolete by then.
- imtringued 3y agoThe 1.58b approach is good for everyone including for quantization. It means that current quantization schemes have room for improvement.
- creshal 3y agoIt's good for everyone on the software side, but it's not so good for the hardware side, because it means that whatever you're designing now for tapeout in 6 months and release in 12 is going to be obsolete in 2 weeks.