3 ms·
This may be the status quo because of the so called "hardware lottery" which has historically been optimized for floating point. I'm speculating, but if hardwar
by markisus 3y ago
This may be the status quo because of the so called "hardware lottery" which has historically been optimized for floating point. I'm speculating, but if hardware designers were instead only concerned about raw xnor density and throughput, we might end up with chips powerful enough that giant 1-bit nets could be trained purely through evolution.
- imtringued 3y agoHow do you optimize memory for floating point?
- VHRanger 3y agoBF8 and other similar formats?
- VHRanger 3y agoNo, it's a fact at the mathematical level that you can enshrine in big O terms if you want to