4 ms·
So excited to see this - the big advantage of 1.58 bits is there are no multiplications at inference time, so you can run them on radically simpler and cheaper
by yodon 7mo ago
So excited to see this - the big advantage of 1.58 bits is there are no multiplications at inference time, so you can run them on radically simpler and cheaper hardware.
- Animats 7mo agoAt 4 bits, you could just have a hard-wired table lookup. Two 4 bit values in, 256 entry table. You can have saturating arithmetic and a post-processing function for free. Somebody must be building hardware like that.
- Taniwha 7mo agoand so you can at 1-bit too, and the hardware will be even smaller and cheaper
- AlotOfReading 7mo agoA LUT is pretty wasteful. You only have a one bit significand, so the mantissa and sign bits are boolean binops, and the exponent is a 2 bit adder.
- londons_explore 7mo agoLow level software engineers use look up tables. Hardware engineers realise that a compiler will almost always find some combination of gates which is smaller/faster than the contents of any table.