3 ms·
This is such exciting news! Huge thanks to you for your continued work in making sense of AI. I wonder if the recent Bitnet 1.58 paper [the use of ternary bits
by ricopags 3y ago
This is such exciting news! Huge thanks to you for your continued work in making sense of AI.
I wonder if the recent Bitnet 1.58 paper [the use of ternary bits in lieu of fp/int] might be an advancement that could further reduce the computation required for inference?
- jph00 3y agoYes, along with the many other <4 bit quant methods recently developed -- there's been a wonderful boom in low-bit quant methods in the last 6 months, and we've got our own ideas for taking them further too. Along with QLoRA/FSDP, we're likely to see big advances in model training this year on consumer hardware.