4 ms·
With the 4bit quantization they needed to make it is like a jpeg with only 16 pixels per "thought" parameter. It's going to perform like a brilliant person wit
by koheripbal 4y ago
With the 4bit quantization they needed to make it is like a jpeg with only 16 pixels per "thought" parameter.
It's going to perform like a brilliant person with brain injury. 8 bit is far better and it would be interesting to see if they can get it working on any reasonable hardware.
- sebzim4500 4y agoApparently if you quantize carefully you can avoid any loss of performance. See https://arxiv.org/pdf/2210.17323.pdf https://arxiv.org/pdf/2210.17323.pdf
- BulgarianIdiot 4y agoThe first proofs of concept are rarely ideal. But you know no one will stop here. On the software side, on the hardware side.