4 ms·
That's to be expected. 4 bits is too small. But the 8 bit 65b should outperform significantly. Have you tried it?
by koheripbal 4y ago
That's to be expected. 4 bits is too small. But the 8 bit 65b should outperform significantly.
Have you tried it?
- sebzim4500 4y agoIs 4 bits too small? Or is the quantization method just not sophisticated enough. See https://arxiv.org/pdf/2210.17323.pdf https://arxiv.org/pdf/2210.17323.pdf
- enduser 4y agoI’ll give it a try this evening.