3 ms·
A 3 bit quant will run on a 128gb MacBook Pro, it works pretty well.
by FuckButtons 7mo ago
A 3 bit quant will run on a 128gb MacBook Pro, it works pretty well.
- nl 7mo agoA 3 bit quant is quite a lot weaker than the OpenRouter version the OP is using.