2 ms·
> suspect it’s better than say a 9b or 12b model Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
by embedding-shape 3mo ago
> suspect it’s better than say a 9b or 12b model
Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
- kamranjon 3mo agoSorry I should have clarified - I meant that a ternary 27b model would outperform a non-quantized or 8 bit quantized 9 or 12b model - which it is generally close to (or much smaller than) in size. So yeah the comparison I was trying to make was between models of equivalent size or models that could run on similarly sized hardware.
- embedding-shape 3mo agoAh yeah, that makes a ton more sense :) I mean, what you said earlier also makes sense but was too obvious, now it makes sufficient sense, thanks for explaining :)