4 ms·
Unfortunately what I tested was StarCoder 4bit. We really need exllama which should make even 30b viable from what I can tell. Because codellama is llama based
by thewataccount 3y ago
Unfortunately what I tested was StarCoder 4bit.
We really need exllama which should make even 30b viable from what I can tell.
Because codellama is llama based it may just work possibly?