2 ms·
Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
by aarnphm 3y ago
Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
- nacs 3y agoGPTQ support would be amazing (AutoGPTQ is an easy way to integrate GPTQ support - it's basically just importing autogptq and switching out 1 line in the model loading code).