2 ms·
The GPTQ implementation is reasonably fast, I think. Or maybe they are running it at 8 bits? I just saw that they funded this: https://huggingface.co/ehartford
by brucethemoose2 3y ago
The GPTQ implementation is reasonably fast, I think.
Or maybe they are running it at 8 bits? I just saw that they funded this: https://huggingface.co/ehartford/WizardLM-Uncensored-Falcon-40b https://huggingface.co/ehartford/WizardLM-Uncensored-Falcon-...
- behnamoh 3y agoInteresting! I don’t know what they’re up to and what’s the purpose of their project. But using an uncensored model is kinda risky in production, unless they know what they’re doing.