4 ms·
What about something that's maybe more along the lines of a GPT-2.5?
by ineedasername 4y ago
What about something that's maybe more along the lines of a GPT-2.5?
- int_19h 4y agoOf the small models, Vicuna and WizardLM seem to be the best of what's available right now. Quantized 7B models are just under 4 Gb of RAM, so they should fit on most GPUs out there; and even if not, they're "fast enough" on the CPU. But if you have enough RAM to run vicuna-13b (12 Gb should do it quite nicely), I'd start there.