2 ms·
Of the small models, Vicuna and WizardLM seem to be the best of what's available right now. Quantized 7B models are just under 4 Gb of RAM, so they should fit o
by int_19h 4y ago
Of the small models, Vicuna and WizardLM seem to be the best of what's available right now. Quantized 7B models are just under 4 Gb of RAM, so they should fit on most GPUs out there; and even if not, they're "fast enough" on the CPU. But if you have enough RAM to run vicuna-13b (12 Gb should do it quite nicely), I'd start there.