3 ms·
Vicuna v1.5 13B q8 runs locally using a 3060ti 8GB VRAM card on Win 10, dual Xeon (16 core) with 128GB RAM. I use LM Studio (mac, win & linux) which is super ea
by Jack_Sprat_89 3y ago
Vicuna v1.5 13B q8 runs locally using a 3060ti 8GB VRAM card on Win 10, dual Xeon (16 core) with 128GB RAM. I use LM Studio (mac, win & linux) which is super easy to install and it has a local inference server that you can connect clients to using an openai style api. A very fun project so far...