4 ms·
I don't have a fancy GPU :( I do however have a dual xeon with 64GB of ram. Will that work for this?
by pcdoodle 3y ago
I don't have a fancy GPU :(
I do however have a dual xeon with 64GB of ram. Will that work for this?
- lotyrin 3y agoIf you're patient, yes.
- SushiHippie 3y agoWhat is "dual xeon", two xeon CPUs or dual core? On my laptop with an i7 I had some decent performance with it. Don't know token/s , but it took less than 20s to generate an answer. Should note that I used q5 quantisation with llama.cpp: https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGU...