3 ms·I can run 4bit on a beat up 1070 ti. GP talks about higher precision modelsby pennaMan 3y agoI can run 4bit on a beat up 1070 ti. GP talks about higher precision modelssp332 3y agoYou wouldn’t be able to fit the whole model into 8GB VRAM. It’s faster than not using a GPU at all, but most of it would still be computed on the CPU.baq 3y agoIME ollama ran mixtral on a 1070 fast enough.dimask 3y agoThough it most probably does not run in on the 1070 but rather on the cpu. It cannot fit on a 1070, it is not about speed, a 1070 cannot run it period.Dkuku 3y agoIn llama.cpp You can offload some of the layers to gpu with -ngl X. Where x is the number of layers