2 ms·
Why doesn't it work? Try Ollama, or llama.cpp, they should work fine out of the box. They do on my 3070 just fine. Also, try a model that fits your VRAM.
by stavros 3y ago
Why doesn't it work? Try Ollama, or llama.cpp, they should work fine out of the box. They do on my 3070 just fine. Also, try a model that fits your VRAM.