Have you tried Ollama [1]? You should be able to run a 8b model in RAM and a 1b model in VRAM. [1] https://news.ycombinator.com/item?id=42069453 https://news.y

Have you tried Ollama [1]? You should be able to run a 8b model in RAM and a 1b model in VRAM. [1] https://news.ycombinator.com/item?id=42069453 https://news.ycombinator.com/item?id=42069453