I've set up Ollama and this webgui on my proxmox homeserver (HP EliteDesk G3 800 SFF) and it is fast enough to run 7 and 13B models. I use it in vim and vscode

I've set up Ollama and this webgui on my proxmox homeserver (HP EliteDesk G3 800 SFF) and it is fast enough to run 7 and 13B models. I use it in vim and vscode too via the API. It feels great to do the inference locally on my own hardware and it is surprisingly useful for everyday tasks. My next goal is to make some simple shell scripts so I can pipe in random data to Ollama from my terminal!