3 ms·
It's mostly a test of the coding capabilities of the 9B model. The 0.8B is used to make the telegram bot smarter than a if/then/else. I find LM Studio more usa
by advanced-stack 8mo ago
It's mostly a test of the coding capabilities of the 9B model. The 0.8B is used to make the telegram bot smarter than a if/then/else.
I find LM Studio more usable for local setups (desktop/laptop) and I would use directly llama.cpp stack for a (local) server deployment
- ZeroGravitas 8mo agoI revisited this and using the models directly via ollama run was actually surprisingly fast. A bug or misconfiguration with the connection to opencode seemed to be the culprit.
- advanced-stack 8mo agommh.. interesting; I'll try to compare the perf