4 ms·
I've been using a lot of the Chinese open source models like R1 and the Qwen coder series. I've also been trying some community mods of them from HuggingFace.
by dabockster 1y ago
I've been using a lot of the Chinese open source models like R1 and the Qwen coder series. I've also been trying some community mods of them from HuggingFace.
I was using a combo of Ollama + Roo Code for the front/back end but Ollama is kind of dumb when it comes to memory overload protections. I've since switched to LM Studio and it has a very annoying hard timeout of 2-3 minutes on its API server. Local isn't perfect right now, but when it does run you can see the potential as clear as day.