3 ms·
> Download ollama and give it a try Weren't people pissed off at them for a variety of reasons? Not giving attribution etc. In case anyone wants other good al
by KronisLV 1mo ago
> Download ollama and give it a try
Weren't people pissed off at them for a variety of reasons? Not giving attribution etc.
In case anyone wants other good alternatives, there's llama.cpp and vLLM, each harder to setup but opens the door for squeezing more performance out of your hardware.
I've also heard okay things about LM Studio and I think Unsloth had their own thing as well: https://unsloth.ai/docs/new/studio https://unsloth.ai/docs/new/studio
> Mention it whenever it comes up.
You could also vote with your valley and support orgs that release open weights of their near-SOTA models, though nowadays that means giving cash to primarily Chinese companies (e.g. Moonshot and Z.ai). For what it's worth, people are also complaining about them decreasing usage quotas on their subscription offerings as well so seems like the squeeze is everywhere, even DeepSeek raised their prices (which is better than them going broke, I guess).
- wongarsu 1mo agosglang is also worth mentioning. Fills a similar role as vLLM, but with less fiddling at the knobs to get a working setup For a GUI experience that also serves an OpenAI-compatible API, Unsloth studio or LM Studio is probably the way to go
- eloisant 1mo agoI recommend llama.cpp over ollama. Anyway, when you see a model on huggingface you get the exact commands to copy paste to run it locally.