6 ms·
1. Download ollama and give it a try. It doesn't have to be that one but it's an easy on-ramp. Just get a feel for what open source/open weights is capable of
by nucleative 1mo ago
1. Download ollama and give it a try. It doesn't have to be that one but it's an easy on-ramp. Just get a feel for what open source/open weights is capable of
2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools.
3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do.
4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice.
5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs.
I'm sure others will chime in, those are a few that pop the mind.
- KronisLV 1mo ago> Download ollama and give it a try Weren't people pissed off at them for a variety of reasons? Not giving attribution etc. In case anyone wants other good alternatives, there's llama.cpp and vLLM, each harder to setup but opens the door for squeezing more performance out of your hardware. I've also heard okay things about LM Studio and I think Unsloth had their own thing as well: https://unsloth.ai/docs/new/studio https://unsloth.ai/docs/new/studio > Mention it whenever it comes up. You could also vote with your valley and support orgs that release open weights of their near-SOTA models, though nowadays that means giving cash to primarily Chinese companies (e.g. Moonshot and Z.ai). For what it's worth, people are also complaining about them decreasing usage quotas on their subscription offerings as well so seems like the squeeze is everywhere, even DeepSeek raised their prices (which is better than them going broke, I guess).
- wongarsu 1mo agosglang is also worth mentioning. Fills a similar role as vLLM, but with less fiddling at the knobs to get a working setup For a GUI experience that also serves an OpenAI-compatible API, Unsloth studio or LM Studio is probably the way to go
- eloisant 1mo agoI recommend llama.cpp over ollama. Anyway, when you see a model on huggingface you get the exact commands to copy paste to run it locally.
- joelthelion 1mo agoOne issue is that models are only one part of the equation. Search is also extremely important, and for that you pretty much have to depend on a third-party service.