4 ms·
My fear is that these large "AI" companies will lobby to have these open source options removed or banned, growing concern. I'm not sure how else to explain how
by razster 8mo ago
My fear is that these large "AI" companies will lobby to have these open source options removed or banned, growing concern. I'm not sure how else to explain how much I enjoy using what HF provides, I religiously browse their site for new and exciting models to try.
- culi 8mo agoModelScope is the Chinese equivalent of Hugging Face and a good back up. All the open models are Chinese anyways
- thot_experiment 8mo agoNot true! Mistral is really really good, but I agree that there isn't a single decent open model from the USA.
- culi 8mo agoMistral is cool and I wish them success but it consistently ranks extremely low on benchmarks while still being expensive. Chinese models like DeepSeek might rank almost as low as Mistral but they are significantly cheaper. And Kimi is the best of both worlds with incredible benchmark results while still being incredibly cheap I know things change rapidly so I'm not counting them out quite yet but I don't see them as a serious contender currently
- Eupolemos 8mo agoWhy are you talking price when we are talking local AI? That doesn't make any sense to me. Am I missing something?
- culi 8mo agoYour electricity is free?
- cpburns2009 8mo agoIf you have the hardware to run expensive models, is the cost of electricity much of a factor? According to Google, the average price in the Silicon Valley Area is $0.448 per kWh. An RTX 5090 costs about $4,000 and has a peak power consumption of 1000 W. Maxing out that GPU for a whole year would cost $3,925 at that rate. It's not particularly more expensive than that hardware itself.
- culi 8mo agoAt that point it'd be cheaper to get an expensive subscription to a cloud platform AI product. I understand the case for local LLMs but it seems silly to worry about pricing for cloud-based offerings but not worry about pricing for locally run models. Especially since running it locally can often be more expensive
- seanmcdirmid 8mo agoApple silicon is crazy efficient as well as being comparable to GPUs in performance for max and ultra chips.
- thot_experiment 8mo agofor almost the entire year, yes.
- dirasieb 8mo ago15 missed calls from your local power company
- thot_experiment 8mo agoSure, benchmarks are fake and I use Mistral over equivalently sized models most of the time because it's better in real life. It runs plenty fast for me, I don't pay for inference.
- BoredomIsFun 8mo ago> it consistently ranks extremely low on benchmarks As general purpose chatbots small Mistral models are better than comparably sized Chiniese models, as they have better SimpleQA scores and general knowledge of Western culture.
- seanmcdirmid 8mo agoIt’s really hard to beat qwen coder, especially for role play where the instruction following is really useful. I don’t think their corpus is lacking in western knowledge, although I wonder if Chinese users get even better results from it?
- BoredomIsFun 8mo ago> It’s really hard to beat qwen coder, for role play I am not sure if you actually tried that. Mistrals are widely asccepted go-to models for roleplay and creative writing. No Qwens are good at prose, except for their latest big Qwen 3.5. > I don’t think their corpus is lacking in western knowledge, It absolutely does, especially pop culture knowledge.
- seanmcdirmid 8mo agoInstruct and coder just follow instructions so well though. I guess I’ve just never been able to make mistral work well, I guess.
- BoredomIsFun 8mo agoQwen3 30B A3B and that big 400+ B Coder were absolutely terrible at editing fiction. I would tell them what to change in the prose and they'd just regurgitate text with no changes.
- seanmcdirmid 8mo agoDid you try asking Gemini what model to use and how to configure/set it up? It has worked wonders for me, ironically (since I’m using a big model to setup smaller local models).
- CamperBob2 8mo agoTo be fair there are lots of worse models than OpenAI's GPT-OSS-120b. It's not a standout when positioned next to the latest releases from China, but prior to the current wave it was considered one of the stronger local models you can reasonably run.
- ac29 7mo agoArcee is working on that, see a blog post about their newest in progress model here: https://www.arcee.ai/blog/trinity-large https://www.arcee.ai/blog/trinity-large Its still not fully post trained and its a non-reasoning model, but its worth keeping an eye on if you dont want to use the Chinese models that currently are the best open-weight options.
- throwaway27448 8mo agoThey can try. I don't think they'll be able to get the toothpaste back in the tube. The data will just move our of the country.
- seanmcdirmid 8mo agoMany of the models on hugging face are already Chinese. It’s kind of obvious that local AI is going to flourish more in China than the USA due to hardware constraints.
- dotancohen 8mo agoHow do you choose which models to try for which workflows? Do you have objective tests that you run, or do you just get a feel for them while using them in your daily workflow?
- toofy 8mo agoit’s only a matter of time. we have all seen first hand how … wrong … these companies behave, almost on a regular basis. there’s a small tinfoil hat part of me that suspects part of their obscene investments and cornering the hardware market is driven by an conscious attempt to stop open source local from taking off. they want it all, the money, the control, and to be the only source of information to us.