3 ms·
To the extent that the present LLM movement reaches a steady state conclusion it’s highly likely to be open source models on your own hardware that are “good en
by cmiles8 7mo ago
To the extent that the present LLM movement reaches a steady state conclusion it’s highly likely to be open source models on your own hardware that are “good enough” for 95% of use cases.
That blows up the whole “industrial complex” being developed around massive data centers, proprietary models, and everything that goes with that. Complete implosion.
Apple has sat on the sidelines for much of this as it seems clear they know the end game is everyone just does this stuff locally on their phone or computer and then it’s game over for everything going on now.
- draxil 7mo agoI assume you mean open weight models? I wish we had better open source models. It would make LLMs far less icky if we had nice clean open trained models. A breakthrough on the cost of training would be nice.
- cmiles8 7mo agoFair clarification, yes.
- mike_hearn 7mo agoNemotron is genuinely open source at least at the smaller sizes. You can download the datasets.
- marci 7mo agoAlso everything from scratch by allen.ai. Weights, datasets, code, multiple checkpoints... I like their FlexOlmo concept.
- Yizahi 7mo agoWe really can't have open source LLM, because they are all based on the stolen IP, or stolen IP slightly laundered and under different title.
- harlanji 7mo agoI feel like an opt-in model built on AGPL code should output AGPL code. I'd put my work into that. Not the only option just an example. Every great project takes time to build. It's possible.
- BoredomIsFun 7mo agoHmm...no. These two things are orthogonal. Regardless, Olmo are opensource.
- seism 7mo agoCheck out Apertus, the publicly funded model from a research team that goes to great lengths to remove icky content.
- mr_toad 7mo agoStill need massive amounts of compute for training. Nobody is going to be training 400B models on a phone any time soon.
- cmiles8 7mo agoLikely not. We’re seeing a massive slowing in the value of all that additional training. Folks don’t like to talk about that, but absent a completely new break-thru the current math of LLMs has largely run its course. We simply don’t need massive training forever and ever. We’re getting to the point that “good enough” models will solve most use cases. The demonstrated business value is also still broadly missing for AI on the level required to keep funding all this training for much longer.
- mangoman 7mo agoI dunno, I thought that too for a while too, but there are a lot of new ideas in terms of architecture that may warrant massive training runs. Mamba and state space models are pretty interesting, but haven’t had their transformer moment yet because I haven’t really seen anyone go for broke on training it with a huge data set and model size. Even some of the more fundamental changes too like Kolmogorov–Arnold Networks or some of the ideas behind continuous back propagation haven’t really had the opportunity to be pushed to the limit. I think it’s still early days on what these models can do. And I say this as someone who bought a Mac m3 max 128gb ram, based on the hope that the on device training and inference work would eventually move locally. It’s encouraging to see the progress though and I hope it does move locally though.
- parineum 7mo ago> but there are a lot of new ideas in terms of architecture that may warrant massive training runs I don't think the argument is that isn't true, it's that the gains from those massive training runs is diminishing. Eventually, it won't be worth it to do the run for each new idea, you'll have to bundle a bunch together to get any noticeable change.
- 7mo ago
- noemit 7mo agoEven if it runs, this will run slowly, and heat up. I think local will always have a place, but the infrastructure is going to be used in my humble opinion.
- cmiles8 7mo agoToday yes, but between the improved performance of smaller on device models and the hardware itself getting better this issue is short lived.
- plussed_reader 7mo agoI don't want to put information into a black box of mystery that can then be used for other monetization purposes. I am still waiting for a realistic local solution.
- efnx 7mo agoHave you tried qwen3.5 running locally? It’s quite “good enough”.
- throwaway173738 7mo agoCompute evolved from batch systems with time sharing to responsive systems in your pocket. Why wouldn’t that happen here?
- gervwyk 7mo agothere was a time when mainframes was the main thing.. we’ll look back and say data centers was a thing.. (hopefully if we lucky)
- efsavage 7mo ago> “good enough” for 95% of use cases Maybe, for current use cases. I'd argue that anyone who thinks they can do everything a 10kW server can do on their 10W device just isn't being creative enough :)
- hiddencost 7mo agoConsumer market is small compared to headcount reduction and cutting edge science.