3 ms·

There's definitely going to be cheap or open source models

palmotea 4mo ago
> There's definitely going to be cheap or open source models What makes you think your "cheap or open source model" running on your piddling desktop cluster will be able to complete against a SOTA one running in a billion-dollar datacenter? It's a cyberpunk fantasy. It won't work out that way.
- jhbadger 4mo ago
  Local models that run on a laptop (not even needing a "cluster") are already better than ChatGPT from a couple of years ago. Yes, Claude and ChatGPT today are certainly better than these local models, but they can't keep getting better indefinitely -- there's only so much info to scrape. When they hit a plateau, it is only a matter of time that consumer hardware will catch up to it.
  - ryan_n 4mo ago
    > but they can't keep getting better indefinitely Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..
  - Timon3 4mo ago
    While that's most likely true, it rests on the assumption that consumer hardware stays affordable enough, and isn't locked down to disallow running "untrusted" models. I would have never believed that these assumptions could ever turn out false, but the recent developments have shown that even if unlikely, it's not impossible.
- teeray 4mo ago
  I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
  - palmotea 4mo ago
    > I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter. And then they take that model, put it in a billion-dollar datacenter, and kick your desktop cluster's ass with it.

> There's definitely going to be cheap or open source models What makes you think your "cheap or open source model" running on your piddling desktop cluster will be able to complete against a SOTA one running in a billion-dollar datacenter? It's a cyberpunk fantasy. It won't work out that way.

jhbadger 4mo ago
Local models that run on a laptop (not even needing a "cluster") are already better than ChatGPT from a couple of years ago. Yes, Claude and ChatGPT today are certainly better than these local models, but they can't keep getting better indefinitely -- there's only so much info to scrape. When they hit a plateau, it is only a matter of time that consumer hardware will catch up to it.
- ryan_n 4mo ago
  > but they can't keep getting better indefinitely Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..
- Timon3 4mo ago
  While that's most likely true, it rests on the assumption that consumer hardware stays affordable enough, and isn't locked down to disallow running "untrusted" models. I would have never believed that these assumptions could ever turn out false, but the recent developments have shown that even if unlikely, it's not impossible.
teeray 4mo ago
I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
- palmotea 4mo ago
  > I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter. And then they take that model, put it in a billion-dollar datacenter, and kick your desktop cluster's ass with it.