4 ms·

> Open source models like deepseek and llama 3 are rapidly catching up Catching up for how long though? Large models are very expensive to train, and the only

> Open source models like deepseek and llama 3 are rapidly catching up Catching up for how long though? Large models are very expensive to train, and the only reason any of them are open is because ostensibly for-profit companies are absorbing the enormous costs on behalf of the open source scene. What's the plan B when those unprofitable companies (or divisions within companies) pull the ladder up behind them in pursuit of profits?

mgh95 1y ago
> Catching up for how long though? Large models are expensive to train and the money has to come from somewhere, and for now that somewhere is the charity of ostensibly for-profit companies which release their models for free and just eat the losses. That doesn't seem very sustainable to me. Catching up enough to execute a business successfully on the open source (and free as in free beer) alternatives to OpenAI. Once you have a single model that works, you can bootstrap your own internal ML infra for just that one use case and grow from there.
mark_l_watson 1y ago
Martin Casado from a16z stated an opinion that 80% of US startups are likely using less expensive open models, usually from China. Chamath P. on the All In Podcast said that his company is using Chinese models, but hosted in the US and he cited his company as a huge inference user.
- tcdent 1y ago
  Important to distinguish that they're likely using them for internal workflows (agents, etc) where the scope is well defined and they can tune their prompts and evals to accommodate a lower-performance model. Nobody is advocating switching their coding agents to open source (yet), but that's not the bulk of the tokens in companies that have automated workflows integrated into their business.
CheeseFromLidl 1y ago
LLMs can be trained using a distributed approach [1]. So what’s stopping the OSS community from setting up a pool in the likes of seti@home and boinc? [1] I asked Gemini and apparently it’s quite common
- ponector 1y ago
  And then imagine Chinese do distributed training at home. What outcome can they achieve!
sp527 1y ago
I think what you're missing here is that China would be (is?) happy to fund the development, because it's in their national interest and necessary in order for their companies to stay competitive, so long as there are trade restrictions on chips. Another framing for this is that China and certain other entities (e.g. content distribution channels like Meta, Youtube) have a strong incentive to 'commoditize their [AI] complement' (https://gwern.net/complement https://gwern.net/complement).
tasuki 1y ago
Listen to one of the Zuck interviews, he explains this everywhere. To paraphrase, Meta is not releasing open models out of the goodness of their heart, they're doing it as part of their business strategy. - They already have all the eyeballs. - They want to be able to run the best model they can for those eyeballs. - It doesn't make a difference to them whether others have a similar model or not. They don't care. - Ergo, they release their weights, hoping someone can help them improve it. Which has happened: see quantization.

mgh95 1y ago

> Catching up for how long though? Large models are expensive to train and the money has to come from somewhere, and for now that somewhere is the charity of ostensibly for-profit companies which release their models for free and just eat the losses. That doesn't seem very sustainable to me. Catching up enough to execute a business successfully on the open source (and free as in free beer) alternatives to OpenAI. Once you have a single model that works, you can bootstrap your own internal ML infra for just that one use case and grow from there.

mark_l_watson 1y ago

Martin Casado from a16z stated an opinion that 80% of US startups are likely using less expensive open models, usually from China. Chamath P. on the All In Podcast said that his company is using Chinese models, but hosted in the US and he cited his company as a huge inference user.

tcdent 1y ago
Important to distinguish that they're likely using them for internal workflows (agents, etc) where the scope is well defined and they can tune their prompts and evals to accommodate a lower-performance model. Nobody is advocating switching their coding agents to open source (yet), but that's not the bulk of the tokens in companies that have automated workflows integrated into their business.

CheeseFromLidl 1y ago

LLMs can be trained using a distributed approach [1]. So what’s stopping the OSS community from setting up a pool in the likes of seti@home and boinc? [1] I asked Gemini and apparently it’s quite common

ponector 1y ago
And then imagine Chinese do distributed training at home. What outcome can they achieve!

sp527 1y ago

I think what you're missing here is that China would be (is?) happy to fund the development, because it's in their national interest and necessary in order for their companies to stay competitive, so long as there are trade restrictions on chips. Another framing for this is that China and certain other entities (e.g. content distribution channels like Meta, Youtube) have a strong incentive to 'commoditize their [AI] complement' (https://gwern.net/complement https://gwern.net/complement).

tasuki 1y ago

Listen to one of the Zuck interviews, he explains this everywhere. To paraphrase, Meta is not releasing open models out of the goodness of their heart, they're doing it as part of their business strategy. - They already have all the eyeballs. - They want to be able to run the best model they can for those eyeballs. - It doesn't make a difference to them whether others have a similar model or not. They don't care. - Ergo, they release their weights, hoping someone can help them improve it. Which has happened: see quantization.