2 ms·

Frontier models being in the hands of a handful companies does not help either. Let's hope that the open weight movement changes that soon.

by binyu 6mo ago

serf 6mo ago
realistically any 'huge' frontier model that takes a rack of H100s to infer against is probably going to have downtime no matter who runs it. downtime is always going to 'scale' poorly against loads that require a lot of hardware thrown at them, even with lots of good fail-over -- probably worse for the small vendors because they don't have the contracts supplying them with hardware first so availability is already at a premium for them. so, I guess i'm saying yeah I hope frontier-level-models get out soon in the open arenas, but I suspect the same or similar level of exclusivity will exist as long as they take that much compute to operate.
kristianc 6mo ago
If it goes as well as the 'open' / federated social network alternatives of the 2010s, I wouldn't count on it.
- amarant 6mo ago
  Social networks are 100% network effect. AI models are not really effected by that at all. Which doesn't mean the open models will definitely succeed, it just means they have more of a shot than the open social networks ever did
  - kristianc 6mo ago
    >AI models are not really effected by that at all. I don't know about that. More usage means more support, which means more docs and open source projects, wrappers, harnesses built around them etc. Way less demand to build tooling around open weight models if they remain hobbyist.
    - andoando 6mo ago
      The big thing is here is more training and that comes in two flavors: 1. Using AI helps as part of the training process. 2. All the prompts going to openai/claude is a gold mine.
ru552 6mo ago
Gemma 4 has made a lot of progress in this area. The model is phenomenal. It's size is workable. This is the worst it will ever be.
- nextaccountic 6mo ago
  Now we just need the RAM market to get back to normal. Or at least fine OpenAI for speculating on raw wafers. There's an article on the front page [0] with this passage that gives me hope that consumer access to VRAM may improve > On the infrastructure side: OpenAI signed non-binding letters of intent with Samsung and SK Hynix for up to 900,000 DRAM wafers per month, roughly 40% of global output. These were of course non-binding. Micron, reading the demand signal, shut down its 29-year-old Crucial consumer memory brand to redirect all capacity toward AI customers. Then Stargate Texas was cancelled, OpenAI and Oracle couldn’t agree terms, and the demand that had justified Micron’s entire strategic pivot simply vanished. Micron’s stock crashed. [0] https://adlrocha.substack.com/p/adlrocha-how-the-ai-loser-may-end https://adlrocha.substack.com/p/adlrocha-how-the-ai-loser-ma...
  - Lutzb 6mo ago
    Microns stock is still up 470% yoy

Frontier models being in the hands of a handful companies does not help either. Let's hope that the open weight movement changes that soon.

by binyu 6mo ago

Frontier models being in the hands of a handful companies does not help either. Let's hope that the open weight movement changes that soon.

serf 6mo ago

realistically any 'huge' frontier model that takes a rack of H100s to infer against is probably going to have downtime no matter who runs it. downtime is always going to 'scale' poorly against loads that require a lot of hardware thrown at them, even with lots of good fail-over -- probably worse for the small vendors because they don't have the contracts supplying them with hardware first so availability is already at a premium for them. so, I guess i'm saying yeah I hope frontier-level-models get out soon in the open arenas, but I suspect the same or similar level of exclusivity will exist as long as they take that much compute to operate.

kristianc 6mo ago

If it goes as well as the 'open' / federated social network alternatives of the 2010s, I wouldn't count on it.

amarant 6mo ago
Social networks are 100% network effect. AI models are not really effected by that at all. Which doesn't mean the open models will definitely succeed, it just means they have more of a shot than the open social networks ever did
- kristianc 6mo ago
  >AI models are not really effected by that at all. I don't know about that. More usage means more support, which means more docs and open source projects, wrappers, harnesses built around them etc. Way less demand to build tooling around open weight models if they remain hobbyist.
  - andoando 6mo ago
    The big thing is here is more training and that comes in two flavors: 1. Using AI helps as part of the training process. 2. All the prompts going to openai/claude is a gold mine.

ru552 6mo ago

Gemma 4 has made a lot of progress in this area. The model is phenomenal. It's size is workable. This is the worst it will ever be.

nextaccountic 6mo ago
Now we just need the RAM market to get back to normal. Or at least fine OpenAI for speculating on raw wafers. There's an article on the front page [0] with this passage that gives me hope that consumer access to VRAM may improve > On the infrastructure side: OpenAI signed non-binding letters of intent with Samsung and SK Hynix for up to 900,000 DRAM wafers per month, roughly 40% of global output. These were of course non-binding. Micron, reading the demand signal, shut down its 29-year-old Crucial consumer memory brand to redirect all capacity toward AI customers. Then Stargate Texas was cancelled, OpenAI and Oracle couldn’t agree terms, and the demand that had justified Micron’s entire strategic pivot simply vanished. Micron’s stock crashed. [0] https://adlrocha.substack.com/p/adlrocha-how-the-ai-loser-may-end https://adlrocha.substack.com/p/adlrocha-how-the-ai-loser-ma...
- Lutzb 6mo ago
  Microns stock is still up 470% yoy