4 ms·
There are at least two different optimizations happening: 1) optimizing the model training 2) optimizing the model operation The $1B-spend holy grail is that
by svnt 2y ago
There are at least two different optimizations happening:
1) optimizing the model training
2) optimizing the model operation
The $1B-spend holy grail is that it costs a lot of money to train, and almost nothing to operate, a proprietary model that benchmarks and chats better than anyone else’s.
OpenAI’s optimizations fall into the latter category. The risk to the business model is in the former — if someone can train a world-beating model without lots of money, it’s a tough day for the big players.
- ComputerGuru 2y agoI disagree. Not axiomatically because you’re kind of right, but enough to comment. OpenAI doesn’t believe in optimizing the traisning costs of AI but believes in optimizing (read: maxing) the training period. Their billions go to collecting, collating, and transforming as much training data as they can get their hands on. To see what optimizing model operation looks like, groq is a good example. OpenAI isn’t (yet) obviously in that kind of optimization, though I’m sure they’re working on it internally.
- svnt 2y agoMy argument wasn’t that the well-funded entities were optimizing to reduce training costs, but the opposite: they need creative ways to spend $1B that provide some tangible advantage. But they need operating costs to be low or they lose money and try to somehow make it up on volume. I would roll data acquisition/cleaning processes into training costs for purposes of this because what else is the data for if not training? If 4o wasn’t an optimization for model operation costs what was it?