3 ms·
The cost of copying SOTA models though is super cheap and doesn’t take super long.
by vlovich123 7mo ago
The cost of copying SOTA models though is super cheap and doesn’t take super long.
- aurareturn 7mo agoHow do you distill when OpenAI and Anthropic inevitably move to tasks running in the cloud? IE. Go buy this extremely hard to get concert ticket for me. Distilling might only be effective in the chat bot dominant era. We are about to move to an agents era. Furthermore, I’m guessing distilling will get harder and harder. Claude Code leak shows some primitive anti distilling methods already. There’s research showing that models know when it’s being benchmarked. Who’s to say Anthropic and OpenAI aren’t able to detect when their models are being distilled?
- adgjlsfhk1 7mo agoeven ignoring distillation, so long as hardware or ml get better over time, training a new model from scratch is cheaper the later you do it
- ef3dfd 7mo agoYep the poster is assuming efficiencies will not come. Absolutely they will. And this is a huge problem for OAI - given Google is targeting vertical integration, they will acquire a cost-advantage. As long as the model performance is good enough, they will kick OAI and Anthropic out in the long-run. The valuations of OAI and Anthropic are nonsense. A true valuation would incorporate failure risk, which is natural for startups/fast growing and money losing firms. Anyone who takes them serious is incredibly delusional.
- aurareturn 7mo agoIf hardware gets better over time, they also get better for OpenAI.
- vlovich123 7mo agoIt does, but for what it’s worth they have tech and financial tech of their past to pay down. This is how startups disrupt incumbents