4 ms·

For what it worth nearly all public models are distilled versions of bigger internal ones

• arnaudsm 1y ago
  Even flagships like o3 & gemini_2.5_pro ?
  • ffsm8 1y ago
    I doubt you'll get a response from someone with authority on the matter (that actually worked on these models and is willing and authorized to post this publicly)... So I'm gonna add my uninformed consumer perspective: I sincerely doubt the o3/2.5 pro haven't been distilled. It's unimaginable to me they're that price insensitive (or expressed inversely: were so thrifty in training that the final product can be used without optimization for the consumer usage) the only conclusion I can come to is that they're indeed not letting you access the "root" models.
    • creshal 1y ago
      I think OpenAI even mentioned in some papers that the internal o4(?) model used for some tests cost $6000 per query, pre-release. That's absolutely getting distilled down for releases.
    • regularfry 1y ago
      The more conservative version of this is that they'd want distilled models even if only as a speculative decoder to stick in front of the main model. That's an obvious optimisation to make.