4 ms·

If you believe, as an increasing number of folks do, that LLMs are basically commoditized at this point or will be soon then there really isn’t a path forward f

by cmiles8 2mo ago

If you believe, as an increasing number of folks do, that LLMs are basically commoditized at this point or will be soon then there really isn’t a path forward for non open models. There simply isn’t any value in the models and only in the compute to run them and potentially some higher-order coordination layers. That’s very bad for the big model labs that banked their future on the opposite, but there just doesn’t seem to be a viable path forward on that approach anymore. Meta hasn’t been the best global citizen for most of its life, but there may be some mild redemption if it becomes a source of strong open models.

cdata 2mo ago
I strongly prefer a world where open models "win," that said... I don't understand how we will sustain an ecosystem of open models like the one that exists today. My thoughts go down this path every time I try to put the pieces together: - Many open models depend on high quality frontier models as a distillation input - US closed frontier models drive China's open model strategy despite the business model for open models being somewhat fuzzy - What is the business model that benefits from open weights, exactly? How do their vendors recoup infrastructure and training costs? I would be grateful to be enlightened on this topic.
- cmiles8 2mo ago
  Wouldn’t be that different from open source software, which has been very successful in many circles. You make money from value-add on top of the models not the models themselves.
  - redox99 2mo ago
    Yeah but open source is often a few guys working on their free time, or a large company open sourcing libraries that they wouldn't be able to sell anyway, or donations from corporations that are at best in the order of a million dollars. For frontier models you have players investing billions.
  - solenoid0937 2mo ago
    Open source software does not require trillions of dollars in R&D to produce the next iteration

by cmiles8 2mo ago

cdata 2mo ago

I strongly prefer a world where open models "win," that said... I don't understand how we will sustain an ecosystem of open models like the one that exists today. My thoughts go down this path every time I try to put the pieces together: - Many open models depend on high quality frontier models as a distillation input - US closed frontier models drive China's open model strategy despite the business model for open models being somewhat fuzzy - What is the business model that benefits from open weights, exactly? How do their vendors recoup infrastructure and training costs? I would be grateful to be enlightened on this topic.

cmiles8 2mo ago
Wouldn’t be that different from open source software, which has been very successful in many circles. You make money from value-add on top of the models not the models themselves.
- redox99 2mo ago
  Yeah but open source is often a few guys working on their free time, or a large company open sourcing libraries that they wouldn't be able to sell anyway, or donations from corporations that are at best in the order of a million dollars. For frontier models you have players investing billions.
- solenoid0937 2mo ago
  Open source software does not require trillions of dollars in R&D to produce the next iteration