3 ms·

Meta confirms that its Llama 3 open source LLM is coming in the next month

43 points by blackeyeblitzar 3y ago

Seattle3503 3y ago
Is Llama remaining open an example of meta attempting to commoditize its complement?
- deleted 3y ago
  [deleted]
- blackeyeblitzar 3y ago
  In my opinion, no, since it isn’t clear how an LLM is a complement to Meta’s core businesses or who is affected by that commoditization. Maybe others can educate me about that. My view is that they have the key capabilities to build high performance LLMs - talent, cash (for hardware), data - so they might as well do it just to stay relevant. They may ultimately create innovations that are ahead of everyone else (like the JEPA architecture that Yann LeCun is pushing) but until then, at least they’re proving to users and investors that they still have bleeding edge technical capabilities. And sure they may find ways to use these models in new incremental features in their existing products (Facebook, Instagram, WhatsApp) - but they don’t have a great way to reach users even if they had new AI-centric products to offer (more below). At least they get some good marketing by putting open weight models out there as an alternative to big players pushing closed models and services. I think a big problem for Meta is that they don’t have a way to put a new AI-centric standalone product or service in front of users. That’s because the main platform owners - Microsoft for Windows, Google (or Samsung) for Android, and Apple for iOS/MacOS - will act in anti-competitive ways, push their own services, bundle their own services, and otherwise prevent others companies from getting a fair shot at the market. We see this already with Microsoft forcing Copilot buttons on every keyboard, effectively creating a default choice of Microsoft-owned AI services. It’s the 90s all over again, and at least in the US, competition laws are woefully inadequate and long overdue for a rewrite. So what’s someone like Meta going to do without that ability to freely compete for users? Apart from minor uses within their apps, I think they are just biding their time.
- unraveller 3y ago
  And it will likely also be a lesson in "performance oversupply", that is never releasing more value than you absolutely have to to stay ahead and always leave them gagging for more a.k.a. apple cult mode
unobatbayar 3y ago
How does Meta benefit from open-source Llama?
- matthews2 3y ago
  Maybe there isn’t a benefit to closed-source Llama either.
- omneity 3y ago
  It seems to me Meta is simply following the old startup tenet: build something people want. They were well positioned to pull this off with their AI/ML research resources and tradition of openness, and happen to have one of the biggest troves of data.
- Deverauxi 3y ago
  Well. Last time round we got our hands on 7B-70b models. Presumably, a company with as much compute as Meta can train even larger models that weren’t released, that would greatly benefit from the massive global efforts expended for free on the smaller open source LLM family tree of models. And their architecture ends up largely adopted by the open source community, helping build advanced tools to utilize those models.
- imjonse 3y ago
  Community goodwill to offset part of the negativity they deservedly receive for the shady aspects of their main products.
  - freedomben 3y ago
    Yes, but also I think it benefits them in the VR market with Quest headsets. It's a small market, but one that they have a huge lead on.

Seattle3503 3y ago

Is Llama remaining open an example of meta attempting to commoditize its complement?

deleted 3y ago
[deleted]
blackeyeblitzar 3y ago
In my opinion, no, since it isn’t clear how an LLM is a complement to Meta’s core businesses or who is affected by that commoditization. Maybe others can educate me about that. My view is that they have the key capabilities to build high performance LLMs - talent, cash (for hardware), data - so they might as well do it just to stay relevant. They may ultimately create innovations that are ahead of everyone else (like the JEPA architecture that Yann LeCun is pushing) but until then, at least they’re proving to users and investors that they still have bleeding edge technical capabilities. And sure they may find ways to use these models in new incremental features in their existing products (Facebook, Instagram, WhatsApp) - but they don’t have a great way to reach users even if they had new AI-centric products to offer (more below). At least they get some good marketing by putting open weight models out there as an alternative to big players pushing closed models and services. I think a big problem for Meta is that they don’t have a way to put a new AI-centric standalone product or service in front of users. That’s because the main platform owners - Microsoft for Windows, Google (or Samsung) for Android, and Apple for iOS/MacOS - will act in anti-competitive ways, push their own services, bundle their own services, and otherwise prevent others companies from getting a fair shot at the market. We see this already with Microsoft forcing Copilot buttons on every keyboard, effectively creating a default choice of Microsoft-owned AI services. It’s the 90s all over again, and at least in the US, competition laws are woefully inadequate and long overdue for a rewrite. So what’s someone like Meta going to do without that ability to freely compete for users? Apart from minor uses within their apps, I think they are just biding their time.
unraveller 3y ago
And it will likely also be a lesson in "performance oversupply", that is never releasing more value than you absolutely have to to stay ahead and always leave them gagging for more a.k.a. apple cult mode

unobatbayar 3y ago

How does Meta benefit from open-source Llama?

matthews2 3y ago
Maybe there isn’t a benefit to closed-source Llama either.
omneity 3y ago
It seems to me Meta is simply following the old startup tenet: build something people want. They were well positioned to pull this off with their AI/ML research resources and tradition of openness, and happen to have one of the biggest troves of data.
Deverauxi 3y ago
Well. Last time round we got our hands on 7B-70b models. Presumably, a company with as much compute as Meta can train even larger models that weren’t released, that would greatly benefit from the massive global efforts expended for free on the smaller open source LLM family tree of models. And their architecture ends up largely adopted by the open source community, helping build advanced tools to utilize those models.
imjonse 3y ago
Community goodwill to offset part of the negativity they deservedly receive for the shady aspects of their main products.
- freedomben 3y ago
  Yes, but also I think it benefits them in the VR market with Quest headsets. It's a small market, but one that they have a huge lead on.