3 ms·

LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't

LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product, it just means that they're competing against a company funding open source stuff for (presumably) strategic reasons, like in many other markets. And like in many other markets over time it's possible the proprietary versions will establish permanent competitive advantage. Meta may not keep releasing Llamas forever. > you would just eventually train and deploy your own model Just train an LLM? It's really not that easy! Even if it was, it'd be like how people can "just" run their own email service. Hosted email API is not rocket science but in practice companies all choose to pay Microsoft or Google to do it. Doing these things isn't a core competitive advantage so it gets outsourced.

throwaway2037 2y ago
> The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product Real question: Why are so many LLMs given away for free? Are they hoping to crush non-free alternatives? EDIT Your last paragraph makes an excellent point. In the near future, I could see big corps paying OpenAI (or a competitor) to train a private LLM on their squillion internal documents and build a very good helpdesk agent. (Legal and compliance would love it.)
- mike_hearn 2y ago
  Paying OpenAI for fine tunes on internal docs is already happening. Giving expensive things away for free is a great marketing technique that has been used since time immemorial, so why startups like Stability do it is somewhat understandable. And OpenAI uses free API access as a loss leader for their API product so that's understandable too. Why Meta/Google/others do open weight releases is a bit less clear. Recall though that the first Llama wasn't really an open source release. You had to sign a document saying you were a researcher to get the weights, and that document was an agreement to keep the weights secret. Two people signed the documents, anonymously compared their weights, discovered they weren't watermarked (i.e. Meta didn't take this seriously, it was a sop to their AI politics/safety people) and promptly leaked them. Presumably this was useful for the more libertarian wing of Meta as they could then prove the sky wouldn't fall, and so the influence shifted towards those arguing for more openness in research in general. With that Rubicon crossed other companies didn't see competitive advantage in withholding their similar sized models anymore and followed the leader, so to speak. Sometimes it also feels like Meta may have over-purchased GPUs and - lacking a public cloud - have just decided to let their researchers do what they wanted. Which is great for the public! But we mustn't be too overconfident. This is really only possible because of Zuckerberg's unique corporate structure that makes him unfirable, combined with Meta being a big data company. It's really benefiting all of humanity here because he's invulnerable to board action so doesn't have to worry about heat from shareholders over 'wasting' money like this. There's a lot of R&D being done right now on shrinking models whilst preserving quality, so hopefully the Zuck's generosity is enough to ride the open AI research community through the hard times when you needed billions to train LLMs.

throwaway2037 2y ago

> The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product Real question: Why are so many LLMs given away for free? Are they hoping to crush non-free alternatives? EDIT Your last paragraph makes an excellent point. In the near future, I could see big corps paying OpenAI (or a competitor) to train a private LLM on their squillion internal documents and build a very good helpdesk agent. (Legal and compliance would love it.)

mike_hearn 2y ago
Paying OpenAI for fine tunes on internal docs is already happening. Giving expensive things away for free is a great marketing technique that has been used since time immemorial, so why startups like Stability do it is somewhat understandable. And OpenAI uses free API access as a loss leader for their API product so that's understandable too. Why Meta/Google/others do open weight releases is a bit less clear. Recall though that the first Llama wasn't really an open source release. You had to sign a document saying you were a researcher to get the weights, and that document was an agreement to keep the weights secret. Two people signed the documents, anonymously compared their weights, discovered they weren't watermarked (i.e. Meta didn't take this seriously, it was a sop to their AI politics/safety people) and promptly leaked them. Presumably this was useful for the more libertarian wing of Meta as they could then prove the sky wouldn't fall, and so the influence shifted towards those arguing for more openness in research in general. With that Rubicon crossed other companies didn't see competitive advantage in withholding their similar sized models anymore and followed the leader, so to speak. Sometimes it also feels like Meta may have over-purchased GPUs and - lacking a public cloud - have just decided to let their researchers do what they wanted. Which is great for the public! But we mustn't be too overconfident. This is really only possible because of Zuckerberg's unique corporate structure that makes him unfirable, combined with Meta being a big data company. It's really benefiting all of humanity here because he's invulnerable to board action so doesn't have to worry about heat from shareholders over 'wasting' money like this. There's a lot of R&D being done right now on shrinking models whilst preserving quality, so hopefully the Zuck's generosity is enough to ride the open AI research community through the hard times when you needed billions to train LLMs.