3 ms·

Perhaps for now, but I wouldn't count on there always being a Meta spending $$$ to train enormous models and then giving them away for free. What's the long-ter

Perhaps for now, but I wouldn't count on there always being a Meta spending $$$ to train enormous models and then giving them away for free. What's the long-term game plan for open-source models when the corporate charity inevitably dries up?

mepian 3y ago
Maybe they are commoditizing their complement here.
- jsheard 3y ago
  Maybe, but at some point vague strategic moves like that will need to actually justify themselves at the earnings call. Throwing endless millions of dollars down the drain to what, depreciate the value of another companies product, which doesn't even directly compete with Metas main breadwinners? What do they actually get out of this? I suppose if there's any consolation for the open-source AI community it's that Meta has demonstrated a willingness to burn a lot of money for unclear benefit, they're still single-handedly keeping the VR industry on life support at the expense of about $4 billion per quarter. A decade on from acquiring Oculus and no closer to making it profitable, if Metas AI efforts get the same treatment then the free models will probably keep coming for a while.
  - suriyaG 3y ago
    Zuckerberg's reasoning on AI/ML seemed very justifiable to me in the Dwarkesh podcast IIRC, - Having a better model is a competitive advantage in fighting against spam - Better models enables facebook itself to understand their code vulnerabilities, better employee productivity etc. - Being in the frontier of open source, keeps them at an advantage in terms of updates from community
    - tyre 3y ago
      Also Meta is in the business of sharing content. Instagram, Facebook, Threads, and WhatsApp are all about sharing content. They’re highly incentivized to expand the options for generating content. OpenAI and Anthropic have to make money from the generation, which raises the barrier to creation. Meta can say fuck it, let everyone create and well sell ads against what they create.
sharkjacobs 3y ago
Unless Meta’s bet is that, going forward, models themselves won’t be the competitive differentiator, that it will be about integration. They can give away Llama3,4,5 for free, because no one else can put them Whatsapp or whatever. idk
jeffnappi 3y ago
Building open models is a very strong approach to cornering the market on top tier AI researchers. And as other commenters have mentioned, the raw models are not the product - the vast majority of the value is in how they are integrated into useful products.
- _giorgio_ 3y ago
  Exactly, a free standalone AI is an unsustainable product. You can't offer it for free, unless you make money in another way.
ozr 3y ago
If your product is an AI model (OpenAI, Anthropic, etc) you can't give it away for free. If your product is a social graph w/ ads (Meta), you can. It's hardly corporate charity: * Meta releasing these models creates an improvement and tuning ecosystem around it, giving them access to tons of free developer time. * It's also a strong recruiting tool, for engineers and researchers frustrated by, e.g., Google and OpenAI becoming increasingly closed. They know they can publish at Meta. * The cost is insignificant. Meta had 30B in revenue just in Q2 2023.
- drakenot 3y ago
  Commoditize your complement and all that.
- faeriechangling 3y ago
  It's great PR. I hear people refer to Mark Zuckerberg as a "real engineer" and other such platitudes and it's doing wonders to reduce the stigma around Meta amongst the tech savvy minority. Despite Meta being very similar to Google in terms of incentives, and Google nowadays being decidedly uncool. Doesn't hurt that Mark shipped something worth a damn whereas Google has been floundering for ages.
  - talldayo 3y ago
    With credit to Google, they were basically neck-and-neck with Facebook AI Research in a lot of ways. Both were publishing text transformers (BERT vs FastText), both were maintaining SOTA inference libraries (Tensorflow vs Pytorch) and both were investing heavily into researching the field further. I'd even argue that Google was the largest contributor to making open-source AI more like Linux and less like a shitty proprietary product. There's a whole history of recent machine-learning development where both Google and Facebook have worked together and against each other to push things forward. I think it's entirely mistaken to characterize Google as the understudy when in many ways it's the other way around.
    - faeriechangling 3y ago
seydor 3y ago
They spent much more on metaverse
polio 3y ago
The corporate charity will not dry up. AI makes it easier to generate content, and Meta's in the business of facilitating the sharing of that content. Content is surface area for ads. AI will also make the virtual realities of the "metaverse", as defined by Mark, easier to reify. It's also a giant marketing and recruiting strategy.
- jsheard 3y ago
  AI does make it easier to generate content, but the type of content it lends itself towards the most is spam. Whether a Facebook where the majority of the content is neural net slurry is something that people will want to engage with once they realise what's going on is an open question I think. Anecdotally it seems like older demographics are the prime target of the current wave of AI engagement farming on Facebook, because they just don't understand that this technology exists now and assume that all of the "photos" they're seeing are real.
lunfardl 3y ago
I guess many here can think a lot 5D chess business strategies. For me it is just Zuck trying to reach greatness, down in history his name will prob pop up when people think about who brought LLM/AI/AGI? to the masses.
dragonwriter 3y ago
> What’s the long-term game plan for open-source models when the corporate charity inevitably dries up? Once open models reach and stay at near parity for a while, it’ll make sense for commercial downstream users to support open source community efforts rather than building their own, same as has happened in many other categories of key infrastructure software.

mepian 3y ago

Maybe they are commoditizing their complement here.

jsheard 3y ago
Maybe, but at some point vague strategic moves like that will need to actually justify themselves at the earnings call. Throwing endless millions of dollars down the drain to what, depreciate the value of another companies product, which doesn't even directly compete with Metas main breadwinners? What do they actually get out of this? I suppose if there's any consolation for the open-source AI community it's that Meta has demonstrated a willingness to burn a lot of money for unclear benefit, they're still single-handedly keeping the VR industry on life support at the expense of about $4 billion per quarter. A decade on from acquiring Oculus and no closer to making it profitable, if Metas AI efforts get the same treatment then the free models will probably keep coming for a while.
- suriyaG 3y ago
  Zuckerberg's reasoning on AI/ML seemed very justifiable to me in the Dwarkesh podcast IIRC, - Having a better model is a competitive advantage in fighting against spam - Better models enables facebook itself to understand their code vulnerabilities, better employee productivity etc. - Being in the frontier of open source, keeps them at an advantage in terms of updates from community
  - tyre 3y ago
    Also Meta is in the business of sharing content. Instagram, Facebook, Threads, and WhatsApp are all about sharing content. They’re highly incentivized to expand the options for generating content. OpenAI and Anthropic have to make money from the generation, which raises the barrier to creation. Meta can say fuck it, let everyone create and well sell ads against what they create.

sharkjacobs 3y ago

Unless Meta’s bet is that, going forward, models themselves won’t be the competitive differentiator, that it will be about integration. They can give away Llama3,4,5 for free, because no one else can put them Whatsapp or whatever. idk

jeffnappi 3y ago

Building open models is a very strong approach to cornering the market on top tier AI researchers. And as other commenters have mentioned, the raw models are not the product - the vast majority of the value is in how they are integrated into useful products.

_giorgio_ 3y ago
Exactly, a free standalone AI is an unsustainable product. You can't offer it for free, unless you make money in another way.

ozr 3y ago

If your product is an AI model (OpenAI, Anthropic, etc) you can't give it away for free. If your product is a social graph w/ ads (Meta), you can. It's hardly corporate charity: * Meta releasing these models creates an improvement and tuning ecosystem around it, giving them access to tons of free developer time. * It's also a strong recruiting tool, for engineers and researchers frustrated by, e.g., Google and OpenAI becoming increasingly closed. They know they can publish at Meta. * The cost is insignificant. Meta had 30B in revenue just in Q2 2023.

drakenot 3y ago
Commoditize your complement and all that.
faeriechangling 3y ago
It's great PR. I hear people refer to Mark Zuckerberg as a "real engineer" and other such platitudes and it's doing wonders to reduce the stigma around Meta amongst the tech savvy minority. Despite Meta being very similar to Google in terms of incentives, and Google nowadays being decidedly uncool. Doesn't hurt that Mark shipped something worth a damn whereas Google has been floundering for ages.
- talldayo 3y ago
  With credit to Google, they were basically neck-and-neck with Facebook AI Research in a lot of ways. Both were publishing text transformers (BERT vs FastText), both were maintaining SOTA inference libraries (Tensorflow vs Pytorch) and both were investing heavily into researching the field further. I'd even argue that Google was the largest contributor to making open-source AI more like Linux and less like a shitty proprietary product. There's a whole history of recent machine-learning development where both Google and Facebook have worked together and against each other to push things forward. I think it's entirely mistaken to characterize Google as the understudy when in many ways it's the other way around.
  - faeriechangling 3y ago

seydor 3y ago

They spent much more on metaverse

polio 3y ago

The corporate charity will not dry up. AI makes it easier to generate content, and Meta's in the business of facilitating the sharing of that content. Content is surface area for ads. AI will also make the virtual realities of the "metaverse", as defined by Mark, easier to reify. It's also a giant marketing and recruiting strategy.

jsheard 3y ago
AI does make it easier to generate content, but the type of content it lends itself towards the most is spam. Whether a Facebook where the majority of the content is neural net slurry is something that people will want to engage with once they realise what's going on is an open question I think. Anecdotally it seems like older demographics are the prime target of the current wave of AI engagement farming on Facebook, because they just don't understand that this technology exists now and assume that all of the "photos" they're seeing are real.

lunfardl 3y ago

I guess many here can think a lot 5D chess business strategies. For me it is just Zuck trying to reach greatness, down in history his name will prob pop up when people think about who brought LLM/AI/AGI? to the masses.

dragonwriter 3y ago

> What’s the long-term game plan for open-source models when the corporate charity inevitably dries up? Once open models reach and stay at near parity for a while, it’ll make sense for commercial downstream users to support open source community efforts rather than building their own, same as has happened in many other categories of key infrastructure software.