6 ms·

Why is FB doing this. I am so perplexed. Like, I am still waiting for the "gotcha!". Purely to mess with MS?

krasin 3y ago
One possible explanation is to make PyTorch victory over TensorFlow/JAX even more apparent: with Google ceasing to release interesting open-source models, and Facebook / Meta continuing doing so (even if under a less permissive license than usual), it means that there's nothing interesting in the TensorFlow/JAX ecosystem, and a lot in the PyTorch one. (context: I work for Google DeepMind, opinions are my own)
- iudqnolq 3y ago
  Another possible reason might be to attract and retain talent? Speculating wildly but I'd guess some top researchers would value having their work be the industry standard rather than internal.
  - desperate 3y ago
    Your speculation is pretty accurate afaik. Mark commented that releasing open source models is what is needed to retain the best so it is what they will do (paraphrasing) on his interview with Lex Friedman.
- whimsicalism 3y ago
  Hmm, but I feel like Jax/Pytorch interop is only getting better. Tensorflow abstraction has already firmly lost.
  - krasin 3y ago
    > Hmm, but I feel like Jax/Pytorch interop is only getting better There's definitely some convergence. It's good. It still stands for a fact that most people do/think about PyTorch nowadays, not JAX (speaking as someone who likes JAX).
ilaksh 3y ago
Phind did the fine-tuning, not Facebook. This is actually an excellent demonstration of a good reason to open source something. Another company immediately made a dramatic improvement. Now anyone from Facebook or any companies they have invested in can benefit from that improvement and information about how they did it. Facebook could even take this exact model and add a coding assistant into Messenger if they wanted (not that they are interested in that). Or apply it to other things. OpenAI has a big lead. The most effective counter to that is actually to team up.
smoldesu 3y ago
Facebook has had an AI research lab for the better half of a decade now. Their open release model worked well in the past and it seems to be working well with Llama too. That looks like the extent of it. We should expect more FAANG-sized companied to do this, frankly. It's surprising to see people antipate a 'gotcha' considering how influental and transparent Pytorch was.
- chpatrick 3y ago
  Given that LLMs might be a generation-defining technology it's certainly weird that one of the biggest public companies is giving theirs away for free.
  - dragonwriter 3y ago
    Web browsers were also a generation-defining technology.
    - chpatrick 3y ago
      This would be more like Google giving away their search algorithm in 2000.
      - meiraleal 3y ago
        That would be it if it was OpenAI releasing GPT4. Facebook releasing Llama is like Google releasing Chrome.
  - thentherewere2 3y ago
    They're actually not giving it away for free. At least not LLAMA v2. Once a product successfully monetizes and gains over a certain number of users, an official license from Meta has to be negotiated otherwise Meta could revoke usage of the model. At least that's roughly what the license for LLAMAv2 implies.
    - ahahahahah 3y ago
elorant 3y ago
If Facebook were to implement a language model inside the platform now it would raise a lot of privacy issues. By publishing free to use models they can rely on others to do the work of training the society into using such tools at mass scale. So by the time they adopt in in all their services there will be much less backlash.
- ethbr1 3y ago
  Also, it's cheaper. Not just for the engineer-hours, but for the endless VP/PM political meetings arguing over this approach or that. Way cheaper to let the market decide, and then reincorporate the winner into their products. A winner which happens to already be based on a semi-compatible tech stack.
dontupvoteme 3y ago
Why wouldn't they? It is their best strategy. I didn't like them that much before, now I do.
drpancake 3y ago
They're in the business of encouraging users to post content to get more eyeballs and therefore more ad revenue. So commoditising a way of generating a lot more content is in their interests.
- ppsreejith 3y ago
  This. I recall in their earnings report, they gave a similar answer, I.e the more this technology is made accessible, the better the content put on social media becomes. They also pointed out that any improvements made can be incorporated into their stack like what happened with open compute and PyTorch.
gmerc 3y ago
It degrades Googles and OpenAis ability to monetize. That alone is worth gold because tech is not a blue ocean anymore - neither from a market opportunity nor from an investor perspective. Your enemies loss is your benefit. They can’t use money they don’t make to mount attacks to dislodge customers, investors, mindshare from you. Secondly it commoditizes AI. Zuck believes that his platforms ultimately will benefit if there is more content. Because his platforms sell the ability to show YOUR content over the rest, for coin. Just as with news/mobile games, driving the value of your content lower and lower by fostering the creation/supply of more of it is good for FB and bad for you. You have to advertise to rise above the noise. (And no it doesn’t really “affect users”) because they see the same amount of content (attention is finite), it’s entirely FBs discretion across all their surfaces what you see. Disclosure: Ex FB but not specific insights into current developments.
mythz 3y ago
> Purely to mess with MS? Unlikely as Microsoft is the preferred partner for Llama 2 [2]. It's not clear what Meta's end goal is, but large companies can loss lead investments to grab mind share then worry about Business models later, an approach which seems to be working well [2] where Meta went from not being in the leading AI companies conversation to being the company behind the models that most OSS innovation is happening around. At the very least they've already created a strong brand around Llama whose eyes are watching their every release - which is invaluable reach that other companies could ever hope to dream of, making it easy to promote and gain instant traction for new Llama based products (like this). [1] https://about.fb.com/news/2023/07/llama-2/ https://about.fb.com/news/2023/07/llama-2/ [2] https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322
- kkielhofner 3y ago
  My little theory is Microsoft actually learned a lesson from their experience going up against LAMP in the 90s. They got positively wrecked with their "Proprietary, $, closed-source Windows, IIS, ActiveX or whatever" approach. They release of ton of AI stuff themselves and partner on things like this. They're also big backers/investors in OpenAI and their own proprietary products, of course. I think they're taking that lesson and hedging their bets this time. It's like how a lot of corporations will donate to both political candidates (in the US) to guarantee they'll have access/influence regardless of how things shake out.
haxiel 3y ago
A previous post here on HN [1] discussed the leak of the original LLaMA model and Stable Diffusion. The article points out that the open-source community's effort resulted in more innovation, and an ecosystem built up around these models leading to rapid adoption. That gives more control to the original authors of the model against their competitors. I think Meta has recognized this and wants to keep their lead going. [1] https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322
- jiggawatts 3y ago
  They’re end-users of AI, they’re not vendors selling AI. Giving it away for free is not a loss to them.
- nwoli 3y ago
  They’re also already giving out millions of dollars in man hours in releasing and maintaining open source software, as well as lots of other much less used ML models. So they have an existing open source strategy and it makes sense to double down on this one since it’s clearly more impactful than a lot of the other ones
somenameforme 3y ago
I think a smart company plans for what they see the future as holding, as opposed to optimizing for what maximizing gains in the short term. And text generation programs seem to be following near to the exact same path as image generation programs did after DALL-E. So in the future we're probably not going to see any single centralized player, but rather a wide decentralization with many options. It seems in this future, providing a 'decentralization ready' to go package that people can use and deploy for all sorts of things is far smarter than blowing through billions of dollars (and goodwill) desperately trying to become a centralized behemoth when you have no moat, no network effect, and perhaps a temporary technical advantage at best. And while OpenAI has blown through a bunch of good will going from Open-ForSociety-AI to Closed-ForProfit-AI, Facebook has simultaneously gained a bunch of good will. What this translates to is really difficult to even imagine, but it's certainly desirable. And really I think this intangible of good will is something many modern corporations don't give enough value to, and it's hurting them - catching more flies with honey than vinegar, and all that.
quickthrower2 3y ago
> perplexed Apt choice of word

One possible explanation is to make PyTorch victory over TensorFlow/JAX even more apparent: with Google ceasing to release interesting open-source models, and Facebook / Meta continuing doing so (even if under a less permissive license than usual), it means that there's nothing interesting in the TensorFlow/JAX ecosystem, and a lot in the PyTorch one. (context: I work for Google DeepMind, opinions are my own)

iudqnolq 3y ago
Another possible reason might be to attract and retain talent? Speculating wildly but I'd guess some top researchers would value having their work be the industry standard rather than internal.
- desperate 3y ago
  Your speculation is pretty accurate afaik. Mark commented that releasing open source models is what is needed to retain the best so it is what they will do (paraphrasing) on his interview with Lex Friedman.
whimsicalism 3y ago
Hmm, but I feel like Jax/Pytorch interop is only getting better. Tensorflow abstraction has already firmly lost.
- krasin 3y ago
  > Hmm, but I feel like Jax/Pytorch interop is only getting better There's definitely some convergence. It's good. It still stands for a fact that most people do/think about PyTorch nowadays, not JAX (speaking as someone who likes JAX).

ilaksh 3y ago

Phind did the fine-tuning, not Facebook. This is actually an excellent demonstration of a good reason to open source something. Another company immediately made a dramatic improvement. Now anyone from Facebook or any companies they have invested in can benefit from that improvement and information about how they did it. Facebook could even take this exact model and add a coding assistant into Messenger if they wanted (not that they are interested in that). Or apply it to other things. OpenAI has a big lead. The most effective counter to that is actually to team up.

smoldesu 3y ago

Facebook has had an AI research lab for the better half of a decade now. Their open release model worked well in the past and it seems to be working well with Llama too. That looks like the extent of it. We should expect more FAANG-sized companied to do this, frankly. It's surprising to see people antipate a 'gotcha' considering how influental and transparent Pytorch was.

chpatrick 3y ago
Given that LLMs might be a generation-defining technology it's certainly weird that one of the biggest public companies is giving theirs away for free.
- dragonwriter 3y ago
  Web browsers were also a generation-defining technology.
  - chpatrick 3y ago
    This would be more like Google giving away their search algorithm in 2000.
    - meiraleal 3y ago
      That would be it if it was OpenAI releasing GPT4. Facebook releasing Llama is like Google releasing Chrome.
- thentherewere2 3y ago
  They're actually not giving it away for free. At least not LLAMA v2. Once a product successfully monetizes and gains over a certain number of users, an official license from Meta has to be negotiated otherwise Meta could revoke usage of the model. At least that's roughly what the license for LLAMAv2 implies.
  - ahahahahah 3y ago

elorant 3y ago

If Facebook were to implement a language model inside the platform now it would raise a lot of privacy issues. By publishing free to use models they can rely on others to do the work of training the society into using such tools at mass scale. So by the time they adopt in in all their services there will be much less backlash.

ethbr1 3y ago
Also, it's cheaper. Not just for the engineer-hours, but for the endless VP/PM political meetings arguing over this approach or that. Way cheaper to let the market decide, and then reincorporate the winner into their products. A winner which happens to already be based on a semi-compatible tech stack.

dontupvoteme 3y ago

Why wouldn't they? It is their best strategy. I didn't like them that much before, now I do.

drpancake 3y ago

They're in the business of encouraging users to post content to get more eyeballs and therefore more ad revenue. So commoditising a way of generating a lot more content is in their interests.

ppsreejith 3y ago
This. I recall in their earnings report, they gave a similar answer, I.e the more this technology is made accessible, the better the content put on social media becomes. They also pointed out that any improvements made can be incorporated into their stack like what happened with open compute and PyTorch.

gmerc 3y ago

It degrades Googles and OpenAis ability to monetize. That alone is worth gold because tech is not a blue ocean anymore - neither from a market opportunity nor from an investor perspective. Your enemies loss is your benefit. They can’t use money they don’t make to mount attacks to dislodge customers, investors, mindshare from you. Secondly it commoditizes AI. Zuck believes that his platforms ultimately will benefit if there is more content. Because his platforms sell the ability to show YOUR content over the rest, for coin. Just as with news/mobile games, driving the value of your content lower and lower by fostering the creation/supply of more of it is good for FB and bad for you. You have to advertise to rise above the noise. (And no it doesn’t really “affect users”) because they see the same amount of content (attention is finite), it’s entirely FBs discretion across all their surfaces what you see. Disclosure: Ex FB but not specific insights into current developments.

mythz 3y ago

> Purely to mess with MS? Unlikely as Microsoft is the preferred partner for Llama 2 [2]. It's not clear what Meta's end goal is, but large companies can loss lead investments to grab mind share then worry about Business models later, an approach which seems to be working well [2] where Meta went from not being in the leading AI companies conversation to being the company behind the models that most OSS innovation is happening around. At the very least they've already created a strong brand around Llama whose eyes are watching their every release - which is invaluable reach that other companies could ever hope to dream of, making it easy to promote and gain instant traction for new Llama based products (like this). [1] https://about.fb.com/news/2023/07/llama-2/ https://about.fb.com/news/2023/07/llama-2/ [2] https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322

kkielhofner 3y ago
My little theory is Microsoft actually learned a lesson from their experience going up against LAMP in the 90s. They got positively wrecked with their "Proprietary, $, closed-source Windows, IIS, ActiveX or whatever" approach. They release of ton of AI stuff themselves and partner on things like this. They're also big backers/investors in OpenAI and their own proprietary products, of course. I think they're taking that lesson and hedging their bets this time. It's like how a lot of corporations will donate to both political candidates (in the US) to guarantee they'll have access/influence regardless of how things shake out.

haxiel 3y ago

A previous post here on HN [1] discussed the leak of the original LLaMA model and Stable Diffusion. The article points out that the open-source community's effort resulted in more innovation, and an ecosystem built up around these models leading to rapid adoption. That gives more control to the original authors of the model against their competitors. I think Meta has recognized this and wants to keep their lead going. [1] https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322

jiggawatts 3y ago
They’re end-users of AI, they’re not vendors selling AI. Giving it away for free is not a loss to them.
nwoli 3y ago
They’re also already giving out millions of dollars in man hours in releasing and maintaining open source software, as well as lots of other much less used ML models. So they have an existing open source strategy and it makes sense to double down on this one since it’s clearly more impactful than a lot of the other ones

somenameforme 3y ago

I think a smart company plans for what they see the future as holding, as opposed to optimizing for what maximizing gains in the short term. And text generation programs seem to be following near to the exact same path as image generation programs did after DALL-E. So in the future we're probably not going to see any single centralized player, but rather a wide decentralization with many options. It seems in this future, providing a 'decentralization ready' to go package that people can use and deploy for all sorts of things is far smarter than blowing through billions of dollars (and goodwill) desperately trying to become a centralized behemoth when you have no moat, no network effect, and perhaps a temporary technical advantage at best. And while OpenAI has blown through a bunch of good will going from Open-ForSociety-AI to Closed-ForProfit-AI, Facebook has simultaneously gained a bunch of good will. What this translates to is really difficult to even imagine, but it's certainly desirable. And really I think this intangible of good will is something many modern corporations don't give enough value to, and it's hurting them - catching more flies with honey than vinegar, and all that.

quickthrower2 3y ago

> perplexed Apt choice of word