12 ms·

> We release all our models to the research community. This is yet more evidence for the "AI isn't a competitive advantage" thesis. State-of-the-Art is a publi

> We release all our models to the research community. This is yet more evidence for the "AI isn't a competitive advantage" thesis. State-of-the-Art is a public resource, so competing with AI offers no "moat".

jstx1 4y ago
The research is open, you still need 1) to have a good idea for a product that people want 2) lots of talent and resources to actually build and run everything
- mjburgess 4y ago
  And then facebook will release a version which beats it ? :)
- ilaksh 4y ago
  Actually no it's not available for commercial purposes or for anyone they do not approve.
minimaxir 4y ago
Business orgs have been finetuning open-source models like these on their own internal data to create a moat since BERT in 2018.
- Der_Einzige 4y ago
  Glad to see you still on HN! You've done amazing work in this domain! I'd argue that this goes further back to the word2vec/glove days too. I was working for a company in 2018 who leveraged my skills for fine-tuning word2vec/fasttext even before BERT/attention is all you need paper.
fwlr 4y ago
In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_ to make and release competitive models, effectively filling in the moat by giving it to everyone.
- recuter 4y ago
  I don't see such grand stratagems as being a likely explanation. It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given as nobody can realistically manage AI researches. Or at least, it seems that much like in political governance, events are suddenly outpacing the ability of corporate to react. In the case of OpenAI it is a "nudge humanity into a more wholesome direction" because a lot of them went on an acid fueled toxic affective altruism bender. And in this case it is "release all the things while Zuck is still obsessed with VR --for science!". I like this other group better. But it is disturbing that it can stop at any moment. Probably why they are doing it, while they still can.
  - fwlr 4y ago
    The researchers and engineers and other assorted dorks who built it weren’t thinking about moats, for sure, I agree with you there. But I guarantee you that the metaphor of medieval warfare was on the minds of the executives and legal team deciding whether to let the team release their work in this way.
  - titzer 4y ago
    They almost certainly spent at least a few million dollars on this research project. Hard to say when the decision was made to open source this (from the outset or after it started showing results), but the decision was conscious and calculated. Nothing this high-profile is going to escape the strategic decision making processes of upper management.
  - mliker 4y ago
    this is a pretty biased and uninformed opinion. Pretty condescending to call it a "bunch of dorks...running around unsupervised."
    - recuter 4y ago
      I don't know what my bias is supposed to be but I called them dorks affectionately for one. The other replies are literally arguing that they are very extremely supervised where as I am speculating they are just eager to share their work for the right reasons and the eye of Sauron has yet to turn upon them. Inside knowledge I never claimed. Anything else I can help you with today? :)
layer8 4y ago
They are only releasing it on a case-by-case basis and with a non-commercial license.
cardine 4y ago
And yet there are still no publicly available models that could actually compete with ChatGPT. I'm not even talking about RLHF (although data like that is also a huge moat) - just simple things like larger context sizes. There are still plenty of AI advantages to be had if you go just a little bit outside of what is currently possible with off the shelf models.
lr1970 4y ago
> We release all our models to the research community. And from the FB blogpost [0] "Request Form Thank you for your interest in Meta AI’s LLaMA (Large Language Model Meta AI) models. To request access to the models, please fill out this form, and we'll review and let you know if your use case is approved. The information you provide below will be used solely to assess eligibility to access these models." So much for "releasing" the model for research community. [0] https://ai.facebook.com/blog/large-language-model-llama-meta-ai/ https://ai.facebook.com/blog/large-language-model-llama-meta...

jstx1 4y ago

The research is open, you still need 1) to have a good idea for a product that people want 2) lots of talent and resources to actually build and run everything

mjburgess 4y ago
And then facebook will release a version which beats it ? :)
ilaksh 4y ago
Actually no it's not available for commercial purposes or for anyone they do not approve.

minimaxir 4y ago

Business orgs have been finetuning open-source models like these on their own internal data to create a moat since BERT in 2018.

Der_Einzige 4y ago
Glad to see you still on HN! You've done amazing work in this domain! I'd argue that this goes further back to the word2vec/glove days too. I was working for a company in 2018 who leveraged my skills for fine-tuning word2vec/fasttext even before BERT/attention is all you need paper.

fwlr 4y ago

In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_ to make and release competitive models, effectively filling in the moat by giving it to everyone.

recuter 4y ago
I don't see such grand stratagems as being a likely explanation. It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given as nobody can realistically manage AI researches. Or at least, it seems that much like in political governance, events are suddenly outpacing the ability of corporate to react. In the case of OpenAI it is a "nudge humanity into a more wholesome direction" because a lot of them went on an acid fueled toxic affective altruism bender. And in this case it is "release all the things while Zuck is still obsessed with VR --for science!". I like this other group better. But it is disturbing that it can stop at any moment. Probably why they are doing it, while they still can.
- fwlr 4y ago
  The researchers and engineers and other assorted dorks who built it weren’t thinking about moats, for sure, I agree with you there. But I guarantee you that the metaphor of medieval warfare was on the minds of the executives and legal team deciding whether to let the team release their work in this way.
- titzer 4y ago
  They almost certainly spent at least a few million dollars on this research project. Hard to say when the decision was made to open source this (from the outset or after it started showing results), but the decision was conscious and calculated. Nothing this high-profile is going to escape the strategic decision making processes of upper management.
- mliker 4y ago
  this is a pretty biased and uninformed opinion. Pretty condescending to call it a "bunch of dorks...running around unsupervised."
  - recuter 4y ago
    I don't know what my bias is supposed to be but I called them dorks affectionately for one. The other replies are literally arguing that they are very extremely supervised where as I am speculating they are just eager to share their work for the right reasons and the eye of Sauron has yet to turn upon them. Inside knowledge I never claimed. Anything else I can help you with today? :)

layer8 4y ago

They are only releasing it on a case-by-case basis and with a non-commercial license.

cardine 4y ago

And yet there are still no publicly available models that could actually compete with ChatGPT. I'm not even talking about RLHF (although data like that is also a huge moat) - just simple things like larger context sizes. There are still plenty of AI advantages to be had if you go just a little bit outside of what is currently possible with off the shelf models.

lr1970 4y ago

> We release all our models to the research community. And from the FB blogpost [0] "Request Form Thank you for your interest in Meta AI’s LLaMA (Large Language Model Meta AI) models. To request access to the models, please fill out this form, and we'll review and let you know if your use case is approved. The information you provide below will be used solely to assess eligibility to access these models." So much for "releasing" the model for research community. [0] https://ai.facebook.com/blog/large-language-model-llama-meta-ai/ https://ai.facebook.com/blog/large-language-model-llama-meta...