3 ms·

You are misreading my argument. I’m saying Facebook is degrading google and openai investor narrative. If Llama cost hypothetical one billion, they inflict a mu

by gmerc 3y ago

You are misreading my argument. I’m saying Facebook is degrading google and openai investor narrative. If Llama cost hypothetical one billion, they inflict a multiple on that on their competitors with this move while gaining massive technological advantages. The improvements made to llama by open source community people already have propelled it past Bard by many accounts and this is a model that a few months ago was absolutely non competitive and downright bad. So it’s a win win. I don’t see the problem

smoldesu 3y ago
Facebook has been open-sourcing AI research longer than OpenAI has even had the concept of an "investor narrative". I struggle to understand how someone could jump to the conclusion of this being a "scorched earth" maneuver with so many other reasonable explanations. Facebook has a laboratory (FAIR) with a long history of research and releases like this. > If Llama cost hypothetical one billion, they inflict a multiple on that on their competitors with this move while gaining massive technological advantages. If Llama cost a hypothetical one billion, then they amortized the cost over the value of the end product and the free advertisement alone. Maybe their competitors got scooped, but GPT-3 and GPT-4 haven't gone anywhere. Not to mention, there were lots of other language models from FAANG before Llama arrived. It's not like those were made and released to spite their competitors; it was research. Google and Microsoft have lots of open Transformer research you can find. Inflicting "damage" and gaining massive technological advantages is quite literally not their goal nor what they've done for the past half-decade. If it is, they've done a terrible job so far by collaborating with Microsoft to open their model format and provide inferencing acceleration for outdated hardware platforms. > The improvements made to llama by open source community people already have propelled it past Bard by many accounts and this is a model that a few months ago was absolutely non competitive and downright bad. This is something the original Llama paper acknowledged before the community "discovered" it: > In this section, we show that briefly finetuning on instructions data rapidly leads to improvements on MMLU. Although the non-finetuned version of LLaMA-65B is already able to follow basic instructions, we observe that a very small amount of finetuning improves the performance on MMLU, and further improves the ability of the model to follow instructions. https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf > So it’s a win win. I don’t see the problem Neither does Meta, nor Microsoft, nor Google, who have all been content to work on progressive and open AI research. Who do you perceive as their "competitors"? Each other?

by gmerc 3y ago

smoldesu 3y ago

Facebook has been open-sourcing AI research longer than OpenAI has even had the concept of an "investor narrative". I struggle to understand how someone could jump to the conclusion of this being a "scorched earth" maneuver with so many other reasonable explanations. Facebook has a laboratory (FAIR) with a long history of research and releases like this. > If Llama cost hypothetical one billion, they inflict a multiple on that on their competitors with this move while gaining massive technological advantages. If Llama cost a hypothetical one billion, then they amortized the cost over the value of the end product and the free advertisement alone. Maybe their competitors got scooped, but GPT-3 and GPT-4 haven't gone anywhere. Not to mention, there were lots of other language models from FAANG before Llama arrived. It's not like those were made and released to spite their competitors; it was research. Google and Microsoft have lots of open Transformer research you can find. Inflicting "damage" and gaining massive technological advantages is quite literally not their goal nor what they've done for the past half-decade. If it is, they've done a terrible job so far by collaborating with Microsoft to open their model format and provide inferencing acceleration for outdated hardware platforms. > The improvements made to llama by open source community people already have propelled it past Bard by many accounts and this is a model that a few months ago was absolutely non competitive and downright bad. This is something the original Llama paper acknowledged before the community "discovered" it: > In this section, we show that briefly finetuning on instructions data rapidly leads to improvements on MMLU. Although the non-finetuned version of LLaMA-65B is already able to follow basic instructions, we observe that a very small amount of finetuning improves the performance on MMLU, and further improves the ability of the model to follow instructions. https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf > So it’s a win win. I don’t see the problem Neither does Meta, nor Microsoft, nor Google, who have all been content to work on progressive and open AI research. Who do you perceive as their "competitors"? Each other?