4 ms·

I don't know, the pitch seemed generic? Seems to me that it's the team + market that won the money.

sheepscreek 3y ago
Elaborating on that: 1. The founding team’s track record and credentials. 2. there is currently no major independent player in the EU of OpenAI’s scale. 3. EU entities will prefer working with an EU based partner over OpenAI. Having the LLaMA tech lead is a win. Having someone ex-OpenAI can bolster their position even more. Now the anti-thesis: 1. It is gradually becoming apparent that models trained on high quality data don’t need to be so big. 2. phi-1 from Microsoft is < 1.3B parameters and can compete with GPT-3.5 reasonably well. 3. In addition, it seems GPT-4 is not one model, but a collection of 8 models trained on different datasets. Along with the above, all of a sudden - training GPT-4 like models is coming within the reach of an average person. All in - nobody has won this yet. And Meta open-sourcing their research was the cleverest strategic play that I’ve seen in a while. It has done so much to weaken their competitors while really pushing AI forward in an open way (so much for “OpenAI”).
Shrezzing 3y ago
> Seems to me that it's the team + market that won the money. That's pretty much it. Most analysts expect there's an addressable market of $20bn this year, growing to $100bn per year by the end of the decade. The market will be split between four or five companies. The number of people globally with the required expertise to create these models is in the tens (generously one hundred individuals). Three of those people are involved in this company now, and another five are joining in the next few weeks, each leaving senior positions at competitors. This one company has secured something in the order of 5-10% of the global talent pool for a $100bn/yr industry.
- user_named 3y ago
  I don't at all believe the people able to build models are that few. Aren't tons of people doing open source models already? I believe the number of people who have already built models to be low, but the number of people with the capability to build models to be very large. It's not magic.
  - whiplash451 3y ago
    The number of people who can actually pre-train a true LLM is very small. It remains a major feat with many tweaks and tricks. Case in point: the 114 pages of OPT175B logbook [1] [1] https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec...

Elaborating on that: 1. The founding team’s track record and credentials. 2. there is currently no major independent player in the EU of OpenAI’s scale. 3. EU entities will prefer working with an EU based partner over OpenAI. Having the LLaMA tech lead is a win. Having someone ex-OpenAI can bolster their position even more. Now the anti-thesis: 1. It is gradually becoming apparent that models trained on high quality data don’t need to be so big. 2. phi-1 from Microsoft is < 1.3B parameters and can compete with GPT-3.5 reasonably well. 3. In addition, it seems GPT-4 is not one model, but a collection of 8 models trained on different datasets. Along with the above, all of a sudden - training GPT-4 like models is coming within the reach of an average person. All in - nobody has won this yet. And Meta open-sourcing their research was the cleverest strategic play that I’ve seen in a while. It has done so much to weaken their competitors while really pushing AI forward in an open way (so much for “OpenAI”).

Shrezzing 3y ago

> Seems to me that it's the team + market that won the money. That's pretty much it. Most analysts expect there's an addressable market of $20bn this year, growing to $100bn per year by the end of the decade. The market will be split between four or five companies. The number of people globally with the required expertise to create these models is in the tens (generously one hundred individuals). Three of those people are involved in this company now, and another five are joining in the next few weeks, each leaving senior positions at competitors. This one company has secured something in the order of 5-10% of the global talent pool for a $100bn/yr industry.

user_named 3y ago
I don't at all believe the people able to build models are that few. Aren't tons of people doing open source models already? I believe the number of people who have already built models to be low, but the number of people with the capability to build models to be very large. It's not magic.
- whiplash451 3y ago
  The number of people who can actually pre-train a true LLM is very small. It remains a major feat with many tweaks and tricks. Case in point: the 114 pages of OPT175B logbook [1] [1] https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec...