5 ms·

Big Tech hasnt dominated anything. They got trounced by a 400 person startup. Alexa, Google Search, FB research all spend untold money on ML systems. Neither is

Big Tech hasnt dominated anything. They got trounced by a 400 person startup. Alexa, Google Search, FB research all spend untold money on ML systems. Neither is releasing game changing products anywhere close to OpenAI.

neel8986 4y ago
Yes and tech was so path breaking and innovative that the company sold its 49% share for 10 billion dollar /s. Sam Altman is extremely experienced in the VC world and could have easily raised billions for potential value. He wouldn't have sold half of his company within 6 months of releasing chatGPT if there was any thing remotely defensible in the product
galaxytachyon 4y ago
OpenAI was backed by probably the biggest tech company around, Microsoft. Without the computing afforded by Azure, they would have problems training and running the LLM.
tomComb 4y ago
It is true that big companies have brands and existing revenue stream to protect (Google), so maybe only a little company with nothing to lose could do what OpenAI did. And maybe MS found the perfect arrangement giving them an ability to adjust how closely to associate themself with the tech according to need. But is this all necessarily a good thing? OpenAI gave MS access to GPT and MS clearly wants to use GPT to destroy Google, so Google now has respond aggressively, and now we have exactly the AI arms race that many predicted would lead to the worst outcomes.
- pydry 4y ago
  In that case they might have picked the worst time to push through layoffs.
  - shitpostbot 4y ago
    [dead]
xigency 4y ago
I disagree. All of the big players have been involved in some way, creating their own systems (which are quite competitive) as well as laying much of the foundation for existing models with research they published.
danbrooks 4y ago
I wouldn't say big tech was "trounced" by OpenAI. Advances in LLMs are driven by the research community. OpenAI is a small, well-resourced organization with the sole purpose of quickly deploying and hyping state of the art AI models. Big tech naturally has a slower product development cycle - and will make full use of LLMs in due time.
- rcme 4y ago
  > I wouldn't say big tech was "trounced" by OpenAI. I mean not a single “big tech” company is offering any LLM products not powered by Open AI. It’s been almost 3 years since GPT-3 was released. I’m not sure they can catch up at this point. And I think the GPs point was a good one: big tech threw billions and billions at personal assistants. Nothing innovative was delivered.
  - 6gvONxR4sf7o 4y ago
    Aren’t google search and google translate powered by transformer models?
    - rcme 4y ago
      Probably, but using a transformer isn’t Open AI’s advantage. A widely available language model with “emergent abilities” is Open AI’s advantage. The fact that it’s a transformer is more of an implementation detail.
Karrot_Kream 4y ago
The Transformer architecture that powers GPT models is based on a now-famous Google Brain paper, Attention is All You Need. You're right that OpenAI has productized this research much faster than Big Tech, but the research itself came from established R&D in Academia and Big Tech.
- nr2x 4y ago
  Yes, but only one of the authors is still at google.
  - taneq 4y ago
    So if you work on a kickass product at FlorpCorp and then leave, FlorpCorp didn’t develop that product?
    - nr2x 4y ago
      Yes, the people are where the value is, not a dumb fuck like Sundar.
    - ethbr0 4y ago
      Not if FlorpCorp didn't release it.
- pclmulqdq 4y ago
  Well, Google had to figure out how to cancel itself somehow. That's the only Google project, aside from search, that has had staying power.
- yreg 4y ago
  And Stable Diffusion is based on Google's Imagen paper.
  - 6gvONxR4sf7o 4y ago
    The diffusion papers most analogous to the transformer paper are from Berkeley, Stanford, and Freiburg. Imagen wasn’t foundational in the way you’re suggesting.
Forestessential 4y ago
I cant even discuss this without going powerlevel 9000+
lwhi 4y ago
I think you forget what money and connections bring. Anything that can be provided by a micro or small sized company can be provided by billion dollar corporations too. Maybe size brings a drawback, in the same way a mega ship finds it difficult to turn around without a day's notice — but these companies can and will. Microsoft once betted all chips on the internet being fad; look at them now.

neel8986 4y ago

Yes and tech was so path breaking and innovative that the company sold its 49% share for 10 billion dollar /s. Sam Altman is extremely experienced in the VC world and could have easily raised billions for potential value. He wouldn't have sold half of his company within 6 months of releasing chatGPT if there was any thing remotely defensible in the product

galaxytachyon 4y ago

OpenAI was backed by probably the biggest tech company around, Microsoft. Without the computing afforded by Azure, they would have problems training and running the LLM.

tomComb 4y ago

It is true that big companies have brands and existing revenue stream to protect (Google), so maybe only a little company with nothing to lose could do what OpenAI did. And maybe MS found the perfect arrangement giving them an ability to adjust how closely to associate themself with the tech according to need. But is this all necessarily a good thing? OpenAI gave MS access to GPT and MS clearly wants to use GPT to destroy Google, so Google now has respond aggressively, and now we have exactly the AI arms race that many predicted would lead to the worst outcomes.

pydry 4y ago
In that case they might have picked the worst time to push through layoffs.
- shitpostbot 4y ago
  [dead]

xigency 4y ago

I disagree. All of the big players have been involved in some way, creating their own systems (which are quite competitive) as well as laying much of the foundation for existing models with research they published.

danbrooks 4y ago

I wouldn't say big tech was "trounced" by OpenAI. Advances in LLMs are driven by the research community. OpenAI is a small, well-resourced organization with the sole purpose of quickly deploying and hyping state of the art AI models. Big tech naturally has a slower product development cycle - and will make full use of LLMs in due time.

rcme 4y ago
> I wouldn't say big tech was "trounced" by OpenAI. I mean not a single “big tech” company is offering any LLM products not powered by Open AI. It’s been almost 3 years since GPT-3 was released. I’m not sure they can catch up at this point. And I think the GPs point was a good one: big tech threw billions and billions at personal assistants. Nothing innovative was delivered.
- 6gvONxR4sf7o 4y ago
  Aren’t google search and google translate powered by transformer models?
  - rcme 4y ago
    Probably, but using a transformer isn’t Open AI’s advantage. A widely available language model with “emergent abilities” is Open AI’s advantage. The fact that it’s a transformer is more of an implementation detail.

Karrot_Kream 4y ago

The Transformer architecture that powers GPT models is based on a now-famous Google Brain paper, Attention is All You Need. You're right that OpenAI has productized this research much faster than Big Tech, but the research itself came from established R&D in Academia and Big Tech.

nr2x 4y ago
Yes, but only one of the authors is still at google.
- taneq 4y ago
  So if you work on a kickass product at FlorpCorp and then leave, FlorpCorp didn’t develop that product?
  - nr2x 4y ago
    Yes, the people are where the value is, not a dumb fuck like Sundar.
  - ethbr0 4y ago
    Not if FlorpCorp didn't release it.
pclmulqdq 4y ago
Well, Google had to figure out how to cancel itself somehow. That's the only Google project, aside from search, that has had staying power.
yreg 4y ago
And Stable Diffusion is based on Google's Imagen paper.
- 6gvONxR4sf7o 4y ago
  The diffusion papers most analogous to the transformer paper are from Berkeley, Stanford, and Freiburg. Imagen wasn’t foundational in the way you’re suggesting.

Forestessential 4y ago

I cant even discuss this without going powerlevel 9000+

lwhi 4y ago

I think you forget what money and connections bring. Anything that can be provided by a micro or small sized company can be provided by billion dollar corporations too. Maybe size brings a drawback, in the same way a mega ship finds it difficult to turn around without a day's notice — but these companies can and will. Microsoft once betted all chips on the internet being fad; look at them now.