9 ms·

Big tech and the pursuit of AI dominance

71 points by MoSattler 4y ago

MoSattler 4y ago
https://archive.ph/ot8EH https://archive.ph/ot8EH
HPMOR 4y ago
"in which Microsoft is implanting machine intelligence" I thought it was extremely adept change of referring to it as machine intelligence instead of artificial. There's nothing artificial about this intelligence, merely different. I think this will probably be increasingly more correct moving forward with additional advances.
- fnordpiglet 4y ago
  As I’m sure someone will point out LLMs aren’t intelligent. But then again, I don’t think humans are either.
  - acchow 4y ago
    Probably half of Americans today would fail the Turing Test.
    - Yoric 4y ago
      Probably all of Americans today would fail one half of Americans on the Turing Test. Not all Americans the same half, of course.
    - galaxytachyon 4y ago
      Probably half of Americans and some government officials will fail the US citizenship test so not much to expect from the general populace...
  - peteradio 4y ago
    No you are wrong this is good and getting gooder.
gsatic 4y ago
They will not dominate anything cuz of the number of useless teams within all fighting each other about who should own it. Big tech is going the way of IBM and good riddance.
- cardosof 4y ago
  This. Some folks will spend their lives trying to find faster horses and build lighter carriages, and all is good and well until someone shows up with a car.
13years 4y ago
It will be interesting to see how long anyone can dominate. AI might actually result in actually the opposite. Meaning that the era of years and decades of dominance is not sustainable any longer. Why would this be? Because AI allows you to copy your competitor in ways that was never possible before. It essentially becomes a skill and technology replicator. For example, look how Alpaca was able to somewhat replicate the multi million dollar models with $600 dollar cost.
- crop_rotation 4y ago
  But Alpaca was possible only after Llama, and Llama is not gonna happen with 600 or 60000 USD.
  - bogwog 4y ago
    I don't have the link, but someone estimated that training a Llama clone from scratch would cost around ~$80k. That's pocket change for most corporations and some individuals.
    - nickthegreek 4y ago
      And low enough to be crowdsourced for the right project.
- acchow 4y ago
  The data that you train your model on is certainly a huge moat...
  - sacnoradhq 4y ago
    The other moat is tech to capture feedback with actionable metadata from users.
  - 13years 4y ago
    It is a moat to be the first. The question is can you defend that moat.
crop_rotation 4y ago
The prospect of not having your own LLM would be frightening to any big tech CEO. LLMs might be a massive productivity boost very soon (IMHO GPT4 is already there), and if you don't have one, but your competitor does, you will either have to use their product or perish.
fdgsdfogijq 4y ago
Big Tech hasnt dominated anything. They got trounced by a 400 person startup. Alexa, Google Search, FB research all spend untold money on ML systems. Neither is releasing game changing products anywhere close to OpenAI.
- neel8986 4y ago
  Yes and tech was so path breaking and innovative that the company sold its 49% share for 10 billion dollar /s. Sam Altman is extremely experienced in the VC world and could have easily raised billions for potential value. He wouldn't have sold half of his company within 6 months of releasing chatGPT if there was any thing remotely defensible in the product
- galaxytachyon 4y ago
  OpenAI was backed by probably the biggest tech company around, Microsoft. Without the computing afforded by Azure, they would have problems training and running the LLM.
- tomComb 4y ago
  It is true that big companies have brands and existing revenue stream to protect (Google), so maybe only a little company with nothing to lose could do what OpenAI did. And maybe MS found the perfect arrangement giving them an ability to adjust how closely to associate themself with the tech according to need. But is this all necessarily a good thing? OpenAI gave MS access to GPT and MS clearly wants to use GPT to destroy Google, so Google now has respond aggressively, and now we have exactly the AI arms race that many predicted would lead to the worst outcomes.
  - pydry 4y ago
    In that case they might have picked the worst time to push through layoffs.
    - shitpostbot 4y ago
      [dead]
gumby 4y ago
I already have (before GPT 3 or 4) a plug in for Zoom which summarizes zoom calls, makes transcripts, and does all sorts of other things I don't care about (e.g. tries to gauge engagement and sentiment). It's pretty handy. Of course like everything these days it's a cloud silo: the transcripts, videos etc are in their cloud. There's no way to have them simply posted to slack channels so they can be found later in searches etc. The economic models and thinking of 2023 are seemingly still stuck in the PC era rather than being focused on the needs of the customer.
- mostlysimilar 4y ago
  What do you mean by the “PC” era? At least with PCs as the dominant computing platform we had files and programs separate from each other. Smartphones began the trend of app-bound data.
  - gumby 4y ago
    The PC era was marked by app-specific file formats and little interchange. To me, when the PC was released it was a shock as I already used to (somewhat) interchangeable file formats on the networked machines we used back then. By the late 90s there were some interchangable formats for photos and audio at least, though often you still had to install plug ins, drivers, codecs etc for various proprietary formats especially video. Basically a silo situation as bad as today.
christkv 4y ago
I’m sure we will get model foundry businesses where we can buy a model and a runtime to run the models ourselves very soon.
Forestessential 4y ago
It is like reading the news or a study paper but you know it is correct because a robot is telling you. This is the closest thing to a prophet people will have. Go downvote me.
nico 4y ago
Dominance is such a charged word. We have been corrupted by words like that. Why should we care about dominance? We should care about collaboration and people’s well being. Barring a catastrophe, AI is not going to stop. At the current rate, by next month we will have some previously-never-seen model/algorithm/chip/tech that will run locally on our phones and computers. At that point is game over for the big server-model companies. After that… well, pretty much all companies will be at stake. There’s a much bigger revolution happening right now than just the tech industry. We will be forced to ask ourselves what it means to be human. No company is going to dominate AI, AI is going to dominate AI.
- atleastoptimal 4y ago
  AI will dominate but only on huge distributed compute clusters. Local will be able to run models at reasonable speeds only when they're 4 years old.
  - thatjoeoverthr 4y ago
    Four years will come and go.
  - yreg 4y ago
    Doesn't the recent development (Stable Diffusion, LLaMA) tell us that these things actually run surprisingly well locally and that the models can be compressed much further? And aren't we finding out that the amount of parameters seems to matter less than we used to believe?
  - xyzzy123 4y ago
    All indications are that you can scale LLMs up & down pretty smoothly, I bet there will be lots of workflows in future using AI on pc-or mobile scale hardware which can call out to cloud AI for "deep thinking" or to double check stuff. It seems likely there will be AIs specialised in taxes or medicine, people/systems will consult multiple AIs to smooth out errors and look for consensus etc. In short, I agree with OP that we will probably end up with something that looks more like an ecosystem than a hegemony.
  - 4y ago
samspenc 4y ago
> The $11bn that Microsoft has reportedly put into Openai would, at the startup’s latest rumoured valuation of $29bn, give the software giant a stake of 38%. That's not quite accurate, Microsoft first invested $1 billion in OpenAI in 2019 (four years ago) and another $10 billion a few months ago when it was valued at $29B. So it's latest $10B investment gives it a 35% ownership, but it's unclear how much the $1B 2019 investment gave Microsoft. Some news reports peg Microsoft's current ownership at 49% of OpenAI, which means Microsoft's 2019 investment gave it 14% of the company (which would further indicate that OpenAI was valued at $7-8 billion at that time).

71 points by MoSattler 4y ago

https://archive.ph/ot8EH https://archive.ph/ot8EH

"in which Microsoft is implanting machine intelligence" I thought it was extremely adept change of referring to it as machine intelligence instead of artificial. There's nothing artificial about this intelligence, merely different. I think this will probably be increasingly more correct moving forward with additional advances.

fnordpiglet 4y ago
As I’m sure someone will point out LLMs aren’t intelligent. But then again, I don’t think humans are either.
- acchow 4y ago
  Probably half of Americans today would fail the Turing Test.
  - Yoric 4y ago
    Probably all of Americans today would fail one half of Americans on the Turing Test. Not all Americans the same half, of course.
  - galaxytachyon 4y ago
    Probably half of Americans and some government officials will fail the US citizenship test so not much to expect from the general populace...
- peteradio 4y ago
  No you are wrong this is good and getting gooder.

gsatic 4y ago

They will not dominate anything cuz of the number of useless teams within all fighting each other about who should own it. Big tech is going the way of IBM and good riddance.

cardosof 4y ago
This. Some folks will spend their lives trying to find faster horses and build lighter carriages, and all is good and well until someone shows up with a car.

13years 4y ago

It will be interesting to see how long anyone can dominate. AI might actually result in actually the opposite. Meaning that the era of years and decades of dominance is not sustainable any longer. Why would this be? Because AI allows you to copy your competitor in ways that was never possible before. It essentially becomes a skill and technology replicator. For example, look how Alpaca was able to somewhat replicate the multi million dollar models with $600 dollar cost.

crop_rotation 4y ago
But Alpaca was possible only after Llama, and Llama is not gonna happen with 600 or 60000 USD.
- bogwog 4y ago
  I don't have the link, but someone estimated that training a Llama clone from scratch would cost around ~$80k. That's pocket change for most corporations and some individuals.
  - nickthegreek 4y ago
    And low enough to be crowdsourced for the right project.
acchow 4y ago
The data that you train your model on is certainly a huge moat...
- sacnoradhq 4y ago
  The other moat is tech to capture feedback with actionable metadata from users.
- 13years 4y ago
  It is a moat to be the first. The question is can you defend that moat.

crop_rotation 4y ago

The prospect of not having your own LLM would be frightening to any big tech CEO. LLMs might be a massive productivity boost very soon (IMHO GPT4 is already there), and if you don't have one, but your competitor does, you will either have to use their product or perish.

fdgsdfogijq 4y ago

Big Tech hasnt dominated anything. They got trounced by a 400 person startup. Alexa, Google Search, FB research all spend untold money on ML systems. Neither is releasing game changing products anywhere close to OpenAI.

neel8986 4y ago
Yes and tech was so path breaking and innovative that the company sold its 49% share for 10 billion dollar /s. Sam Altman is extremely experienced in the VC world and could have easily raised billions for potential value. He wouldn't have sold half of his company within 6 months of releasing chatGPT if there was any thing remotely defensible in the product
galaxytachyon 4y ago
OpenAI was backed by probably the biggest tech company around, Microsoft. Without the computing afforded by Azure, they would have problems training and running the LLM.
tomComb 4y ago
It is true that big companies have brands and existing revenue stream to protect (Google), so maybe only a little company with nothing to lose could do what OpenAI did. And maybe MS found the perfect arrangement giving them an ability to adjust how closely to associate themself with the tech according to need. But is this all necessarily a good thing? OpenAI gave MS access to GPT and MS clearly wants to use GPT to destroy Google, so Google now has respond aggressively, and now we have exactly the AI arms race that many predicted would lead to the worst outcomes.
- pydry 4y ago
  In that case they might have picked the worst time to push through layoffs.
  - shitpostbot 4y ago
    [dead]

gumby 4y ago

I already have (before GPT 3 or 4) a plug in for Zoom which summarizes zoom calls, makes transcripts, and does all sorts of other things I don't care about (e.g. tries to gauge engagement and sentiment). It's pretty handy. Of course like everything these days it's a cloud silo: the transcripts, videos etc are in their cloud. There's no way to have them simply posted to slack channels so they can be found later in searches etc. The economic models and thinking of 2023 are seemingly still stuck in the PC era rather than being focused on the needs of the customer.

mostlysimilar 4y ago
What do you mean by the “PC” era? At least with PCs as the dominant computing platform we had files and programs separate from each other. Smartphones began the trend of app-bound data.
- gumby 4y ago
  The PC era was marked by app-specific file formats and little interchange. To me, when the PC was released it was a shock as I already used to (somewhat) interchangeable file formats on the networked machines we used back then. By the late 90s there were some interchangable formats for photos and audio at least, though often you still had to install plug ins, drivers, codecs etc for various proprietary formats especially video. Basically a silo situation as bad as today.

christkv 4y ago

I’m sure we will get model foundry businesses where we can buy a model and a runtime to run the models ourselves very soon.

Forestessential 4y ago

It is like reading the news or a study paper but you know it is correct because a robot is telling you. This is the closest thing to a prophet people will have. Go downvote me.

nico 4y ago

Dominance is such a charged word. We have been corrupted by words like that. Why should we care about dominance? We should care about collaboration and people’s well being. Barring a catastrophe, AI is not going to stop. At the current rate, by next month we will have some previously-never-seen model/algorithm/chip/tech that will run locally on our phones and computers. At that point is game over for the big server-model companies. After that… well, pretty much all companies will be at stake. There’s a much bigger revolution happening right now than just the tech industry. We will be forced to ask ourselves what it means to be human. No company is going to dominate AI, AI is going to dominate AI.

atleastoptimal 4y ago
AI will dominate but only on huge distributed compute clusters. Local will be able to run models at reasonable speeds only when they're 4 years old.
- thatjoeoverthr 4y ago
  Four years will come and go.
- yreg 4y ago
  Doesn't the recent development (Stable Diffusion, LLaMA) tell us that these things actually run surprisingly well locally and that the models can be compressed much further? And aren't we finding out that the amount of parameters seems to matter less than we used to believe?
- xyzzy123 4y ago
  All indications are that you can scale LLMs up & down pretty smoothly, I bet there will be lots of workflows in future using AI on pc-or mobile scale hardware which can call out to cloud AI for "deep thinking" or to double check stuff. It seems likely there will be AIs specialised in taxes or medicine, people/systems will consult multiple AIs to smooth out errors and look for consensus etc. In short, I agree with OP that we will probably end up with something that looks more like an ecosystem than a hegemony.
- 4y ago

samspenc 4y ago

> The $11bn that Microsoft has reportedly put into Openai would, at the startup’s latest rumoured valuation of $29bn, give the software giant a stake of 38%. That's not quite accurate, Microsoft first invested $1 billion in OpenAI in 2019 (four years ago) and another $10 billion a few months ago when it was valued at $29B. So it's latest $10B investment gives it a 35% ownership, but it's unclear how much the $1B 2019 investment gave Microsoft. Some news reports peg Microsoft's current ownership at 49% of OpenAI, which means Microsoft's 2019 investment gave it 14% of the company (which would further indicate that OpenAI was valued at $7-8 billion at that time).