5 ms·

> Basically the prime bet that they made (that one needs extremely expensive hardware to have useful AI) has already failed. I thought the prime bet was that t

> Basically the prime bet that they made (that one needs extremely expensive hardware to have useful AI) has already failed. I thought the prime bet was that the winning lab who reaches takeoff through recursive self improvement will make a galactic superintelligence. Not saying I believe this but the people running the labs do. Under this scenario if you are a few months behind at the pivotal time you might as well not exist at all.

zem 6mo ago
only if said galactic superintelligence takes immediate steps to kill all its potential competitors, or hoover up all the world's resources, or some other aggressively zero sum thing. otherwise I don't see what difference it makes down the line of you have the second superintelligence rather than the first. and that's under the assumption that you can create a superintelligence that will continue to slavishly serve your agenda rather than establishing and following its own goals.
- sroussey 6mo ago
  One could argue that AI has already started to hoover up all the world’s resources. AI buildout as a percent of GDP is already high and still rising.
  - munk-a 6mo ago
    Don't blame machines for our folly. This is just standard bubble behavior.
    - pocksuppet 6mo ago
      What if that's just the mechanism the machines take over the world? Natural selection doesn't care why something replicated a lot.
- fwipsy 6mo ago
  Anthropic/OpenAI aren't planning to have their superintelligence take over the world, but they're still afraid that someone else will do it.
- zozbot234 6mo ago
  If OpenAI has the second superintelligence they have to merge with the first and cooperate. It's a provision in their charter.
  - airstrike 6mo ago
    I'm not sure anyone thinks their charter carries much weight at this point.
- ethin 6mo ago
  This is also assuming that AGI is even possible. So far there is no evidence that this is actually doable over anything but billions of years (and even then we have no idea how nature really managed it). Edit: Meant to say AGI (superintelligence didn't make sense). Superintelligence is undefinable at the moment so even considering if it's possible or not is more of a philosophical thing/si-fi thought experiment than anything else.
  - zem 6mo ago
    oh absolutely, no argument there, the case for AGI is pretty weak. I was just saying that I am even more sceptical that any of this is a "first or nothing" scenario - that is one of my biggest pet peeves about the entire tech sector.
    - ethin 6mo ago
      Right, but I never said it was a first-or-nothing scenario to begin with. Given that both AGI and ASI are so ambiguous as to be nothingburgers, talking about them is just a performative thought experiment IMO. An interesting one, certainly, but neither are even remotely close to being realized. Until we have some kind of clear definition that can be scientifically proven and reproduced, that will remain the case.
      - zem 6mo ago
        the original comment I was replying to said "Under this scenario if you are a few months behind at the pivotal time you might as well not exist at all."
  - josephg 6mo ago
    ASI is the acronym you’re looking for. It stands for Artificial Superintelligence. Arguably it’s already here. ChatGPT knows more than any human who has ever lived. It can carry out millions of conversations at once. And it has better working memory (“context”) than humans. And it can speak and write code much faster than humans. Humans still have some advantages: Specialists are smarter than chatgpt in most domains. We’re better at using imagination. We understand the physical world better. But it seems like we’re watching the gap close in real time. A few years ago chatgpt could barely program. Now you can give it complex prompts and it can write large, complex programs which mostly work. If you extrapolate forward, is there any good reason to think humans will retain a lead?
- dullcrisp 6mo ago
  Well no because no one is going to be coming in to work building the next AI model after the Singularity. We’ll all be bblbrvkxn46?/4!gfbxf’mgv5fhxtgcsgjcucz to buvtcibycuvinovrYdyvuctYcrzuvhxh gcuch7…:!
ahepp 6mo ago
One thing I don’t understand about this viewpoint (which I understand isn’t your own): why does one benefit so tremendously from getting there a month before competitors? I’m sure having a month of superintelligence with no competition would be lucrative, but do they think achieving superintelligence first will impede competitors from also achieving it a month later?
- Philpax 6mo ago
  A month with a superintelligence at your hands could be quite impactful, especially if you're willing to break the law / normal operating decorum in the pursuit of protecting what you have. A superintelligence, if wielded so, could destroy your competitors in a great many ways, including the relatively-benign solution of outcompeting them, to exploiting them and tearing them apart from the inside. A genuine superintelligence is a very, very scary thing to have under the control of one person or organisation.
  - remexre 6mo ago
    If I interpret "a machine superintelligence" as "a classroom of 300IQ humans," I'm not really sure how this is true? You still have material and energy constraints, you can't think your way out of those.
    - Philpax 6mo ago
      For the concrete problem we're discussing, you can hack your competitors out of existence, replace all of your knowledge workers to shed costs, hyperoptimise your logistics, etc. It's not just intelligence, it's speed and scale. Bostrom's Superintelligence (2014) is a bit of a dreary read, and I didn't finish it, but it pulls no punches about the leverage that a superintelligence might have in our highly-connected world.
      - vkou 6mo ago
        > For the concrete problem we're discussing, you can hack your competitors out of existence, replace all of your knowledge workers to shed costs, hyperoptimise your logistics, etc. It's not just intelligence, it's speed and scale. For the concrete problem we're discussing, that hypothetical belongs in a Marvel movie, not reality. In the real world, you can't 'hack your competitors out of existence', and you'll be going to prison very quickly for trying this sort of thing.
stkdump 6mo ago
I don't think this race to superintelligence idea should be taken too seriously. It is great for headlines and get peoples imaginations up. It is mostly a marketing gag. I look at superintelligence this way: software engineering used to be considered amoung the most mentally demanding jobs one can have. And in this field more and more people give up large parts of their job and become approximately product managers to let the machine do the engineering part. So we are about there. Who cares that there are some puzzles in some "synthetic" benchmark in which humans outsmart AIs?
- ashdksnndck 6mo ago
  The people in that community have been talking about superintelligence for decades and it’s part of an ideology. It’s not some recently-invented story for headlines.
pocksuppet 6mo ago
That's just what they told the gullible investors to get money.
jaspanglia 6mo ago
[dead]

zem 6mo ago

only if said galactic superintelligence takes immediate steps to kill all its potential competitors, or hoover up all the world's resources, or some other aggressively zero sum thing. otherwise I don't see what difference it makes down the line of you have the second superintelligence rather than the first. and that's under the assumption that you can create a superintelligence that will continue to slavishly serve your agenda rather than establishing and following its own goals.

sroussey 6mo ago
One could argue that AI has already started to hoover up all the world’s resources. AI buildout as a percent of GDP is already high and still rising.
- munk-a 6mo ago
  Don't blame machines for our folly. This is just standard bubble behavior.
  - pocksuppet 6mo ago
    What if that's just the mechanism the machines take over the world? Natural selection doesn't care why something replicated a lot.
fwipsy 6mo ago
Anthropic/OpenAI aren't planning to have their superintelligence take over the world, but they're still afraid that someone else will do it.
zozbot234 6mo ago
If OpenAI has the second superintelligence they have to merge with the first and cooperate. It's a provision in their charter.
- airstrike 6mo ago
  I'm not sure anyone thinks their charter carries much weight at this point.
ethin 6mo ago
This is also assuming that AGI is even possible. So far there is no evidence that this is actually doable over anything but billions of years (and even then we have no idea how nature really managed it). Edit: Meant to say AGI (superintelligence didn't make sense). Superintelligence is undefinable at the moment so even considering if it's possible or not is more of a philosophical thing/si-fi thought experiment than anything else.
- zem 6mo ago
  oh absolutely, no argument there, the case for AGI is pretty weak. I was just saying that I am even more sceptical that any of this is a "first or nothing" scenario - that is one of my biggest pet peeves about the entire tech sector.
  - ethin 6mo ago
    Right, but I never said it was a first-or-nothing scenario to begin with. Given that both AGI and ASI are so ambiguous as to be nothingburgers, talking about them is just a performative thought experiment IMO. An interesting one, certainly, but neither are even remotely close to being realized. Until we have some kind of clear definition that can be scientifically proven and reproduced, that will remain the case.
    - zem 6mo ago
      the original comment I was replying to said "Under this scenario if you are a few months behind at the pivotal time you might as well not exist at all."
- josephg 6mo ago
  ASI is the acronym you’re looking for. It stands for Artificial Superintelligence. Arguably it’s already here. ChatGPT knows more than any human who has ever lived. It can carry out millions of conversations at once. And it has better working memory (“context”) than humans. And it can speak and write code much faster than humans. Humans still have some advantages: Specialists are smarter than chatgpt in most domains. We’re better at using imagination. We understand the physical world better. But it seems like we’re watching the gap close in real time. A few years ago chatgpt could barely program. Now you can give it complex prompts and it can write large, complex programs which mostly work. If you extrapolate forward, is there any good reason to think humans will retain a lead?
dullcrisp 6mo ago
Well no because no one is going to be coming in to work building the next AI model after the Singularity. We’ll all be bblbrvkxn46?/4!gfbxf’mgv5fhxtgcsgjcucz to buvtcibycuvinovrYdyvuctYcrzuvhxh gcuch7…:!

ahepp 6mo ago

One thing I don’t understand about this viewpoint (which I understand isn’t your own): why does one benefit so tremendously from getting there a month before competitors? I’m sure having a month of superintelligence with no competition would be lucrative, but do they think achieving superintelligence first will impede competitors from also achieving it a month later?

Philpax 6mo ago
A month with a superintelligence at your hands could be quite impactful, especially if you're willing to break the law / normal operating decorum in the pursuit of protecting what you have. A superintelligence, if wielded so, could destroy your competitors in a great many ways, including the relatively-benign solution of outcompeting them, to exploiting them and tearing them apart from the inside. A genuine superintelligence is a very, very scary thing to have under the control of one person or organisation.
- remexre 6mo ago
  If I interpret "a machine superintelligence" as "a classroom of 300IQ humans," I'm not really sure how this is true? You still have material and energy constraints, you can't think your way out of those.
  - Philpax 6mo ago
    For the concrete problem we're discussing, you can hack your competitors out of existence, replace all of your knowledge workers to shed costs, hyperoptimise your logistics, etc. It's not just intelligence, it's speed and scale. Bostrom's Superintelligence (2014) is a bit of a dreary read, and I didn't finish it, but it pulls no punches about the leverage that a superintelligence might have in our highly-connected world.
    - vkou 6mo ago
      > For the concrete problem we're discussing, you can hack your competitors out of existence, replace all of your knowledge workers to shed costs, hyperoptimise your logistics, etc. It's not just intelligence, it's speed and scale. For the concrete problem we're discussing, that hypothetical belongs in a Marvel movie, not reality. In the real world, you can't 'hack your competitors out of existence', and you'll be going to prison very quickly for trying this sort of thing.

stkdump 6mo ago

I don't think this race to superintelligence idea should be taken too seriously. It is great for headlines and get peoples imaginations up. It is mostly a marketing gag. I look at superintelligence this way: software engineering used to be considered amoung the most mentally demanding jobs one can have. And in this field more and more people give up large parts of their job and become approximately product managers to let the machine do the engineering part. So we are about there. Who cares that there are some puzzles in some "synthetic" benchmark in which humans outsmart AIs?

ashdksnndck 6mo ago
The people in that community have been talking about superintelligence for decades and it’s part of an ideology. It’s not some recently-invented story for headlines.

pocksuppet 6mo ago

That's just what they told the gullible investors to get money.

jaspanglia 6mo ago

[dead]