5 ms·

I find it quite extraordinary how many on here are dismissing that there is any risk at all. I also find statements like Yann LeCunn's that "The most common rea

I find it quite extraordinary how many on here are dismissing that there is any risk at all. I also find statements like Yann LeCunn's that "The most common reaction by AI researchers to these prophecies of doom is face palming." to be lacking in awareness. "Experts disagree on risk of extinction" isn't quite as reassuring as he thinks it is. The reality is, despite the opinions of the armchair quarterbacks commenting here, no-one in the world has any clue whether AGI is possible in the next twenty years, just as no-one predicted scaling up transformers would result in GPT-4.

kordlessagain 3y ago
Oh it’s possible and there’s absolutely nothing wrong with saying it’s possible without “proof” given that’s how all hypothesis starts. That said, the risk may exist but isn’t manifest yet, so being positive (as opposed to the scientific method which seeks to negate a truth of something) is just holding out hope.
JamesLeonis 3y ago
> I find it quite extraordinary how many on here are dismissing that there is any risk at all. The fear over AI is a displaced fear of unaccountable social structures with extinction-power that currently exist and we allow to continually exist. Without these structures AI is harmless to the species, even superintelligence. Your (reasonable) counter-argument might be that somebody (like, say, my dumb self) accidentally mixes their computers just right and creates an intelligence that escapes into the wild. The plot of Ex Machina is a reasonable stand-in for such an event. I am also going to assume the intelligence would desire to kill all humans. Either the AI would have to find already existing extinction-power in society, or it would need to build it. In either case the argument is against building extinction-power in the first place. My (admittedly cynical) take about this round of regulation is about several first-movers in AI to write legislation that is favorable to them and prevents any meaningful competition. ... Ok, enough cynicism. Lets talk some solutions. Nuclear weapons are an instructive case of both handling (or not) of extinction-power and the international diplomacy the world can engage to manage such a power. One example is the Outer Space Weapons Ban treaty - we can have a similar ban of AI in militaries. Politically one can reap benefits of deescalation and peaceful development, while logistically it prevents single-points-of-failure in a combat situation. Those points-of-failures sure are juicy targets for the opponent! As a consequence of these bans and treaties, institutions arose that monitor and regulate trans-national nuclear programs. AI can likewise have similar institutions. The promotion and sharing of information would prevent any country from gaining an advantage, and the inspections would deter their military application. This is only what I could come up with off the top of my head, but I hope it shows a window into the possibilities of meaningful political commitments towards AI.
- MrScruff 3y ago
  I don't really have a notion of whether an actual AGI would have a desire to kill all humans. I do however think that one entity seeking to create another entity that it can control, yet is more intelligent than it, seems arbitrarily challenging in the long run. I think having a moratorium on AI development will be impossible to enforce, and as you stretch the timeline out, these negative outcomes become increasingly likely as the technical barriers to entry continue to fall. I've personally assumed this for thirty years, the only difference now is that the timeline seems to be accelerating.
- PoignardAzur 3y ago
  > One example is the Outer Space Weapons Ban treaty - we can have a similar ban of AI in militaries It's extremely unclear, in fact, whether such a ban would be enforceable. Detecting outer space weapons is easy. Detecting whether a country is running advanced AIs in their datacenter is a lot harder.
juve1996 3y ago
What's more realistic: regulatory capture and US hegemony on AI or general intelligence destroying the world in the next 20 years? Go ahead and bet. I doubt you're putting your money on AGI.
- sebzim4500 3y ago
  Probably yeah but those aren't equally bad.
- MrScruff 3y ago
  I think it very unlikely you could call a coin flip ten times in a row, but I wouldn't want to bet my life savings on it.
- mitthrowaway2 3y ago
  Why would I put money on a bet that would only pay off when I, and everyone else, was dead?
  - deleted 3y ago
    [deleted]
  - juve1996 3y ago
    Sam Altman can bet right now. If he truly believes in this risk, he could bet his entire company, shut everything down, and lobby for a complete ban on AI research. If the outcome is certain death, this seems like a great bet to make.
    - mitthrowaway2 3y ago
      Indeed. It's probably what I would do if I were him. Or direct OpenAI entirely into AI safety research and stop doing capabilities research. I watched his interview with Lex Fridman,[1] and I didn't think he seemed very sincere. On the other hand I think there are a lot of people who are very sincere, like Max Tegmark.[2] [1] https://www.youtube.com/watch?v=L_Guz73e6fw&pp=ygUWbGV4IGZyaWRtYW4gc2FtIGFsdG1hbg%3D%3D https://www.youtube.com/watch?v=L_Guz73e6fw&pp=ygUWbGV4IGZya... [2] https://www.youtube.com/watch?v=VcVfceTsD0A&pp=ygUXbGV4IGZyaWRtYW4gbWF4IHRlZ21hcms%3D https://www.youtube.com/watch?v=VcVfceTsD0A&pp=ygUXbGV4IGZya...
meroes 3y ago
Anyone who sees digestion for example can’t be reduced to digital programs knows it’s far, far away. Actual AGI will require biology and psychology, not better programs.
deleted 3y ago
[deleted]
dmbche 3y ago
My main thing is that I can't find a simple explanation of what the exctinction risk actually is from "agi". It's all very vague and "handwavy". How is it going to kill us all? Why do we subsidize it if it is so dangerous? Almost all risks they mention would be better mitigated by making governemental use of any AI related system stop immediately as the risks are too high there (misapplication of the force of governement is much more dangerous than people playing around with chat gpt and being misled) and put on hold until dangers and benefits are better understood. Maybe keeping liscences for developpment and nationalising the labs doing it? Fines for anyone caught working on one?

kordlessagain 3y ago

Oh it’s possible and there’s absolutely nothing wrong with saying it’s possible without “proof” given that’s how all hypothesis starts. That said, the risk may exist but isn’t manifest yet, so being positive (as opposed to the scientific method which seeks to negate a truth of something) is just holding out hope.

JamesLeonis 3y ago

> I find it quite extraordinary how many on here are dismissing that there is any risk at all. The fear over AI is a displaced fear of unaccountable social structures with extinction-power that currently exist and we allow to continually exist. Without these structures AI is harmless to the species, even superintelligence. Your (reasonable) counter-argument might be that somebody (like, say, my dumb self) accidentally mixes their computers just right and creates an intelligence that escapes into the wild. The plot of Ex Machina is a reasonable stand-in for such an event. I am also going to assume the intelligence would desire to kill all humans. Either the AI would have to find already existing extinction-power in society, or it would need to build it. In either case the argument is against building extinction-power in the first place. My (admittedly cynical) take about this round of regulation is about several first-movers in AI to write legislation that is favorable to them and prevents any meaningful competition. ... Ok, enough cynicism. Lets talk some solutions. Nuclear weapons are an instructive case of both handling (or not) of extinction-power and the international diplomacy the world can engage to manage such a power. One example is the Outer Space Weapons Ban treaty - we can have a similar ban of AI in militaries. Politically one can reap benefits of deescalation and peaceful development, while logistically it prevents single-points-of-failure in a combat situation. Those points-of-failures sure are juicy targets for the opponent! As a consequence of these bans and treaties, institutions arose that monitor and regulate trans-national nuclear programs. AI can likewise have similar institutions. The promotion and sharing of information would prevent any country from gaining an advantage, and the inspections would deter their military application. This is only what I could come up with off the top of my head, but I hope it shows a window into the possibilities of meaningful political commitments towards AI.

MrScruff 3y ago
I don't really have a notion of whether an actual AGI would have a desire to kill all humans. I do however think that one entity seeking to create another entity that it can control, yet is more intelligent than it, seems arbitrarily challenging in the long run. I think having a moratorium on AI development will be impossible to enforce, and as you stretch the timeline out, these negative outcomes become increasingly likely as the technical barriers to entry continue to fall. I've personally assumed this for thirty years, the only difference now is that the timeline seems to be accelerating.
PoignardAzur 3y ago
> One example is the Outer Space Weapons Ban treaty - we can have a similar ban of AI in militaries It's extremely unclear, in fact, whether such a ban would be enforceable. Detecting outer space weapons is easy. Detecting whether a country is running advanced AIs in their datacenter is a lot harder.

juve1996 3y ago

What's more realistic: regulatory capture and US hegemony on AI or general intelligence destroying the world in the next 20 years? Go ahead and bet. I doubt you're putting your money on AGI.

sebzim4500 3y ago
Probably yeah but those aren't equally bad.
MrScruff 3y ago
I think it very unlikely you could call a coin flip ten times in a row, but I wouldn't want to bet my life savings on it.
mitthrowaway2 3y ago
Why would I put money on a bet that would only pay off when I, and everyone else, was dead?
- deleted 3y ago
  [deleted]
- juve1996 3y ago
  Sam Altman can bet right now. If he truly believes in this risk, he could bet his entire company, shut everything down, and lobby for a complete ban on AI research. If the outcome is certain death, this seems like a great bet to make.
  - mitthrowaway2 3y ago
    Indeed. It's probably what I would do if I were him. Or direct OpenAI entirely into AI safety research and stop doing capabilities research. I watched his interview with Lex Fridman,[1] and I didn't think he seemed very sincere. On the other hand I think there are a lot of people who are very sincere, like Max Tegmark.[2] [1] https://www.youtube.com/watch?v=L_Guz73e6fw&pp=ygUWbGV4IGZyaWRtYW4gc2FtIGFsdG1hbg%3D%3D https://www.youtube.com/watch?v=L_Guz73e6fw&pp=ygUWbGV4IGZya... [2] https://www.youtube.com/watch?v=VcVfceTsD0A&pp=ygUXbGV4IGZyaWRtYW4gbWF4IHRlZ21hcms%3D https://www.youtube.com/watch?v=VcVfceTsD0A&pp=ygUXbGV4IGZya...

meroes 3y ago

Anyone who sees digestion for example can’t be reduced to digital programs knows it’s far, far away. Actual AGI will require biology and psychology, not better programs.

deleted 3y ago

[deleted]

dmbche 3y ago

My main thing is that I can't find a simple explanation of what the exctinction risk actually is from "agi". It's all very vague and "handwavy". How is it going to kill us all? Why do we subsidize it if it is so dangerous? Almost all risks they mention would be better mitigated by making governemental use of any AI related system stop immediately as the risks are too high there (misapplication of the force of governement is much more dangerous than people playing around with chat gpt and being misled) and put on hold until dangers and benefits are better understood. Maybe keeping liscences for developpment and nationalising the labs doing it? Fines for anyone caught working on one?