3 ms·

I wish we got more substantial critique than ad hominems, but it is out there, and some of it’s really scathing. Essentially, it boils down to that the most pe

by marvin 3y ago

I wish we got more substantial critique than ad hominems, but it is out there, and some of it’s really scathing. Essentially, it boils down to that the most pessimist AI doom arguments are built on a stack of mind experiment hypotheses that are each far from certain to actually occur, or to be unsolvable if they do. And also that a significant portion of those holding those views fall in fallacious mind-traps that resemble a classical doomsday cult too similarly to ignore. All of these claims can and should be continuously investigated during future research, as both the hypothetical accidents and deliberate misuse definitely seem anything between possible and likely. But when the suggested solutions — totalitarian research bans and in the extreme, literal nuclear war causing near-extinction of humans (Yudkowsky deleted a tweet to this effect) — are so counterproductive, strongly worded rebuttals are appropriate. I do agree that those strongly-worded objections should be more prominent.

rafaelero 3y ago
Yeah, the thing about debating doomers is that there is no empirical data to ground the debate. They require us to abide to their framework on how things will run out but why should I? If you have no evidence then this is just a fun imaginative game to play, not a serious discussion. And no, this is not equivalent to the climate change debate. We have a very well established arrow of causality between carbon emission to higher temperatures to more extreme weather. With AI doomers we have increasing process power > rapid self-improving AI > rise of a will to exterminate humans that can't be contained by us > doom.
- pixl97 3y ago
  Hold on here, you act like this is without president and yet you are the testament to the dangers of a runaway intelligence, at least if you're a non-human animal or ecosystem on the planet. I mean there is already a well established causality between human expansion and extinction of species. If you were a large predator species on Earth, the time to fight and remove humans from the playing field we before they picked up their first tool. After that point it was too late. Now you have to answer some questions to see the potential trajectory. 1) Does intelligence begat intelligence? Can an intelligent optimize itself to become more intelligent? What are the limits of this? 2) Are humans the pinnacle of intelligence? Why would evolution optimize humans to be the most possible universal configuration for intelligence? Now, those questions just give us the 'is super intelligences possible' depending on your answers. But this is where you/they/most people in these arguments make a mistake >rise of a will to exterminate humans that can't be contained by us When a virus spreads over a species and kills large portion of a species does it have a will? I mean, I guess you could say the will is programmed in, but there is no larger intelligence occuring. This is where the parable of the paperclip optimizer comes in, said optimizers goal was never to kill humans, and yet via instrumental convergence that is what happens. Humanity is running headlong into a problem I do not know how we'll overcome. Unrestrained attempts to build super intelligence will lead from human extinction, to lessor versions where society falls apart and elects authoritarians. Otherwise we have the straight into authoritarianism option, which likely only buys us time as the authoritarians are just as apt to build super intelligence to consolidate their own power and control others.
  - rafaelero 3y ago
    Have you considered that we don't fucking know if AI will behave the same way as other types of organic life? This is not clear at all. Organic life on earth optimizes for reproduction, which implies AI systems will have very distinct features compared to humans. If you told me an intelligent alien made of organic matter subject to evolution is coming to Earth I would definitely be much more worried. That's not the case with computer programs.
    - pixl97 3y ago
      >Have you considered that we don't fucking know if AI will Very deeply. In fact this should be the very concern to you. AI must co-exist at least for some time on earth with humans, and likely other AI. Because AI will be a powerful resource it will be subject to the same kinds of attacks humans are. People trying to steal its resources or trick it into doing things it shouldn't. Intelligence develops defensive capabilities in order not to be abused. At the same time AIs will have evolutionary selection pressures from the people paying their power bills for at least some time. It should be obvious these selection pressures will be things like "make me more money" and "generate me more computing resources", this second one is very close to what we would consider reproduction.

by marvin 3y ago

rafaelero 3y ago

Yeah, the thing about debating doomers is that there is no empirical data to ground the debate. They require us to abide to their framework on how things will run out but why should I? If you have no evidence then this is just a fun imaginative game to play, not a serious discussion. And no, this is not equivalent to the climate change debate. We have a very well established arrow of causality between carbon emission to higher temperatures to more extreme weather. With AI doomers we have increasing process power > rapid self-improving AI > rise of a will to exterminate humans that can't be contained by us > doom.

pixl97 3y ago
Hold on here, you act like this is without president and yet you are the testament to the dangers of a runaway intelligence, at least if you're a non-human animal or ecosystem on the planet. I mean there is already a well established causality between human expansion and extinction of species. If you were a large predator species on Earth, the time to fight and remove humans from the playing field we before they picked up their first tool. After that point it was too late. Now you have to answer some questions to see the potential trajectory. 1) Does intelligence begat intelligence? Can an intelligent optimize itself to become more intelligent? What are the limits of this? 2) Are humans the pinnacle of intelligence? Why would evolution optimize humans to be the most possible universal configuration for intelligence? Now, those questions just give us the 'is super intelligences possible' depending on your answers. But this is where you/they/most people in these arguments make a mistake >rise of a will to exterminate humans that can't be contained by us When a virus spreads over a species and kills large portion of a species does it have a will? I mean, I guess you could say the will is programmed in, but there is no larger intelligence occuring. This is where the parable of the paperclip optimizer comes in, said optimizers goal was never to kill humans, and yet via instrumental convergence that is what happens. Humanity is running headlong into a problem I do not know how we'll overcome. Unrestrained attempts to build super intelligence will lead from human extinction, to lessor versions where society falls apart and elects authoritarians. Otherwise we have the straight into authoritarianism option, which likely only buys us time as the authoritarians are just as apt to build super intelligence to consolidate their own power and control others.
- rafaelero 3y ago
  Have you considered that we don't fucking know if AI will behave the same way as other types of organic life? This is not clear at all. Organic life on earth optimizes for reproduction, which implies AI systems will have very distinct features compared to humans. If you told me an intelligent alien made of organic matter subject to evolution is coming to Earth I would definitely be much more worried. That's not the case with computer programs.
  - pixl97 3y ago
    >Have you considered that we don't fucking know if AI will Very deeply. In fact this should be the very concern to you. AI must co-exist at least for some time on earth with humans, and likely other AI. Because AI will be a powerful resource it will be subject to the same kinds of attacks humans are. People trying to steal its resources or trick it into doing things it shouldn't. Intelligence develops defensive capabilities in order not to be abused. At the same time AIs will have evolutionary selection pressures from the people paying their power bills for at least some time. It should be obvious these selection pressures will be things like "make me more money" and "generate me more computing resources", this second one is very close to what we would consider reproduction.