4 ms·

Okay, here's the thing. It's certainly possible that strong AI could exist. Pretty much the only way of finding out is by attempting to build it. If it's possib

Okay, here's the thing. It's certainly possible that strong AI could exist. Pretty much the only way of finding out is by attempting to build it. If it's possible, we'll be able to build it eventually. If not, we won't. If the magical singularity happens, it'll flow naturally from the strong AI. And if it doesn't, it doesn't. And, well, if we get strong AI but no singularity, great, we still get strong AI and we can use it to do some very clever shit that makes the sort of things the AI community are doing now seem primitive. Given this, I'm not sure why singularity promoters give that much of a shit whether people believe it. I'm certainly skeptical of it. Given I pretty much use a 40 year old text editor to write in a language that's about 20 years old and is just about approaching the discoveries that the Lispies made back in the 50s, I'm not sure what progress we're talking about. I write code that basically instructs a computer with pretty much the same level of semantic complexity as one does when giving instructions to a particularly stupid child. But perhaps I've got this wrong. Perhaps the singularity will happen. I don't understand why the singularity promoters are so angry about people who are skeptical of their claims. While I was doing the rounds on Wikipedia a while back, I found this image... https://commons.wikimedia.org/wiki/File:Singularity_Deniers_Dystopia_Holocaust_Denial_LOW_RESOLUTION.png https://commons.wikimedia.org/wiki/File:Singularity_Deniers_... Have a look at the image description before I cleaned it up: https://commons.wikimedia.org/w/index.php?title=File:Singularity_Deniers_Dystopia_Holocaust_Denial_LOW_RESOLUTION.png&diff=69988810&oldid=57242687 https://commons.wikimedia.org/w/index.php?title=File:Singula... Apparently, if someone tells me that a super-intelligent AI will create a brain many hundreds of times better than our own and then allow us to live forever and my reaction isn't an immediate "oh my god, you are so, so right!" then I'm a 'singularity denier', equivalent in status to a holocaust denier. If the singularity idea is pretty much bound to happen as a result of emergent technological progress or whatever, why is my doxastic consent such an important requirement for it coming about? I freely admit I'm probably not as smart a guy as Ray Kurzweil. But let's not forget: the history of bizarre cults shows us that smart people can believe some very stupid things.

loup-vaillant 14y ago
> Given this, I'm not sure why singularity promoters give that much of a shit whether people believe it. In my case that would be because the default scenario for a technological singularity is Eternal Doom. By default, if AI researchers do not pay extreme attention, the first smarter-than-human AI will have different goals than we do, and recognize us as a threat to those goals and eliminate us all. Or squash us like bugs without even paying much attention. Or use our constituent atoms to maximize its goals. Or keep us drugged and lobotomised in soft cages, so that we're safe and happy. Or something. And of course, we won't be able to stop it, because it will outsmart us at every turn (we can do the same to chimps). This is why this singularity thing is at least worth looking into. Being able to nuke away our civilization is bad enough, but this could be even worse. Or it could turn our world into a paradise. And of course, with more believers, there will be more donors to the organizations working on this, more researchers working on it… So belief does count, to some extent. (On the other hand, becoming angry at non-believers like a crusader is unlikely to spur positive reactions. By cleaning up this image description, you actually helped our cause.)
- jpadkins 14y ago
  It's not the first smarter than human AI that is the threat. It's the first self-improving AI that people have to worry about. It can start out at a much lower than human intelligence (whatever that means) but as long it can keep improving it could turn into the nightmare scenario. The safeguards need to be built into the self improving AI, no matter how 'dumb' it starts out as.
  - loup-vaillant 14y ago
    Well, any path that lead to superhuman intelligence is dangerous. Self-improving AI may be the likeliest, but it's not the only one. Think Upload, brain-computer interfaces, or computing overhang, for instance.

loup-vaillant 14y ago

> Given this, I'm not sure why singularity promoters give that much of a shit whether people believe it. In my case that would be because the default scenario for a technological singularity is Eternal Doom. By default, if AI researchers do not pay extreme attention, the first smarter-than-human AI will have different goals than we do, and recognize us as a threat to those goals and eliminate us all. Or squash us like bugs without even paying much attention. Or use our constituent atoms to maximize its goals. Or keep us drugged and lobotomised in soft cages, so that we're safe and happy. Or something. And of course, we won't be able to stop it, because it will outsmart us at every turn (we can do the same to chimps). This is why this singularity thing is at least worth looking into. Being able to nuke away our civilization is bad enough, but this could be even worse. Or it could turn our world into a paradise. And of course, with more believers, there will be more donors to the organizations working on this, more researchers working on it… So belief does count, to some extent. (On the other hand, becoming angry at non-believers like a crusader is unlikely to spur positive reactions. By cleaning up this image description, you actually helped our cause.)

jpadkins 14y ago
It's not the first smarter than human AI that is the threat. It's the first self-improving AI that people have to worry about. It can start out at a much lower than human intelligence (whatever that means) but as long it can keep improving it could turn into the nightmare scenario. The safeguards need to be built into the self improving AI, no matter how 'dumb' it starts out as.
- loup-vaillant 14y ago
  Well, any path that lead to superhuman intelligence is dangerous. Self-improving AI may be the likeliest, but it's not the only one. Think Upload, brain-computer interfaces, or computing overhang, for instance.