4 ms·
"Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb." A bomb is dumb, its explosion just raw chemistry. I
by flashman 11y ago
"Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb."
A bomb is dumb, its explosion just raw chemistry. I would argue we're like small children playing with a sleeping parent: when it finally wakes up, it's going to have its own agenda for us.
- Filligree 11y agoThat gets the 'intelligence', but misses the 'amoral'. Parents tend to have their children's best interests in mind. If we're playing this game, then humanity is more like a small child trying to wake up Cthulhu in hopes that he'll play nice.
- Retra 11y agoYou're going to build an amoral, powerful computer and it's going to do what? Slap the fork out of your hand every time you try to eat something? We build machines using a highly human-dependent artificial selection process. That's not going to change because the number one feature being selected for is usefulness to humans. I'm confused about how that's going to result in an even modestly-powerful, fully integrated machine that is not demonstrably useful to us. That's a bit like saying we should reconsider having sex because our babies might evolve grenades for hands, and that would be suicidal.
- TheOtherHobbes 11y agoHumans are dumb too, but still manage to explode regularly - although not usually with intelligence. I'm completely failing to understand why a hypothetical god-AI is being taken seriously as a threat to survival, but the many potential and actual horrors of the human world are being ignored here. We're perfectly capable of destroying ourselves without needing to invent an AI to do it for us. Why is this not taken seriously as a problem?
- jimbokun 11y agoYou should actually read the article. It directly discusses the questions you raise.