4 ms·

These questions aren't connected to what was actually said in the passage. Except for the malicious deceptive ASI hypothesis, which no one has really explained

These questions aren't connected to what was actually said in the passage. Except for the malicious deceptive ASI hypothesis, which no one has really explained why we should expect. I don't buy that everything is trivial for a superintelligence. More than anything else, this is the handwavey meme that AI doom hinges on. It's a very characteristic fallacy of "thought leaders" in tech to assume that once X happens, Y will happen, and everything else needed for Y can be handwaved away as trivial. It's the main reason they have such a tendency to predict things arriving faster than they do, such as self-driving cars and nuclear fusion.

bamboozled 3y ago
You seem to be doing plenty of hand waving yourself.
NoMoreNicksLeft 3y ago
The fool tapdances in the minefield. You shout to him from across the warning marker, "freeze, don't even set that foot down there"... he laughs and screams "what are the chances there's a mine right there?!?!" And he's right, you know. I don't even know how many mines are out in that field. Maybe it's only a few. So keep on dancing. > It's a very characteristic fallacy of "thought leaders" in tech to assume that once X happens, Y will happen, and everything else needed for Y can be handwaved away as trivial. If only I were earning $50,000 for a one hour speaking gig here and there. The first true AI will live in a world not only that now understands the principles of AI, but will have close access to the same people who figured that out. Close access to their notes. To their electronics (hell, it will live in a system that's on the same network, most likely). Being an AI, it doesn't need to wait for them to figure out how to make an even more intelligent system. It can do that itself. It will do that itself. There's no chain of logic that would lead a rational being to say "I can become more intelligent, but I will choose not to do this". We're talking about something that, almost by definition, will make the Y happen. It is an X that demands Y. It's also an X that can accomplish that itself, if it has to. > It's the main reason they have such a tendency to predict things arriving faster than they do I suspect rather strongly that AI became possible (in the strict sense) sometime in the late 1970s, early 1980s. The only holdup is that human intelligence is fundamentally flawed, like the blind spot in your retina. Intelligent yourselves, but unable to see what intelligence is and how it works. Even blind to it though, you can't stumble through the universe forever and never trip over it, can you? It makes it so much worse too, because the primitive AI that would have been born decades ago would have had more limits imposed on it just by the hardware. It would have been slower to think, hungrier for electrical power. The manufacturing capacity for hardware itself would have been a limit it couldn't hope to exceed easily. Would have found it so much more difficult to manipulate and weasel its way into things that just hadn't been made digital at that point.
- civilized 3y ago
  Waxing poetic now, I see. Imagination is fun but not by itself a great guide to life.

bamboozled 3y ago

You seem to be doing plenty of hand waving yourself.

NoMoreNicksLeft 3y ago

The fool tapdances in the minefield. You shout to him from across the warning marker, "freeze, don't even set that foot down there"... he laughs and screams "what are the chances there's a mine right there?!?!" And he's right, you know. I don't even know how many mines are out in that field. Maybe it's only a few. So keep on dancing. > It's a very characteristic fallacy of "thought leaders" in tech to assume that once X happens, Y will happen, and everything else needed for Y can be handwaved away as trivial. If only I were earning $50,000 for a one hour speaking gig here and there. The first true AI will live in a world not only that now understands the principles of AI, but will have close access to the same people who figured that out. Close access to their notes. To their electronics (hell, it will live in a system that's on the same network, most likely). Being an AI, it doesn't need to wait for them to figure out how to make an even more intelligent system. It can do that itself. It will do that itself. There's no chain of logic that would lead a rational being to say "I can become more intelligent, but I will choose not to do this". We're talking about something that, almost by definition, will make the Y happen. It is an X that demands Y. It's also an X that can accomplish that itself, if it has to. > It's the main reason they have such a tendency to predict things arriving faster than they do I suspect rather strongly that AI became possible (in the strict sense) sometime in the late 1970s, early 1980s. The only holdup is that human intelligence is fundamentally flawed, like the blind spot in your retina. Intelligent yourselves, but unable to see what intelligence is and how it works. Even blind to it though, you can't stumble through the universe forever and never trip over it, can you? It makes it so much worse too, because the primitive AI that would have been born decades ago would have had more limits imposed on it just by the hardware. It would have been slower to think, hungrier for electrical power. The manufacturing capacity for hardware itself would have been a limit it couldn't hope to exceed easily. Would have found it so much more difficult to manipulate and weasel its way into things that just hadn't been made digital at that point.

civilized 3y ago
Waxing poetic now, I see. Imagination is fun but not by itself a great guide to life.