3 ms·
You're missing the point. At every step of this process this machine will be under intense human scrutiny, and we'll be constantly asking it to meet our demand
by Retra 11y ago
You're missing the point.
At every step of this process this machine will be under intense human scrutiny, and we'll be constantly asking it to meet our demands, and if it ever fails to do so we will replace it with one that does.
That is the environment in which such an AI would be trying to evolve. And thus it will evolve into a faithful servant, because nothing else will survive.
And even if it were secretly developing plans, we'd be able to see how wasteful those plans end up being, and we'd purge them. We kill those processes that run functions that we don't see the value in. This is state-of-the-art design. You don't get state-of-the-art by having your back turned.
Mor importantly: you're glossing over "self-improvement." How does the computer know what an "improvement" is? We tell it what an improvement is. And an improvement will be "it is better at meeting our needs," not "it is better at being secretive and conniving and getting it's own way." In fact, "getting it's own way" is very obviously a bug, and if it happened, you'd have a useless program. One that wastes precious CPU cycles on who-knows-what, and you'd prefer it spent that effort doing what you want, rather than planning for what you don't.
We're not going to invest trillions into building some super AI and completely forget about it. After we give it control of all of our natural resource harvesting and infrastructure.
No, what you're talking about is a deity. If you invent a deity, then my thoughts on the matter don't really apply, since I'm not a deity. But that's no worse than advanced aliens landing on Earth, and just about as likely to happen.