4 ms·
That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel
by DanAndersen 11y ago
That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario.
My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal.
The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.