4 ms·

I think our goals come in because we have evolved drives vs. being programmed, not because of processing power. At the same time, giving an AI a goal is the mo

by ericb 10y ago

I think our goals come in because we have evolved drives vs. being programmed, not because of processing power. At the same time, giving an AI a goal is the moment when we become subject to an indifferent genie who doesn't share our values. In terms of your original question "what use would they have for us?" I guess my answer is something like: if we program them correctly, helping us will be their only goal. If we give them other goals, even slightly off the mark and don't have a tightly integrated way to course correct (like the neural lace), then we will be like ants crushed by an indifferent boot-heel.

sonink 10y ago
'If we program them correctly, helping us will be their only goal' - I think this is too optimistic a view of the future. I suspect that it is also a bit simplistic. Even if we assume that we have a way to 'program' them, the problem is that we ourselves dont know what it means to 'help' humans. If those humans were a part of the US military facing a situation similar to ww2, it would mean that AI 'help' those humans drop some nuclear bombs. If those humans belong to a religious fundamentalist group, it might mean that the AI help further the cause of their gods. One probable outcome, that this chain of thought leads to, is that perhaps the last AI standing will be the one which kills all other AI, or has the capability to. A possible corollary being that to do so the last AI standing will be the one which is able to prioritise its own survival over everyone else - even other humans. Which was not the plan that we started with.