'If we program them correctly, helping us will be their only goal' - I think this is too optimistic a view of the future. I suspect that it is also a bit simpli

'If we program them correctly, helping us will be their only goal' - I think this is too optimistic a view of the future. I suspect that it is also a bit simplistic. Even if we assume that we have a way to 'program' them, the problem is that we ourselves dont know what it means to 'help' humans. If those humans were a part of the US military facing a situation similar to ww2, it would mean that AI 'help' those humans drop some nuclear bombs. If those humans belong to a religious fundamentalist group, it might mean that the AI help further the cause of their gods. One probable outcome, that this chain of thought leads to, is that perhaps the last AI standing will be the one which kills all other AI, or has the capability to. A possible corollary being that to do so the last AI standing will be the one which is able to prioritise its own survival over everyone else - even other humans. Which was not the plan that we started with.