Ah, but there's a difference between mechanism and policy. AIs are piloting our planes and boats, and soon our cars and trucks, so that they will go where we w

Ah, but there's a difference between mechanism and policy. AIs are piloting our planes and boats, and soon our cars and trucks, so that they will go where we want them to go without demanding so much effort and attention from us. They implement the mechanism, but we still set the policy -- and we are still responsible for the consequences. Now read 'austinl's comment at the top of this page, about a fictional world that cedes its policy decisions to AIs. That, I would argue, is a fundamentally different thing. I think there is a temptation in human psychology to want to be taken care of, to want someone (or something) else to make the hard decisions. And I think there will be points where we're tempted to turn our power and responsibility over to machines. I also think that the concern about machines taking over is misplaced: machines, though they may one day be unimaginably more sophisticated than our current ones, will never be conscious or make conscious choices. We will always be able to pull the plug, unless we ourselves set things up so that pulling the plug is too risky. So the situation I see us in is where concern about a "singularity" that I don't think will ever happen is somewhat distracting us from more important, if rather more mundane, questions about exactly which decisions we want our machines making, and which we should reserve to ourselves. The stock market is a good example here.