> An important point is that AI does not need to be sentient or conscious to do a lot of damage. It's not 100% clear that consciousness is necessary even for t

> An important point is that AI does not need to be sentient or conscious to do a lot of damage. It's not 100% clear that consciousness is necessary even for things like creative problem solving or goal-setting (i.e. choosing which problems to solve). That is, advanced intelligences that can do all we can and more—and could even "decide", in the worst case, that they'd be better off without humans—may not need consciousness as a precondition for that. Cf. Peter Watts' Blindsight http://www.rifters.com/real/Blindsight.htm http://www.rifters.com/real/Blindsight.htm