An awesome thought experiment that helps to illustrate the concerns with AI is the paper clip maximizer: https://wiki.lesswrong.com/wiki/Paperclip_maximizer htt

An awesome thought experiment that helps to illustrate the concerns with AI is the paper clip maximizer: https://wiki.lesswrong.com/wiki/Paperclip_maximizer https://wiki.lesswrong.com/wiki/Paperclip_maximizer While this particular essay starts off with the AGI premise, this could easily be reworked to assume a relatively "dumb" by highly connected AI. For example, one that is able to buy/sell stocks, place orders from suppliers, sell product and adjust prices, and rebalance workforce and automation lines could still cause some major damage before anyone would notice. There's also a pretty fun clicker game based on the essay: http://www.decisionproblem.com/paperclips/ http://www.decisionproblem.com/paperclips/