>too-dumb AIs that people treat like they're smart. I think this is one of the most overlooked too. This is also a very bad problem with people, and big thing

>too-dumb AIs that people treat like they're smart. I think this is one of the most overlooked too. This is also a very bad problem with people, and big things can really crumble fast when a situation comes up which truly calls for more intelligence than is at hand. Artificial or not. It can really seem like smooth sailing for years before an event like that rears it's ugly head, compounding the lurking weakness at a time when it can already be too late. Now human-led recovery from failures of human-fallible systems does have at least a few centuries head start compared to machine recovery of AI-fallible systems. So there is that, which is not exactly fair. As AI progresses I guess you can eventually expect the stuff that works to achieve comparable validation over time.