> There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we

> There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works. I agree with you on that point, but not with the conclusion that this is in any way a good thing. > Evolution led to human intelligence just fine on its own; why are humans not simply the catalyst in the next stage of this natural process from which something more complex arises? I think I have the complete opposite view here. Why would "being the catalyst in the next stage of evolution" be anything desirable? Evolution has no "stages", it's just chaos and life trying to deal in the best way with the circumstances at hand. If we blew everything up in nuclear armageddon tomorrow and the only remaining life were archaeae, those would be a perfectly fine "next stage of evolution". Similarly, we can build a would in which only robots can survive, but why would we want to? I think, gaining more understanding how the world works is the one thing AI can really bring to the table. If it solves practical problems on the way to that, that's great, but we evidently survived so far pretty well by doing that stuff ourselves.