Agreed about poor phrasing, it isn't really about "understanding". But we know that humans are robust through millions of years of evolution, and we know that,

Agreed about poor phrasing, it isn't really about "understanding". But we know that humans are robust through millions of years of evolution, and we know that, mostly, their incentives are aligned with other humans. I don't think it has much to do with intuition either though. For example, I believe I have a pretty intuitive understanding of what most current AI systems do (that's the field I work in), and that's exactly the reason I would not trust them to make fully autonomous decisions in cases where they need to be robust to humans trying to exploit them. I have absolutely no intuitive understanding of how airplanes work. > There are some cases where the answer is yes e.g. we trust airplanes built and ratified by experts even if we don't know how airplanes work on the presumption that, at least in the conditions we care about, airplanes have been tested and behave correctly. A big difference is that airplanes do not interact with humans, and humans have no incentive to exploit airplanes. They are a system that purely interacts with the physical world, or physics, which seems to be simpler to model than human behavior and is not fundamentally adversarial. With many blockchain networks, e.g. Bitcoin, you are in an adversarial environment - exploiting the network yields huge payoffs. With a fully autonomous AI-based decision making system you can get something similar, depending on where it's applied. For example, tricking recommendation systems to uprank your product, automated trading systems (responsible for flash crashes), impersonalization through voice/face recognition, etc. To be fair though, most AI systems are not deployed in adversarial environments, but blockchains seem to be, since you can almost always profit from exploiting them.