4 ms·

> This is similar to the AI question. Are you willing to put your full trust into a system with dynamics not fully understood, or do you put your trust into hum

> This is similar to the AI question. Are you willing to put your full trust into a system with dynamics not fully understood, or do you put your trust into humans running centralized systems? Tangent, but this is poor phrasing. If posed this way, an astute response would be that we don't fully know how human brains or emergent group behaviour in humans works either (which is why the social sciences are so hard!), so how is the second option meaningfully different from the first beyond tradition? The key thing that separates AI systems and human systems is lack of intuition for AI behaviour. So the question should really be phrased: is it safe to trust a system you don't have an intuitive model for? There are some cases where the answer is yes e.g. we trust airplanes built and ratified by experts even if we don't know how airplanes work on the presumption that, at least in the conditions we care about, airplanes have been tested and behave correctly.

WnZ39p0Dgydaz1 6y ago
Agreed about poor phrasing, it isn't really about "understanding". But we know that humans are robust through millions of years of evolution, and we know that, mostly, their incentives are aligned with other humans. I don't think it has much to do with intuition either though. For example, I believe I have a pretty intuitive understanding of what most current AI systems do (that's the field I work in), and that's exactly the reason I would not trust them to make fully autonomous decisions in cases where they need to be robust to humans trying to exploit them. I have absolutely no intuitive understanding of how airplanes work. > There are some cases where the answer is yes e.g. we trust airplanes built and ratified by experts even if we don't know how airplanes work on the presumption that, at least in the conditions we care about, airplanes have been tested and behave correctly. A big difference is that airplanes do not interact with humans, and humans have no incentive to exploit airplanes. They are a system that purely interacts with the physical world, or physics, which seems to be simpler to model than human behavior and is not fundamentally adversarial. With many blockchain networks, e.g. Bitcoin, you are in an adversarial environment - exploiting the network yields huge payoffs. With a fully autonomous AI-based decision making system you can get something similar, depending on where it's applied. For example, tricking recommendation systems to uprank your product, automated trading systems (responsible for flash crashes), impersonalization through voice/face recognition, etc. To be fair though, most AI systems are not deployed in adversarial environments, but blockchains seem to be, since you can almost always profit from exploiting them.

WnZ39p0Dgydaz1 6y ago

Agreed about poor phrasing, it isn't really about "understanding". But we know that humans are robust through millions of years of evolution, and we know that, mostly, their incentives are aligned with other humans. I don't think it has much to do with intuition either though. For example, I believe I have a pretty intuitive understanding of what most current AI systems do (that's the field I work in), and that's exactly the reason I would not trust them to make fully autonomous decisions in cases where they need to be robust to humans trying to exploit them. I have absolutely no intuitive understanding of how airplanes work. > There are some cases where the answer is yes e.g. we trust airplanes built and ratified by experts even if we don't know how airplanes work on the presumption that, at least in the conditions we care about, airplanes have been tested and behave correctly. A big difference is that airplanes do not interact with humans, and humans have no incentive to exploit airplanes. They are a system that purely interacts with the physical world, or physics, which seems to be simpler to model than human behavior and is not fundamentally adversarial. With many blockchain networks, e.g. Bitcoin, you are in an adversarial environment - exploiting the network yields huge payoffs. With a fully autonomous AI-based decision making system you can get something similar, depending on where it's applied. For example, tricking recommendation systems to uprank your product, automated trading systems (responsible for flash crashes), impersonalization through voice/face recognition, etc. To be fair though, most AI systems are not deployed in adversarial environments, but blockchains seem to be, since you can almost always profit from exploiting them.