4 ms·
The issue with AI safety and unanticipated AI outcomes in general is that it’s always just a cock-up with incentives. It’s easy to sort out in narrowly specifi
by ncallaway 5y ago
The issue with AI safety and unanticipated AI outcomes in general is that it’s always just a cock-up with incentives.
It’s easy to sort out in narrowly specified areas, but an extremely hard problem as the tasks become more general.
- qayxc 5y agoEven worse: if simulations are used, you now have two problems - formulating correct incentives and protecting against abusing flaws in the simulation.
- shrimpx 5y agoIsn’t this true about all systems, not just “AI”? The definition of a software bug is an unintended behavior. In a large system, myriad intents overlap and combine in unexpected ways. You might imagine a complex enough system where the confidence that a modification doesn’t introduce an unintended behavior is near zero.
- ncallaway 5y agoI think it’s true for many systems, not just AI that’s true. AI is worth calling out in this regard because, if the field is successful enough, it can create dangerous systems that don’t behave how we want. Building a safe general AI is much harder than building a general AI, which is why it’s worth considering AI as it’s own problem domain.