3 ms·
The crux of the issue is "how can we be certain that we've addressed all the failure modes?" This isn't like proving the correctness of a program where we can r
by kelseyfrog 3y ago
The crux of the issue is "how can we be certain that we've addressed all the failure modes?" This isn't like proving the correctness of a program where we can rely on exhaustivness or universal quantification.
"It worked in the simulation," is the embodied AI equivalent of "It worked on my machine." It's not acceptable.
- vimax 3y agoAs a first step, if addressing the issues were the intent rather than stoking AI fears, they would mention details about the AI/LLM and promoting being used
- kelseyfrog 3y ago> if addressing the issues How confident are you that the issues can be addressed? How confident would you need to be to endorse deploying it out in the wild? How confident would you need to be that no "unforeseen" issues would arise? I believe it's necessary to establish these levels of confidence even before discussing specific failure cases. We would normally discuss failures in terms of base-rate comparison, but we don't have one for "how often autonomous weapons are unaligned?" Enumerating the failure modes isn't a substitute for the empirical distribution of failure mode probabilities.