I think you’ve hit on the concept when humans in the loop are needed. For many black box systems, the average performance may be comparable or better than huma

I think you’ve hit on the concept when humans in the loop are needed. For many black box systems, the average performance may be comparable or better than humans, but when errors occur, they should fail predictably. If systems fail unpredictably (ie unbounded in the types of errors it makes), then they can never be deployed in the field without humans in the loop. This is why self driving cars can, at the same time, be on average safer but still not be accepted by public.