In your leg-example your condition #2 was not met, let's imagine that still happens despite using "AI magic": At the point when "amputate the right leg" was fo

In your leg-example your condition #2 was not met, let's imagine that still happens despite using "AI magic": At the point when "amputate the right leg" was found to be the correct diagnosis, an authority should be able to audit and reconstruct the wrong "amputate left leg" diagnosis, trace the root-cause of this decision and verify the corrective action. For this, it is important to be able to reconstruct the process of any computer-aided recommendation used. A corporate statement like "we don't know why the model recommended the left leg back then, now it correctly recommends the right leg when feeding the same data" should not be sufficient.