What happens when an AI is used to make decisions where the user/victim is entitled to know exactly why the AI did what it did? From a business and legal perspe

What happens when an AI is used to make decisions where the user/victim is entitled to know exactly why the AI did what it did? From a business and legal perspective I think the current AI solutions are dangerous and should be used very sparsely, exactly because even the creators can't point to the exact pieces of information that caused the AI to make the choices it did. I totally agree that we need explainability (probably through symbolic systems, not fancier models), but I think you’re overestimating how much more satisfying explanations from more traditional AI are. “My rules told me to do X” is a bit more helpful for a troubleshooting engineer than “my training data trained me to do X”, but from a ‘business and legal perspective’ the difference is much less pronounced IMO. Both answers mean you did something wrong in creating the machine. The fault will always lie with the creator.