There are two things that ML/AI developers are going to have to deal with once the technologies become widespread in things like self-driving cars, hiring/firin

There are two things that ML/AI developers are going to have to deal with once the technologies become widespread in things like self-driving cars, hiring/firing decisions, and the criminal justice system: "Why did it do that?" and "Make it stop doing that!" The first time a self-driving car accident results in a court case, these things are going to come up. I very much doubt that people are going to be satisfied without clear explanations, and they shouldn't be. When these systems take on roles of increasing importance to society, some level of accountability is going to be necessary.