On the contrary, I've found that one of the worst things you can do when demonstrating / deploying a machine learning model is to show the underlying prediction

On the contrary, I've found that one of the worst things you can do when demonstrating / deploying a machine learning model is to show the underlying prediction confidences. You can be right 95% of the time, but critics will only remember, and fault the whole system, because of a single, incorrect, but high confidence error. Which would be reasonable if that erroneous prediction had fat tail consequences, but in cases I've had this happen, it has not