I think this is a common scenario when deploying machine learning models or just operating in a complex world. Netflix won't always recommend you the best video

I think this is a common scenario when deploying machine learning models or just operating in a complex world. Netflix won't always recommend you the best video, automated inspections in a factory line can fail, COVID tests have imperfect precision and recall, etc. All models are wrong, but some are useful.