I don't think there will ever be an answer to this. There's a similar problem with choosing a threshold for a decision. The common thing to do is to make a RoC

I don't think there will ever be an answer to this. There's a similar problem with choosing a threshold for a decision. The common thing to do is to make a RoC curve to compare the trade-off between true positive rate and false positive rate. If you have a system you can reason about completely, then sometimes you have a number that gives the absolute answer. Say you get error below floating point resolution. But I guess it's otherwise what is perceptible or meaningful, either in quantity or percentage. A penny is not a life changing amount of money and something happening 0.01% of the time is rare enough to be tolerable.