> On its own, a bimodal distribution does not tell you that a model is good. (A vacuous model that randomly assigns probabilities of just 0.0 and 1.0 would also

> On its own, a bimodal distribution does not tell you that a model is good. (A vacuous model that randomly assigns probabilities of just 0.0 and 1.0 would also have a bimodal score distribution.) However, in the presence of evidence that transactions with a low score are not fraudulent and transactions with a high score are fraudulent, an increasingly bimodal distribution is a sign of improved efficacy for a model. To do this more precisely, a scoring rule (https://wiki.lesswrong.com/wiki/Scoring_rule https://wiki.lesswrong.com/wiki/Scoring_rule) gives a system credit for both (1) making accurate predictions and (2) being confident at the right times.