3 ms·

Why do machine learners use F1 score instead of a proper scoring rule (MSE, log-loss, etc.)?

• Tarq0n 7y ago
  F1 score is harder to misrepresent, and relates more closely to business objectives. MSE is mostly for regression. Log-loss is very useful for training because it represents a useful signal for how well your model is progressing in fitting the data, but you'll still want to evaluate precision and recall. In practice you'll usually want to tune the tradeoff between precision and recall for the situation. This way you can make a direct connection between model performance and the costs and benefits as they manifest in practice, as well as your tolerance for risk.
  • solresol 7y ago
    Given that this is fraud prevention, the correct error function should be measured in dollars. The thing you want to minimise is money lost. Is the cost of allowing a fraudulent transaction through much greater than the cost of blocking a non-fraudulent transaction? Then the error function should reflect that. Minimising F1/accuracy/log-loss is not necessarily going to save the most money.
    • Tarq0n 7y ago
      There may also be compliance costs. (Will this model satisfy an auditor?)