4 ms·
When you have a lot of fancy metrics/models/bootsraps to throw at, people would just see what sticks.
by Charon77 1y ago
When you have a lot of fancy metrics/models/bootsraps to throw at, people would just see what sticks.
- jononor 1y agoHappens all the time. Problems come quickly when the datasets used for evaluation are not clean, or the evaluation is incorrect - data leakage, problematic imbalance between groups, distribution shifts vs the actual production data. Or people just checking the average performance, but not typical or worst case. Have seen many people run in circles chasing metrics that are meaningless to the task they are supposed to be solving.