3 ms·
“The latter being when the training or test data follows a different distribution to the in-operation data” This form of ML bug is the most challenging to catc
by saladtoes 5y ago
“The latter being when the training or test data follows a different distribution to the in-operation data”
This form of ML bug is the most challenging to catch. The true in-operation distribution is often unknown which makes testing for such bugs a very challenging problem. Any thoughts on this?
- mklond 5y agoThanks for your comment. The whole field of run-time monitoring is concerned with this problem. It's a tough one to crack when the distribution changes are subtle, but you can and should at least check simple data attributes for consistency.