3 ms·
Data drift is a problem regardless of the model/prediction task. Which is why models should be monitored in production, something everyone agrees is important,
by qsort 4y ago
Data drift is a problem regardless of the model/prediction task. Which is why models should be monitored in production, something everyone agrees is important, but nobody actually does.
It's a new era for "computer says no" types to revel in.
- noobermin 4y agoSometimes though, "computer says no" is in fact the goal for some people. That some people dont realize so just demonstrates naivety.
- qsort 4y agoIt doesn't mean I have to accept that it's the computer's fault, though. Forcing them to take responsibility for their own decision is not naive, it's the counter to that strategy.
- noobermin 4y agoTo be clear, I'm not saying that you specifically are naive. I do believe though that this naiveity exists here in tech amongst some proponents.
- PeterisP 4y agoI strongly believe that for stable tasks not only models should be monitored in production, but there has to be a business process which takes any detected mistakes of the system and feeds them back as training data, creating a "human-in-the-loop" system even if the vast majority of the data is handled automatically - as these mistakes have a much stronger useful signal for updating the weights as any random sample.