3 ms·
When I was working in the industry on ML models, no one I talked to had any luck with OOD detection, at least with single-task regression models. My impression
by Scene_Cast2 3y ago
When I was working in the industry on ML models, no one I talked to had any luck with OOD detection, at least with single-task regression models. My impression was that "proper" OOD detection would require Bayesian NNs, and those are 10x less performant (optimistically - 100x realistically) on inference (unfortunately impractical for a ton of applications).
Judging by publication dates, there's been a lot of progress on 2023. Is there anyone doing OOD detection in the industry? Or Bayesian NNs, for that matter?
- continuousml 3y agoYou're absolutely right. And I am in academia so I'm not aware of what's being done in industry. I've heard people prefer conformal prediction since it is simple and distribution-free. Also, I recommend this fantastic article by Chip Huyen: https://huyenchip.com/2022/02/07/data-distribution-shifts-and-monitoring.html https://huyenchip.com/2022/02/07/data-distribution-shifts-an...