This is why we've been trying to encourage people to think about lightweight data logging as a mitigation for data quality problems. Similar to how we monitor a

This is why we've been trying to encourage people to think about lightweight data logging as a mitigation for data quality problems. Similar to how we monitor applications with Prometheus, we should approach ML monitoring with the same rigor. Disclaimer: I'm one of the authors. We spend a lot of effort to build the standard for data logging here: https://github.com/whylabs/whylogs https://github.com/whylabs/whylogs. It's meant to be a lightweight and open standard for collecting statistical signatures of your data without having to run SQL/expensive analysis.