WhyLabs cofounder here, so my opinions are probably biased. When it comes to MLOPs, data makes it much more complex to handle. Think of it as the curse of dime

WhyLabs cofounder here, so my opinions are probably biased. When it comes to MLOPs, data makes it much more complex to handle. Think of it as the curse of dimensionality. Nobody wants to deal with metrics across tens, if not hundreds or thousands of features. In addition, data is often not stored in nice SQL based system with strong schema enforcement, so we see data bugs creeping up all the time. An example is when an upstream API service returns the 9 digit zip code instead of the 5 digit one. This sort of data issues can creep up at many parts of the ML system, especially when you use JSON to pass data around. You can defend against some of these problems with some basic devops monitoring, but when you deal with tons of features this becomes a tedious task. DevOps tools focus on solving problems around code, deployment and systems health. They are not designed to address the curse of dimensionality above, and you sacrifice a lot by trying to reduce data problems into DevOps signals. To be fair, I don’t think we need some fancy algorithms, but I think we need tools that are optimized around the user experience (I.e. removing frictions) and workflows for these data specific problems. There’s a lot to learn and apply from the DevOps world to think about data health, such as logging and collecting telemetry signals.