5 ms·
Unfortunately it's all of the above. Sometimes the code isn't stable (eg; a library method changed), sometimes the behavior has changed (eg; same library method
by alfalfasprout 4y ago
Unfortunately it's all of the above. Sometimes the code isn't stable (eg; a library method changed), sometimes the behavior has changed (eg; same library method but now it has a different behavior), and sometimes it has some other unwanted effect (the computation produces a totally different result, it runs way slower, or something else).
With a service it's all much easier because you can have pretty robust integration tests and you're good to go. With an ML model it's a lot trickier because even if you have sample payloads and responses there might be other payloads you didn't think about that could cause issues. Or it might rear its ugly head when you finally retrain the model and suddenly model performance is crap.