In general there are rigorous ways to quantify those kinds of model deficiencies. From an informal perspective: if the change in data is small and the change in

In general there are rigorous ways to quantify those kinds of model deficiencies. From an informal perspective: if the change in data is small and the change in model output is large, your model is poor (and in particular, possibly overfit). Formally what you're describing is the bias-variance tradeoff.[1] You can assess this by looking at the conditioning of your model, which measures how sensitive it is to changes.[2] Roughly speaking, the condition number of an estimator (or generally, function) measures how large the change f(x) -> f(y) in the range for the change x -> y in the domain, where x and y are close. That will give you variance. If you try to minimize bias too much, you may overfit your model and it would exhibit high variance in cross validation. If you try to minimize variance, you may fail to capture relations in your underlying data, which would exhibit higher bias. Practically speaking, for your specific example: if a relatively small change in the sample data resulted in the model adjusting its prediction to 99% from 1%, I would assume your model is severely overfit (I can't quantify exactly how small without more context, but let's agree it's small). From a meta Bayesian perspective, it would take quite a lot of further cross validation for me to drop that belief ;) _____________________ 1. https://en.m.wikipedia.org/wiki/Bias–variance_tradeoff https://en.m.wikipedia.org/wiki/Bias–variance_tradeoff 2. https://mathworld.wolfram.com/ConditionNumber.html https://mathworld.wolfram.com/ConditionNumber.html