Well, to test the system sure. That's normal. However, the issue not quite as simple when training a model. For instance, the increasingly popular Transformer

Well, to test the system sure. That's normal. However, the issue not quite as simple when training a model. For instance, the increasingly popular Transformer architecture pushes around internal representations when fed new training samples. These representations are not "causal parameters" - they do not represent unchanging mechanisms in real life. Instead, they depend on data. The question thus becomes: How prominent should edge cases be in the sample? A ML model trained on many edge cases may be more robust for edge cases we feed it, but it may be more unstable on real-world data (in essence, the model may have new edge cases when new data comes in that would not have been edge cases before). All of this comes from the fact that the parameter of the model are not "causal" for the underlying latent DGP. ML, in constrast to stats, does not yet have much theory regarding how to think about parameters relative to the sample of observations one trains on. The question of "causal" identification in stats is easier to tackle, because one can usually define a "scientific" or "structural" model where parameters have meaning, and so we can reason whether we capture "real, causal parameters" (which would be compatible with edge cases) or "reduced form parameters" (which would generally depend on the sample we have). ML will likely get there. Right now, people think in terms of "data shift" (which they shouldn't) and generalizing (missing the point a bit). But we will get there. By the way, this issue arose because ML systems where treated like engineering systems instead of statistical models. For better or worse.