We've known for decades that NNs are prone to overfit. This is just another example of it. With great variance comes great ... uh ... susceptibility to unusual

We've known for decades that NNs are prone to overfit. This is just another example of it. With great variance comes great ... uh ... susceptibility to unusual future inputs?