You're probably right, but from everything I've seen to date, the representations of functions that neural networks learn tend to be smooth, which is to say if

You're probably right, but from everything I've seen to date, the representations of functions that neural networks learn tend to be smooth, which is to say if your input is similar to the domain covered by your training data, you're quite unlikely to get totally crazy outputs...though I can't say I've come across anything that proves this. However, I think the degree to which neural networks can extrapolate is still an open question, so it is important to thoroughly sample your problem space during training. That's a bit of an artform given that these spaces are typically high-dimensional.