I would tend to agree it generalizes for the given dataset, but stretch and say it generalizes for a orders of magnitude larger dataset of "similar" data that e

I would tend to agree it generalizes for the given dataset, but stretch and say it generalizes for a orders of magnitude larger dataset of "similar" data that encompasses your smaller portion of data on which you trained; Then, as he pointed, you have only 2 options: or you need so much hard-work in collecting properly sampled and mapped data that you can model your data by hand then, no need for NNs, the other option is you just pray for the much bigger unknown portion of the data really be "similar" to your training portion, and praying, in this sentence, is ultimately just making a estimation.