> Is it possible to generate infinite data and get almost perfect neural network accuracy? Basic answer is no, but the reason is kind of interesting. Imagine

> Is it possible to generate infinite data and get almost perfect neural network accuracy? Basic answer is no, but the reason is kind of interesting. Imagine that the input to the model is a list of facts, initially the facts are just: * Image 1 has class A * Image 2 has class B * etc... The idea with data augmentation, in a roundabout way, is to add other facts: * Flipping the image does not change its class * Translating the image does not change its class * Adding a small amount of noise to the image does not change its class * etc... However, it is tricky to express those facts as inputs to the model, but it is easy to generate new images based on those facts which the model should be able to learn from. It would likely be more efficient if those facts could be expressed directly though. So, by generating more data the model can progressively learn those "class invariant transformations", but the model would only reach perfect accuracy if all class invariant transformations were taught. Another way to "teach" a model these rules is to embed the rule into the structure of the model itself, eg the idea behind convolutional neural networks is to embed translation independence into the model, so that it doesn't need to be taught that from large batches of translated images.