3 ms·
Right. If the cost of adding more training data remains flat or increases, a logarithmically increasing n will eventually reach a point such that it's more expe
by throwawaymath 8y ago
Right. If the cost of adding more training data remains flat or increases, a logarithmically increasing n will eventually reach a point such that it's more expensive to continue increasing training volume.
In fact, unless the cost of adding and using training data exponentially decreases over time, it's a mathematical certainty that a logarithmically increasing n will quickly incur expensive, diminishing returns for using more data.
So in the context of this Google paper, you could conceive of a situation where training data actually becomes easier to load (albeit subexponentially) and still becomes too expensive to use relatively quickly.