3 ms·
I know it's an empirical result, but it would be interesting if it was truly logarithmic. Classic statistics gives various asymptotic results based on variation
by simonbyrne 8y ago
I know it's an empirical result, but it would be interesting if it was truly logarithmic. Classic statistics gives various asymptotic results based on variations of the central limit theorem, which generally imply sqrt(n) scaling, but these assume various regularity conditions (typically smoothness and fixed dimensionality).
It would be interesting if there was some corresponding law for more flexible models which did indeed give you logarithmic scaling.
- moultano 8y agoIf the model is an ideal encoding, then the description length of every input grows logarithmically. Perhaps the performance is gated on the number of "features" in the ideal encoding.