3 ms·
It's quite amusing. The standard statistical theory does not work at all in estimating data vs model size, and the bounds are all vacuously large. It's a very a
by gwern 6y ago
It's quite amusing. The standard statistical theory does not work at all in estimating data vs model size, and the bounds are all vacuously large. It's a very active area of research, understanding why models act so simple when overparameterized and coming up with real measures of model complexity. Lots to read there if you are interested in such things.
- andrewprock 6y agoThat just means that the parameters are not independent.
- gwern 6y agoBut you can fit randomly-generated labels!
- andrewprock 6y agoThat's not in any way surprising. When you have more parameters than data, this is trivial.