3 ms·
I doubt it was obvious scaling up would magically work. I suspect the experiments were limited for analytic simplicity rather than computational.
by brrrrrm 3y ago
I doubt it was obvious scaling up would magically work. I suspect the experiments were limited for analytic simplicity rather than computational.
- mhh__ 3y agoThe modern slow-but-scales way of coding them also wasn't prevalent
- Solvency 3y agoWhy couldn't mathematical proofs/models have predicted or revealed this to be the case back then?
- actionfromafar 3y agoHow??
- nurbsnn 3y agoOn the contrary, there was a mathematical proof that one-hidden-layer neural network with nonlinearity is enough to represent any function. Using more than 1 hidden seemed a waste. https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- pavon 3y agoThe only ML that I ever did was a single undergrad NN class around ~2001. That was a long time ago, but I vaguely remember being taught at that time that adding more nodes rarely helped, that you were just going to overfit to your dataset and have worse results on items outside the dataset, or worse end up with a completely degenerate NN - eg that best practice was to use the minimum number of nodes that would do the job.
- peterfirefly 3y ago"Optimal Brain Damage", 1989: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=17c0a7de3c17d31f79589d245852b57d083d386e https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...