4 ms·
Note to casual commenters: the precise real-world implications of the NFL theorems proved by Wolpert and collaborators have been difficult to appreciate, even t
by mturmon 10y ago
Note to casual commenters: the precise real-world implications of the NFL theorems proved by Wolpert and collaborators have been difficult to appreciate, even to people well-versed in the computational learning world.
Starting point: http://www.santafe.edu/media/workingpapers/12-10-017.pdf http://www.santafe.edu/media/workingpapers/12-10-017.pdf
where we read: "However, arguably, much of that research has missed the most important implications of the theorems."
- Animats 10y agoIs that a corollary of the information theory theorem that for any lossless compressed representation, there must some data pattern for which the compressed representation is bigger?
- xapata 10y agoAnything with more information/entropy requires more space to store.
- eli_gottlieb 10y agoWell, the real-world implication (at least, as taught to me when I took my ML class) is that you have to make some assumption about P(f). If you assume that "reality" has to "spend energy" on building complicated f's ("reality functions" when we learned it), then you get usual probabilistic assumptions about function learning and everything goes back to normal (as we usually experience it in the real world). If you make other sorts of assumptions, you get many popular and useful ML algorithms. It's all a matter of finding prior assumptions specific enough to bypass NFL, while still general enough to encompass useful real-world tasks.