4 ms·
Don't Cite the No Free Lunch Theorem
- bradknowles 7y agoTANSTAAFL?
- amingilani 7y agoI think you're asking if this is about "there ain't no such thing as a free launch." It isn't, I thought it was at first myself, coming from an Economics background. It's about a machine learning theorm.
- deleted 7y ago[deleted]
- meerita 7y agoThe first thing that came to my mind was that Milton Friedman quote "there's no such free meal, someone has to pay it".
- jkingsbery 7y agoMy first thought was that this headline was referring to the idea in economics too. Today I learned that there's a "No Free Lunch" idea in ML.
- segfaultbuserr 7y ago> Milton Friedman quote This meme was actually popularized by Robert Heinlein in The Moon Is a Harsh Mistress, ten years earlier than Milton Friedman started using it.
- meerita 7y agoI learnt it from Milton, but it's nice to still discover other people figured out this.
- jmmcd 7y agoGood! This article is about the NFL for machine learning (Wolpert), and also mentions NFL for optimisation (Wolpert and Macready). My recent paper [1] is about NFL for optimisation and mentions NFL for ML. The overall message is the same: most people misunderstand it, and you probably shouldn't cite it. And the anthropic principle is a sufficient assumption for this. [1] https://arxiv.org/abs/1906.03280 https://arxiv.org/abs/1906.03280
- yters 7y agoIf the NFL doesn't apply in the real world, then where is the one algorithm to rule them all?
- jmmcd 7y agoThere probably isn't one! But as argued in the paper, our reasons for thinking so are not based on NFL.
- eli_gottlieb 7y agoGLORIOUS VARIATIONAL BAYES
- deleted 7y ago[deleted]
- yters 7y agoEven though the NFL only holds absolutely when problems are closed under permutation, that doesn't imply that most problems are a good fit for a particular algorithm: https://arxiv.org/pdf/1609.08913.pdf https://arxiv.org/pdf/1609.08913.pdf
- eli_gottlieb 7y ago>What it actually (vaguely) says is “You can’t learn from data without making assumptions”. That's what they taught us it means in our graduate machine learning class. What were people citing it to mean?
- contravariant 7y agoThis is pretty much equivalent to 'you can't learn from random noise', isn't it?
- eli_gottlieb 7y agoIt's a little more like, "If you assume nothing is more complicated or costly than anything else, then any set of examples you see could just as easily be generated adversarially to trick you as not."
- lonelappde 7y agoThat's also what they taught us in casual reading and HN, even before ML when it was a theorem about search algorithms and compression. I don't know where the author got their basis for being contrarian.