Looks like someone (GP) beat me to the punch with my sibling comment (https://news.ycombinator.com/item?id=31053031 https://news.ycombinator.com/item?id=3105303

Looks like someone (GP) beat me to the punch with my sibling comment (https://news.ycombinator.com/item?id=31053031 https://news.ycombinator.com/item?id=31053031)! > we have infinite sampling We don't. You're taking the learning out of the process you're discussing by using an infinite sample. Of course, then, the machine learning part becomes trivial, and it's just an optimization problem for minimal training error! If we have all the data, like infinite data, then we have the classification for every one of the (finite) number of possible images already. At that point, why stop at kNN? Use a hashmap!