Umm that's a beautiful theoretical result no doubt. But in practice the choice of the "space" that determines the neighborhood, matters more than anything else

Umm that's a beautiful theoretical result no doubt. But in practice the choice of the "space" that determines the neighborhood, matters more than anything else on the top of that space. Also as another commenter pointed out the result is applicable when infinite labeled samples are present, which is almost never. IMHO the reason Machine Learning is NOT Stats or Information theory because these beautiful fundamental result were found to be futile in practice.