I'm actually wondering how much the no-free-lunch theorem for data compression affects adverserial examples. A neural network can be conceptualized as an extrem

I'm actually wondering how much the no-free-lunch theorem for data compression affects adverserial examples. A neural network can be conceptualized as an extremely efficient compression technique with a very high decoding cost[1]; the NFLT implies that such efficiency must have a cost. If we follow this heuristic intuitively we're led to the hypothesis that an ANN needs to expand its storage space significantly in order to prevent adversarial examples from existing. [1] -- consider the following encoding/decoding scheme: train a NN to recognize someone's face, and decode by generating random images until one of them is recognized as said face. If this works then the Kolmogorov complexity of the network must exceed the sum of the complexities of all "stored" faces.