> Sounds a bit unexpected from an information theoretical point of view It's very common, in machine learning, to use 'dropout layers' [1] during training - wh

> Sounds a bit unexpected from an information theoretical point of view It's very common, in machine learning, to use 'dropout layers' [1] during training - where different, random chosen values are temporarily turned off at each training stage. The intention is to ensure the network learns not to rely overmuch on any single value. Why have your cat-recognition neural network have a single whisker detector, when you could have ten whisker detectors and combine their outputs? I could well believe that, after intentionally ensuring knowledge of whiskers was redundant, removing that knowledge would be complicated. [1] https://dl.acm.org/doi/10.5555/2627435.2670313 https://dl.acm.org/doi/10.5555/2627435.2670313