I assumed it is because the amount of output neurons is small compared to the amount of inputs. For example a digit OCR network takes maybe 10.000 pixels as inp

I assumed it is because the amount of output neurons is small compared to the amount of inputs. For example a digit OCR network takes maybe 10.000 pixels as input, but has only 10 possible outputs. I'm no expert either, any confirmation or refutation would be very welcome :)