> Is there any good reason why a fully-connected network needs more than one hidden layer? A sufficiently (infinitely?) wide shallow fcnn can work in theory, b

> Is there any good reason why a fully-connected network needs more than one hidden layer? A sufficiently (infinitely?) wide shallow fcnn can work in theory, but is basically impossible to train[0]. It might be feasible to transform or 'compile' a trained deep network into a shallow & wide one, but I'm not sure there would be any benefits, absent sufficiently wide parallel hardware[1] [0] Then again, deep networks were impossible to train for quite a while too, due to the exploding gradient problem. [1] Although, the Cerebras Wafer-Scale chip does exist. Hmm.