That's a very interesting result (assuming it's correct). It certainly agrees with intuition based on analogy to boolean circuits where, for example, the parit

That's a very interesting result (assuming it's correct). It certainly agrees with intuition based on analogy to boolean circuits where, for example, the parity function requires exponential circuit size for shallow circuits but only linear size for deep circuits, but I haven't heard of a proof of this for NN's before.