If you have say two networks that are based on different choices of perspective in coding a MDL with wildly differing lengths and better performance of the larg

If you have say two networks that are based on different choices of perspective in coding a MDL with wildly differing lengths and better performance of the larger one, I would be highly suspicious as to whether the the poor performer is properly built from a sensible simplifying metric. In terms of a general intuition of how the choice of similarly expressive representations affect complexity, one can look to Kolmogorov Complexity. For any two languages used to compute the KC of some string, there is only an additive translative constant for each possible input. The asymptotics of how each language grows its representation with respect to input size by far will dominate such translative overheads. This is sufficient for philosophical purposes. Although in the practical case these choices matter and MDL was created specifically to treat these practicalities.