While a code-data distinction is often illusory in practice, figuring out where this separation can be useful runs throughout the information theory underlying

While a code-data distinction is often illusory in practice, figuring out where this separation can be useful runs throughout the information theory underlying learning systems and programming languages. My question about this proposed model is how it handles the tradeoff between generality and efficiency that most learning systems are up against better than "mere" function approximation (e.g. ANNs). Concretely, treating parts of the solution space of some problem as a grammar specification seems like a way of simplifying the representation domain, so that the differentiable bits of a given model have fewer ultimately meaningless degrees of freedom to search over, but at some point you have to choose a grammar, and how would this work in the proposed system?