I don't see anything wrong with that. Its about capturing and expressing structure. My view is that some structures are easy to specify using soft rules, others

I don't see anything wrong with that. Its about capturing and expressing structure. My view is that some structures are easy to specify using soft rules, others by connection architecture, yet another by functional priors, invariants etc etc. In the past we have been looking for one representation/specification language that works for all. I think that's not the most efficient approach even if these modes of representations have uniform approximation property. "Can represent" does not translate to "shortest representation" or "learnable representation" or "easy to manipulate". These properties are very useful and cannot be ignored.