Information theory is the right avenue but it's too often used to imply things it says nothing about. From (1) 1. A determinist transformation can't create mor

Information theory is the right avenue but it's too often used to imply things it says nothing about. From (1) 1. A determinist transformation can't create more information than its inputs you can't derive (2) 2. Later stages don't contain any new information. without restraining your model to a monotonic chain of transformations that can't tap into the information content of the environment.