Well, I think you sorta can. Your boss can say, if you don't stop joking around, and don't follow the rules, I will shoot you. We have an evolutionary predispos

Well, I think you sorta can. Your boss can say, if you don't stop joking around, and don't follow the rules, I will shoot you. We have an evolutionary predisposition to avoid pain and death and the whole social structure on top of that, and this ensures that sometimes we just have to be honest. Which is one of the reason why I think we need at least two levels of learning for general intelligence. The bottom learner is the one which learns the actual task at hand, and the top learner is tasked (with learning) to, given a description of a task, produce (train if needed) a bottom learner that can solve that task. This solves the ambiguity, because if we are confident enough that the top learner can produce correct bottom learners for different descriptions, then adding another rule to the description will likely cause the bottom learner (produced by the top learner) to also obey that rule. It also resolves the understanding problem, because while no learner might be sure if it is itself correct, the top learner has to somehow evaluate the bottom learner, and so the top learner sees where bottom learner is failing to understand (which rules are problematic). What's even more interesting, if the language to express the problems is rich enough, then we can apply the top learner to itself as the bottom learner (express the actual learning setup from within the language), and we get sort of a "philosopher", entity that tries to learn how best to build learners. (This is an interesting analogy you can make with Futamura projection, by thinking about "learners" instead of "programs", and what happens if learners are tasked to produce another learner. The reason why we can be somewhat confident in "correctness" of the bottom learners is similar to reason why we can be somewhat confident that compiled program does what we want, despite it being completely new to the compiler.) There are multiple ways to map this top/bottom learners onto humans. You can think of the bottom learner as the actual intuitive systems that we have, and the top learner as the conscious system, which drives what we are gonna learn and how. Or the bottom learner as the system that contains the model of the environment, and the top learner as a system that provides episodic learning via some kind of simulation (our dreams, imagination) to it, and reflects on it. Or the bottom learners are individual humans, while top learners are the genes that drive the evolution of the human brain. In practice, there might be more than two levels.