I think the real biggest "gotcha" is not understanding where the control loops are in your data. The classic example is the "air conditioned room" thought expe

I think the real biggest "gotcha" is not understanding where the control loops are in your data. The classic example is the "air conditioned room" thought experiment. Imagine feeding an AI model a dataset with a bunch of datapoints like this Room hot | AC on full blast Room cold | AC off Room warm | AC on 50% Then asking the AI how to make a hot room cold. The AI is very likely to say "turn off the AC" because "AC off" is a feature of cold rooms. Maybe we think that if we give the AI more information, it could do better. Lets imagine that our room has a window and an oversized AC. We'll now be feeding it info about the source of heat as well as the source of cold, so surely it will get the right answer this time: Window open | AC on full blast | Room cold Window closed | AC off | Room cold Window 50% | AC on 50% | Room cold This dataset is going to have the AI thinking that neither the window nor the AC impact room temperature, and that you might be able to close a window by turning off the AC. The key thing to understand is that control loops change the "physics" of your system and that knowledge about the control-physics does not transfer to the uncontrolled-physics.