Sure, but it should be possible to run enough tests to show the error over X frames doesn't cross Y. Then you can learn on that model and validate on the real t

Sure, but it should be possible to run enough tests to show the error over X frames doesn't cross Y. Then you can learn on that model and validate on the real thing (and use it to fine-tune if needed). The chosen action has very little effect after a few frames anyway.