my point re: sky cheetah is that there are large domains where the only questions you can ask are stupid —- which is to say, ill-posed. in those situations you

my point re: sky cheetah is that there are large domains where the only questions you can ask are stupid —- which is to say, ill-posed. in those situations you cannot expect RL to come up with “normal” solutions that do what you actually want. The sky-humping is a totally valid answer to the question asked; it’s merely that the reward function for “walk forward” doesn’t (and in many situations, may not ever) fully constrain the search space such that you get sane solutions.