I think this sounds like using a Markov process to model a current state and reachable states, at two steps away, and tryimg to assign transition probabilities

I think this sounds like using a Markov process to model a current state and reachable states, at two steps away, and tryimg to assign transition probabilities between them. The flaw I see in this approach is that one has limited sensing (don't really know the true state of things) and likely a large number of possible transitions to other states that are outside of one's influence or ability to predict. But I guess it may work well in limited domains, with few variables/agents and it's probably better to give it some thought rather than ignoring it completely.