I dunno, I think this sentence in the 2nd paragraph is quite clear: Such games can be described by a Markov process defined by the four probabilities that char

I dunno, I think this sentence in the 2nd paragraph is quite clear: Such games can be described by a Markov process defined by the four probabilities that char- acterize each of the two player’s strategies [9] (because this is an infinitely repeated game, the probability to engage in the first move–which is unconditional–does not play a role here). Each Markov process has a stationary state given by the left eigenvector of the Markov matrix, which in this case describes the equilibrium of the process. ... If we assume that two opponents play a sufficiently large number of games, their payoff approaches the payoff of the Markov stationary state [1,9]. We can use this mean expected payoff as the payoff to be used in the payoff matrix E that will determine the ESS. TLDR: keep varying your strategy until you win in the long term, using your mean expected payoff as a payoff matrix in each subsequent state.