> You know about dynamic programming right? Well, one of the reasons it was invented is kind of related to what we're talking about right now. There are these t

> You know about dynamic programming right? Well, one of the reasons it was invented is kind of related to what we're talking about right now. There are these things called the bellman equations. Yes, I've done dynamic programming in competitions. I'm not familiar with bellman equations, and the explanation you provided about convergence, policy functions, etc. went over my head, sorry. > If you're actually interested in this sort of thing, I suggest checking out the poker research papers by Noam Brown [...] He mentions in the research paper that there was a mistake in another paper where they didn't account for the way subgames influence each other. I see that as the problem here, but I haven't read the other paper, so I don't know if it was the same category of error. Find the paper he referenced, read through it, and see if you're tricked. It is potentially a kind of real world two envelope problem to see if your tools for reasoning about these things is actually helping you to avoid the error in more complicated situations - though, since you disagree with me that this is about handling of imperfect information (or more precisely, the counterfactuals - there is a variant on wikipedia where they removed the probabilites, but counterfactual reasoning still applied to resolve the paradox) maybe you won't see it as related errors. This sounds really interesting! It was a huge deal in the poker scene when the poker AI developed by Brown and Sandholm defeated pro human players. I read the first few pages of the paper now, but the paper becomes very math-heavy after that. I don't have the necessary background to understand the notation they use. That said, the "mistake" in the way "subgames influence each other" that you referenced, I suspect it was of a far simpler kind - the kind that Brown explains in chapter 2 which he concludes with: This shows that a player’s optimal strategy in a subgame can depend on the strategies and outcomes in other parts of the game. Thus, one cannot solve a subgame using information about that subgame alone. This is the central challenge of imperfect-information games as opposed to perfect-information games. Although Brown says "imperfect-information games" here, he actually means a specific type of imperfect-information games: the type where the opponent's strategy is not fixed. We're talking about games where your opponent can change their strategy in order to exploit weaknesses in your strategy. This property was a key requirement of the "coin toss" example game that he provided in chapter 2. If you modified the coin toss game such that the opponent's strategy was fixed, then the situation would change completely. Crucially, Two Envelopes Game is not one of those games where the "opponent" can adapt their strategy according to your strategy. The strategy of the opponent is fixed in Two Envelopes Game. That's why you can solve a single subgame in Two Envelopes game independently of other subgames, even though it's an imperfect-information game. If you are suspectful of this claim, we can verify it by simulations.