> I feel like this is a detour that we can avoid by adding a small cost to switching. I don't think its a detour we actually want to avoid; the implications ar

> I feel like this is a detour that we can avoid by adding a small cost to switching. I don't think its a detour we actually want to avoid; the implications are really fascinating and help us to understand how to solve games more generally. I agree with adding cost as a way to do it. Actually, that is why I've been saying it is zero or undefined. You know about dynamic programming right? Well, one of the reasons it was invented is kind of related to what we're talking about right now. There are these things called the bellman equations. They can look like this when you think of things in terms of a markov decision process. https://wikimedia.org/api/rest_v1/media/math/render/svg/0e358db69fc04ad39781acdc8300059e7a93791a https://wikimedia.org/api/rest_v1/media/math/render/svg/0e35... Since they're recursively defined, you can compute them more quickly by caching the computation back to front then by computing them front to back. Anyways, putting aside that bit of trivia, do you see the symbol that looks a bit like a y? When we have an infinite sequence like we would if we kept switching you can set that term to be some constant, for example 0.9999999999999. That lets you take the limit, because the infinite sequence is obviously going to converge to zero. Check out the formula again and notice one of the things that it does which the wikipedia article doesn't. Do you see the symbol for pi? That is talking about the concept of a policy function. A strategy, that way you compute what the agent would get if they played the game, rather than the envelope contents. Under this formalism we've doing an argmax over the policy function for the equation in order to get the policy that has the highest expected value. > I love it. I love it in the same way I love the trick of a magician who fools me. If you're actually interested in this sort of thing, I suggest checking out the poker research papers by Noam Brown. He had a talk at NIPS 2017 where he won best paper award. In 2019 he was runner up for best scientific advancement of the year. His work is about applying game simplification to poker to create a bluneprint of the game which is simpler, solving the blueprint using a modified form of counterfactual regret minimization, and then refining the solution during actual play with reach subgame solving. You might have heard of his work, because he was involved in the entire AI now better than humans at poker thing even in no limit breakthrough. I have some belief, but not certainty, that his paper contains a correction of someone who made this style of error. He mentions in the research paper that there was a mistake in another paper where they didn't account for the way subgames influence each other. I see that as the problem here, but I haven't read the other paper, so I don't know if it was the same category of error. Find the paper he referenced, read through it, and see if you're tricked. It is potentially a kind of real world two envelope problem to see if your tools for reasoning about these things is actually helping you to avoid the error in more complicated situations - though, since you disagree with me that this is about handling of imperfect information (or more precisely, the counterfactuals - there is a variant on wikipedia where they removed the probabilites, but counterfactual reasoning still applied to resolve the paradox) maybe you won't see it as related errors.