5 ms·

I'll give you Go, but Texas Hold'em? I am only a casual player of both chess and poker, but I don't see any way that the difficulty in building an extremely go

I'll give you Go, but Texas Hold'em? I am only a casual player of both chess and poker, but I don't see any way that the difficulty in building an extremely good automated Texas Hold'em bot is anywhere on the same order of difficulty as building an extremely good Chess bot. What am I missing? Isn't the rage with Hold'em lately that the internet players who play mostly straight are cleaning up vs those who try to read people?

shasta 14y ago
I'm not an expert, but I believe the difficulty with Texas Hold'em is modeling the other human players. "Heads up", where there is an optimal strategy, is essentially solved.
- heyitsnick 14y ago
  This is not the case. Firstly, although it is proven that there is an optimal (game theoretic) strategy to play (Nash), we are not close to 'solving' it. There are attempts to solve limit hold'em, and the current systems beat human players, but they are not close to fully solving the game tree. Attempts to solve big-bet games (NL, PLO) are not even "solved" at a rudimentary level. Modelling human players is the much easier part. There are winning cash game bots in both FL and NL up to midstakes games, and they are a serious issue. Many use default exploitative strategy that perform well against common opponents.
  - bromang 14y ago
    do you know if there have been any attempts to solve other perhaps simpler forms of poker? in particular, five card draw seems like it could be "solved"...
    - betterunix 14y ago
      We discussed this in a course I took a few years ago: https://en.wikipedia.org/wiki/Kuhn_poker https://en.wikipedia.org/wiki/Kuhn_poker
  - shasta 14y ago
    I'm not sure which part of my post this refutes. Nash equilibrium isn't an optimum in the same sense that there is an optimum in the two player game. If you're playing the Nash strategy and everyone else isn't, you can still lose. But the main thrust of my post was that heads up (two players) is mostly solved and you don't seem to address that point. Is that not true? It might not have a closed form solution, but I thought it was approximately solved in that there were algorithms that did no worse than epsilon off ideal play.
    - heyitsnick 14y ago
      > If you're playing the Nash strategy and everyone else isn't, you can still lose. I'm not sure if we're talking in the same language here; you cannot lose (beyond your share of the rake) playing a GT strategy, regardless of what your opponent is doing. This is the basis of what GT is. > But the main thrust of my post was that heads up (two players) is mostly solved and you don't seem to address that point. Is that not true? No, it is not true. Even for FL hold'em, which has a significantly simpler game tree, the best attempts are still far from a complete solution. Large simplifications/assumptions to the game tree need to be made to make it a manageable size computationally. Much simpler representations for HUFLHE have been solved. For all other heads up games, researchers are barely scratching the service; I don't even think there's a clear understanding of how to even go about simplifying the game tree to reduce it to a manageable size to even consider solving it. [I put in the caveat that i (a) haven't read up on the latest in the last couple of years and (b) have not been involved directly with any research projects. I'd love to be corrected from someone who's involved in the latest research.]
      - shasta 14y ago
        > you cannot lose (beyond your share of the rake) playing a GT strategy, regardless of what your opponent is doing. This is the basis of what GT is. What does GT mean here? Not game theoretic, since general Nash equilbria don't have this property. Nash is a strategy where no individual has a motivation to change from the equilibrium. There are multi-player games that have this property that optimal play always results in statistical winnings, but I doubt that poker is one of them. For example, that would imply that collusion isn't effective against an optimal player. Heads up is a different matter. There it's fairly elementary game theory that there exists an unexploitable strategy. A brief search says you're probably right, though, about heads up not being solved.
SeanDav 14y ago
It is extremely difficult to program a computer to respond well to bluffing and other betting patterns. Poker is not just about betting on the probabilities of the cards in front of you but there is a huge element of analysing betting patterns in conjunction with estimated probabilities. This impacts on psychology in addition to maths so is hard to model.
- heyitsnick 14y ago
  I think this is a common misconception with poker. Responding well to bluffing is easy; it's described mathematically by Game Theory. Psychology and "reading" betting patterns is only important if your goal is to maximally exploit your opponent. Solving chess is "simply" the game of finding a game-theoretic solution (computationally, however, this is beyond current computing power; much like "solving" chess is from move 1). For more than 2 players, however, GTO solutions may not exist.
betterunix 14y ago
This might be informative: http://poker.cs.ualberta.ca/ http://poker.cs.ualberta.ca/ Some variants of poker are much easier than others. Texas Hold'em is considered to be one of the most strategically challenging poker games and there has been quite a bit of research on Hold'em bots. The current state of the art is this: computer players can defeat human players below the professional level, but professional players still run circles around the computer.
- heyitsnick 14y ago
  The bot you posted is almost certainly tougher than the best FL heads up players in the world now. I have been a professional player in the game for 5 years now, i'm not the best in the world, but i have discussed strategy with some of them. I don't think any would sit down against this bot, and it is not the toughest (see: annual poker bot competition; rubot i believe is much stronger). There is a HU FL cash game machine in casinos in Nevada. You can find it in the Aria and other places, i think. It is rake free (i.e. the house doesn't have an 'advantage'), it is a fair deal. You can go and play it all day 24/7 if you wish. You can play up to $100/200 and $200/400 i believe, and hands are dealt very fast. Despite some failed attempts, you won't find today any professionals sitting there "running rings" around these machines and making bank.
  - betterunix 14y ago
    Sorry, I should probably have been more clear: I was referring to no-limit games. I did not know that fixed-limit bots had become so advanced that they were beating the best human players, although I am not too surprised (I was paying more attention to this a few years ago). Do you happen to know what the current status of no-limit bots is, or the current status of multiple-player games?
    - heyitsnick 14y ago
      Well the UoA page you linked to is mostly about HU FL, not NL (Polaris, Hyperborian), but they have now expanded into NL and ring games too. I'm not sure on the theory-side how well they compare to humans; however i do know that cash game bots have been discovered beating cash games, both HU and NL, on online sites in the past.

shasta 14y ago

I'm not an expert, but I believe the difficulty with Texas Hold'em is modeling the other human players. "Heads up", where there is an optimal strategy, is essentially solved.

heyitsnick 14y ago
This is not the case. Firstly, although it is proven that there is an optimal (game theoretic) strategy to play (Nash), we are not close to 'solving' it. There are attempts to solve limit hold'em, and the current systems beat human players, but they are not close to fully solving the game tree. Attempts to solve big-bet games (NL, PLO) are not even "solved" at a rudimentary level. Modelling human players is the much easier part. There are winning cash game bots in both FL and NL up to midstakes games, and they are a serious issue. Many use default exploitative strategy that perform well against common opponents.
- bromang 14y ago
  do you know if there have been any attempts to solve other perhaps simpler forms of poker? in particular, five card draw seems like it could be "solved"...
  - betterunix 14y ago
    We discussed this in a course I took a few years ago: https://en.wikipedia.org/wiki/Kuhn_poker https://en.wikipedia.org/wiki/Kuhn_poker
- shasta 14y ago
  I'm not sure which part of my post this refutes. Nash equilibrium isn't an optimum in the same sense that there is an optimum in the two player game. If you're playing the Nash strategy and everyone else isn't, you can still lose. But the main thrust of my post was that heads up (two players) is mostly solved and you don't seem to address that point. Is that not true? It might not have a closed form solution, but I thought it was approximately solved in that there were algorithms that did no worse than epsilon off ideal play.
  - heyitsnick 14y ago
    > If you're playing the Nash strategy and everyone else isn't, you can still lose. I'm not sure if we're talking in the same language here; you cannot lose (beyond your share of the rake) playing a GT strategy, regardless of what your opponent is doing. This is the basis of what GT is. > But the main thrust of my post was that heads up (two players) is mostly solved and you don't seem to address that point. Is that not true? No, it is not true. Even for FL hold'em, which has a significantly simpler game tree, the best attempts are still far from a complete solution. Large simplifications/assumptions to the game tree need to be made to make it a manageable size computationally. Much simpler representations for HUFLHE have been solved. For all other heads up games, researchers are barely scratching the service; I don't even think there's a clear understanding of how to even go about simplifying the game tree to reduce it to a manageable size to even consider solving it. [I put in the caveat that i (a) haven't read up on the latest in the last couple of years and (b) have not been involved directly with any research projects. I'd love to be corrected from someone who's involved in the latest research.]
    - shasta 14y ago
      > you cannot lose (beyond your share of the rake) playing a GT strategy, regardless of what your opponent is doing. This is the basis of what GT is. What does GT mean here? Not game theoretic, since general Nash equilbria don't have this property. Nash is a strategy where no individual has a motivation to change from the equilibrium. There are multi-player games that have this property that optimal play always results in statistical winnings, but I doubt that poker is one of them. For example, that would imply that collusion isn't effective against an optimal player. Heads up is a different matter. There it's fairly elementary game theory that there exists an unexploitable strategy. A brief search says you're probably right, though, about heads up not being solved.

SeanDav 14y ago

It is extremely difficult to program a computer to respond well to bluffing and other betting patterns. Poker is not just about betting on the probabilities of the cards in front of you but there is a huge element of analysing betting patterns in conjunction with estimated probabilities. This impacts on psychology in addition to maths so is hard to model.

heyitsnick 14y ago
I think this is a common misconception with poker. Responding well to bluffing is easy; it's described mathematically by Game Theory. Psychology and "reading" betting patterns is only important if your goal is to maximally exploit your opponent. Solving chess is "simply" the game of finding a game-theoretic solution (computationally, however, this is beyond current computing power; much like "solving" chess is from move 1). For more than 2 players, however, GTO solutions may not exist.

betterunix 14y ago

This might be informative: http://poker.cs.ualberta.ca/ http://poker.cs.ualberta.ca/ Some variants of poker are much easier than others. Texas Hold'em is considered to be one of the most strategically challenging poker games and there has been quite a bit of research on Hold'em bots. The current state of the art is this: computer players can defeat human players below the professional level, but professional players still run circles around the computer.

heyitsnick 14y ago
The bot you posted is almost certainly tougher than the best FL heads up players in the world now. I have been a professional player in the game for 5 years now, i'm not the best in the world, but i have discussed strategy with some of them. I don't think any would sit down against this bot, and it is not the toughest (see: annual poker bot competition; rubot i believe is much stronger). There is a HU FL cash game machine in casinos in Nevada. You can find it in the Aria and other places, i think. It is rake free (i.e. the house doesn't have an 'advantage'), it is a fair deal. You can go and play it all day 24/7 if you wish. You can play up to $100/200 and $200/400 i believe, and hands are dealt very fast. Despite some failed attempts, you won't find today any professionals sitting there "running rings" around these machines and making bank.
- betterunix 14y ago
  Sorry, I should probably have been more clear: I was referring to no-limit games. I did not know that fixed-limit bots had become so advanced that they were beating the best human players, although I am not too surprised (I was paying more attention to this a few years ago). Do you happen to know what the current status of no-limit bots is, or the current status of multiple-player games?
  - heyitsnick 14y ago
    Well the UoA page you linked to is mostly about HU FL, not NL (Polaris, Hyperborian), but they have now expanded into NL and ring games too. I'm not sure on the theory-side how well they compare to humans; however i do know that cash game bots have been discovered beating cash games, both HU and NL, on online sites in the past.