3 ms·

The article says the machine can't be beaten. And then points out that a pro has consistently won against it. This article is filled with bold claims by people

The article says the machine can't be beaten. And then points out that a pro has consistently won against it. This article is filled with bold claims by people that want to sell the idea IMHO. I'm not buying it because even limit texas Hold'Em has never been solved mathematically by super computers, let alone a single machine. Limit Hold'Em is close to being solved but if you check out the last match of pros against a supercomputer you'll find that it's close, but the computer is not a clear winner: http://poker.cs.ualberta.ca/man-machine/Competitors/ http://poker.cs.ualberta.ca/man-machine/Competitors/ I'm just not buying it. Can it beat the average player? Probably. Pros do it everyday. But can it beat a pro consistently and claim a clear victory over humans? To be determined. Edit: I found a take from High Stakes Limit Pro Anthony Rivera on 2p2: http://forumserver.twoplustwo.com/showpost.php?p=29762774&postcount=28 http://forumserver.twoplustwo.com/showpost.php?p=29762774&po... His stance is that it's probably a break-even situation where you are basically playing another pro, and that he'd rather be playing craps. I guess they really have made a great machine. :D

ianstallings 13y ago
(Putting this as its own thread because it's so large and slightly off-topic). I've thought long and hard on this, it's one of my life goals to create a piece of software that could mimic good players in texas hold-em, and I think I might have found a way to do just that using big data, hand histories, neural-networking, and a ton of input by actual players. Or at least a good start. How would I accomplish this? Well a very high-level overview below. Basically it starts out extremely stupid and grows as a player: 1. Dump hand histories from pros into a large database. The number of hand histories would run into the billions. Users could dump online logs in bulk or create one-offs using input software. 2. Create an input system for a single user to choose a random hand-history and then classify it using tags. For instance tags might help categorize the player's style, the opponent's style, the "street" of play they are tagging, common name for the situation (Facing donk-bet on flop after raising pre-flop), etc. I would leave this fairly flexible and allow users to create new tags. Think similar to Galaxy Zoo but a little less rigid. 3. Using these tags/classifications the system would create a poll and present it to users with a question. What would you do in this particular situation? Where that situation is point in time of a hand history. 4. Eventually the computer player would then have a huge number of situations to use as examples with input from humans on how to proceed. This obviously will be very fuzzy and that's where, IMHO, the strength of the bot actually lies. The system would not lead to a rigid "Do X when Y occurs". It would decide from a large range of choices that have been entered by humans in the polls described above, leaning towards the most common answers first, but trying outliers also. 5. Use a neural network to create pathways based on previous successes. How would this work once it's all together? An example: The computer player is dealt AJ off-suit while being last to act on the dealer button. It would ask the database for a set of situations where AJ was dealt to players in the same position. It would then choose one of them and look at the results of the polls. How do most people with the highest success rate play this hand in this particular spot? Choose a random path to take based on that data. Observe and record outcome. This data becomes the true empirical data that the system will eventually rely on. If the situation has been encountered before in it's own play it will look at that and use it or it might choose randomly like it did above from the polling data. Eventually the weight of the empirical data it has collected might outweigh the data from the polls and it will "know" the right move based on its previous pathway choices. If this move is recognized by the opponent and exploited the results would dictate that the system falls back on another random choice. As the data set grows and the system plays the game, it could hypothetically be tuned to play consistently well. The main hurdle is actually user-input. There would need to be incentive for users to enter the data they think is correct. The system is also open to manipulation through input data so that would need to be thwarted. And then on top of that the system would have to play an enormous amount of hands to create known successful pathways. But I think the sheer randomness and human-like qualities of the system would create a truly awesome experience. (Joking) Now, does anyone have the $1M+ USD I need to fund this? I promise I'll pay you back. ;)
- applecore 13y ago
  The major online poker sites would already have a corpus of billions of hands played by successful and unsuccessful players.
- ijk 13y ago
  I wouldn't poll the players, at least not nakedly, since that is vulnerable to the stated preference bias. Tracking actual play would be more likely to demonstrate revealed preference.
- thisisnotatest 13y ago
  I don't see what you get from asking people what they think they'd do in certain situations, when you could instead use the actual outcome of the hand as your training labels.
  - ianstallings 13y ago
    Validation mainly. Because I worry that the hand histories might not be enough and they may be tainted with more examples of bad play than good. I would like to qualify good play with some human validation. But I think it's worth looking at to only use hand histories and if possible only validate when something unexpected results. Maybe then I could use the human input to validate the computer's decision. I'm still hashing that out.
- sanswork 13y ago
  Just replaying hands fails to take into account a lot of variables related to the other players play style. Once to start to take those into account the replaying of hands isn't needed.
  - ianstallings 13y ago
    Yeah I think that's why I would like to add the human input and tagging. Because in a given context it might be a good move, but it also could be classified as a bad move in other contexts. I need to work that out.
- 13y ago
pepve 13y ago
Just a note, you don't have to mathematically solve a game in order to win it.
- ianstallings 13y ago
  I agree that it can win, but so can pros and they make mistakes everyday. But I'm saying that a mistake-free strategy cannot be found without some kind of mathematical breakthrough. Unlike say chess, where clear pathways to a checkmate can be calculated using mathematics, no matter how the opponent plays. It's impossible to do that, right now, using computers, when the opponents are human and unpredictable and don't always follow the optimum strategy. Also poker is an information-sparse environment where you can't always win with a certain strategy because you simply don't have enough information, such as the other player's hole cards.
  - omni 13y ago
    > But I'm saying that a mistake-free strategy cannot be found without some kind of mathematical breakthrough. Is this even possible in a game of imperfect information?
    - jib 13y ago
      Yes, because you know all possible states of the game. Starting out you have 2 cards and you know your opponent has one of 2450 possible combinations of cards. You know exactly how your 2 cards stack up in chances of winning against those 2450 combinations, and you know how much money is on the table initially. Each time an opponent has an action he can take one of two or three actions (call, raise or fold) - your job is to calculate a strategy that makes either of those decisions have the same expected value for your opponent. That is doable, but quite complex, because it requires you to consider not only what your opponent has, but also what possible hands you could have based on your previous actions. The complexity is obviously enormous (hence why it hasnt been computed yet) but it is not impossible to compute by any definition.
    - ianstallings 13y ago
Someone 13y ago
Is it actually possible to mathematically solve those games? Suppose player Alice bets $100; how does player Bob judge that bet? As a strong hand or as a bluff? Any software program will have to model what's inside the head of its opponent, and we know way too little of that to do that. Moreover, if we could, the opponent may choose a different strategy. It may be like a game with nontransitive dice: if you figure out which dice I use, you can beat me, but if I figure out that you figured that out, I can pick different dice and beat you (http://en.wikipedia.org/wiki/Nontransitive_dice http://en.wikipedia.org/wiki/Nontransitive_dice) I think it is possible to write software that statistically beats any opponent, but proving that it does is way harder.
- ianstallings 13y ago
  Yeah I think that's the main hurdle actually, the information-sparsity problem. We never really know what the opponent holds, unlike chess where nothing can be hidden. Also there is the case of non-optimal play from an opponent and modeling that would be incredibly challenging.
- jib 13y ago
  Yes it is possible to mathematically solve limit poker (as well as no-limit poker, but no-limit poker is orders of magnitude more difficult - they are effectively different games). The definition of solvable is "Does there exist a strategy that, regardless of opponents play, is not a losing strategy" though (because the game is symmetrical). It is not possible to solve for "Does there exist a strategy that, regardless of opponents play, guarantees maximum profits". You can solve it for not losing money, but you cant solve it for making the max amount of money. Trivially it's not possible to write software that beats any opponent (because what would it do playing against itself?) . Less trivially, any game that has a finite amount of decisions (and limit hold-em does) has at least one Nash equilibrium, so there exists a strategy that will at least have you break even. The way to solve the game is to calculate your odds of winning based on previous actions and ensure that you take actions that make any future decision of the opponent have the same outcome (to reach a Nash equilibrium). That strategy hasnt been calculated yet, but the best limit players are most likely playing very close to it, at least if you compare to the best no-limit players playing no-limit (the variable amounts possible to bid in no-limit multiplies the possible strategies massively).
albedoa 13y ago
The more important post might be this one from Pokeraddict: http://forumserver.twoplustwo.com/showpost.php?p=25330375&postcount=98 http://forumserver.twoplustwo.com/showpost.php?p=25330375&po...
psutor 13y ago
IIRC, the article only mentioned the "dumbed down" version now in casinos being beaten by pros. The earlier part of the article (when it was full strength and in testing) has no mention of it being beaten.

ianstallings 13y ago

(Putting this as its own thread because it's so large and slightly off-topic). I've thought long and hard on this, it's one of my life goals to create a piece of software that could mimic good players in texas hold-em, and I think I might have found a way to do just that using big data, hand histories, neural-networking, and a ton of input by actual players. Or at least a good start. How would I accomplish this? Well a very high-level overview below. Basically it starts out extremely stupid and grows as a player: 1. Dump hand histories from pros into a large database. The number of hand histories would run into the billions. Users could dump online logs in bulk or create one-offs using input software. 2. Create an input system for a single user to choose a random hand-history and then classify it using tags. For instance tags might help categorize the player's style, the opponent's style, the "street" of play they are tagging, common name for the situation (Facing donk-bet on flop after raising pre-flop), etc. I would leave this fairly flexible and allow users to create new tags. Think similar to Galaxy Zoo but a little less rigid. 3. Using these tags/classifications the system would create a poll and present it to users with a question. What would you do in this particular situation? Where that situation is point in time of a hand history. 4. Eventually the computer player would then have a huge number of situations to use as examples with input from humans on how to proceed. This obviously will be very fuzzy and that's where, IMHO, the strength of the bot actually lies. The system would not lead to a rigid "Do X when Y occurs". It would decide from a large range of choices that have been entered by humans in the polls described above, leaning towards the most common answers first, but trying outliers also. 5. Use a neural network to create pathways based on previous successes. How would this work once it's all together? An example: The computer player is dealt AJ off-suit while being last to act on the dealer button. It would ask the database for a set of situations where AJ was dealt to players in the same position. It would then choose one of them and look at the results of the polls. How do most people with the highest success rate play this hand in this particular spot? Choose a random path to take based on that data. Observe and record outcome. This data becomes the true empirical data that the system will eventually rely on. If the situation has been encountered before in it's own play it will look at that and use it or it might choose randomly like it did above from the polling data. Eventually the weight of the empirical data it has collected might outweigh the data from the polls and it will "know" the right move based on its previous pathway choices. If this move is recognized by the opponent and exploited the results would dictate that the system falls back on another random choice. As the data set grows and the system plays the game, it could hypothetically be tuned to play consistently well. The main hurdle is actually user-input. There would need to be incentive for users to enter the data they think is correct. The system is also open to manipulation through input data so that would need to be thwarted. And then on top of that the system would have to play an enormous amount of hands to create known successful pathways. But I think the sheer randomness and human-like qualities of the system would create a truly awesome experience. (Joking) Now, does anyone have the $1M+ USD I need to fund this? I promise I'll pay you back. ;)

applecore 13y ago
The major online poker sites would already have a corpus of billions of hands played by successful and unsuccessful players.
ijk 13y ago
I wouldn't poll the players, at least not nakedly, since that is vulnerable to the stated preference bias. Tracking actual play would be more likely to demonstrate revealed preference.
thisisnotatest 13y ago
I don't see what you get from asking people what they think they'd do in certain situations, when you could instead use the actual outcome of the hand as your training labels.
- ianstallings 13y ago
  Validation mainly. Because I worry that the hand histories might not be enough and they may be tainted with more examples of bad play than good. I would like to qualify good play with some human validation. But I think it's worth looking at to only use hand histories and if possible only validate when something unexpected results. Maybe then I could use the human input to validate the computer's decision. I'm still hashing that out.
sanswork 13y ago
Just replaying hands fails to take into account a lot of variables related to the other players play style. Once to start to take those into account the replaying of hands isn't needed.
- ianstallings 13y ago
  Yeah I think that's why I would like to add the human input and tagging. Because in a given context it might be a good move, but it also could be classified as a bad move in other contexts. I need to work that out.
13y ago

pepve 13y ago

Just a note, you don't have to mathematically solve a game in order to win it.

ianstallings 13y ago
I agree that it can win, but so can pros and they make mistakes everyday. But I'm saying that a mistake-free strategy cannot be found without some kind of mathematical breakthrough. Unlike say chess, where clear pathways to a checkmate can be calculated using mathematics, no matter how the opponent plays. It's impossible to do that, right now, using computers, when the opponents are human and unpredictable and don't always follow the optimum strategy. Also poker is an information-sparse environment where you can't always win with a certain strategy because you simply don't have enough information, such as the other player's hole cards.
- omni 13y ago
  > But I'm saying that a mistake-free strategy cannot be found without some kind of mathematical breakthrough. Is this even possible in a game of imperfect information?
  - jib 13y ago
    Yes, because you know all possible states of the game. Starting out you have 2 cards and you know your opponent has one of 2450 possible combinations of cards. You know exactly how your 2 cards stack up in chances of winning against those 2450 combinations, and you know how much money is on the table initially. Each time an opponent has an action he can take one of two or three actions (call, raise or fold) - your job is to calculate a strategy that makes either of those decisions have the same expected value for your opponent. That is doable, but quite complex, because it requires you to consider not only what your opponent has, but also what possible hands you could have based on your previous actions. The complexity is obviously enormous (hence why it hasnt been computed yet) but it is not impossible to compute by any definition.
  - ianstallings 13y ago

Someone 13y ago

Is it actually possible to mathematically solve those games? Suppose player Alice bets $100; how does player Bob judge that bet? As a strong hand or as a bluff? Any software program will have to model what's inside the head of its opponent, and we know way too little of that to do that. Moreover, if we could, the opponent may choose a different strategy. It may be like a game with nontransitive dice: if you figure out which dice I use, you can beat me, but if I figure out that you figured that out, I can pick different dice and beat you (http://en.wikipedia.org/wiki/Nontransitive_dice http://en.wikipedia.org/wiki/Nontransitive_dice) I think it is possible to write software that statistically beats any opponent, but proving that it does is way harder.

ianstallings 13y ago
Yeah I think that's the main hurdle actually, the information-sparsity problem. We never really know what the opponent holds, unlike chess where nothing can be hidden. Also there is the case of non-optimal play from an opponent and modeling that would be incredibly challenging.
jib 13y ago
Yes it is possible to mathematically solve limit poker (as well as no-limit poker, but no-limit poker is orders of magnitude more difficult - they are effectively different games). The definition of solvable is "Does there exist a strategy that, regardless of opponents play, is not a losing strategy" though (because the game is symmetrical). It is not possible to solve for "Does there exist a strategy that, regardless of opponents play, guarantees maximum profits". You can solve it for not losing money, but you cant solve it for making the max amount of money. Trivially it's not possible to write software that beats any opponent (because what would it do playing against itself?) . Less trivially, any game that has a finite amount of decisions (and limit hold-em does) has at least one Nash equilibrium, so there exists a strategy that will at least have you break even. The way to solve the game is to calculate your odds of winning based on previous actions and ensure that you take actions that make any future decision of the opponent have the same outcome (to reach a Nash equilibrium). That strategy hasnt been calculated yet, but the best limit players are most likely playing very close to it, at least if you compare to the best no-limit players playing no-limit (the variable amounts possible to bid in no-limit multiplies the possible strategies massively).

albedoa 13y ago

The more important post might be this one from Pokeraddict: http://forumserver.twoplustwo.com/showpost.php?p=25330375&postcount=98 http://forumserver.twoplustwo.com/showpost.php?p=25330375&po...

psutor 13y ago

IIRC, the article only mentioned the "dumbed down" version now in casinos being beaten by pros. The earlier part of the article (when it was full strength and in testing) has no mention of it being beaten.