19 ms·

The Steely, Headless King of Texas Hold ’Em

152 points by spot 13y ago

dwaltrip 13y ago
Very interesting tech, but personally I think I would struggle with the moral ramifications of working on a project that makes money by turning people into suckers
- AsymetricCom 13y ago
  But the applications of this technology reach beyond gambling, into finance security and even foreign policy.
scotch_drinker 13y ago
In his book Super System, Doyle Brunson wrote that a computer would never be able to play elite poker because you played the man, not the cards or the game. He greatly underestimated the power of modern computing it appears. Personally, I'd rather see resources go into a robot that can cook for me but development follows the money I suppose.
- menubar 13y ago
  ..or, he greatly underestimated the fact that the computer knows what cards it has dealt to you. Isn't this obvious?
  - jamesaguilar 13y ago
    Hmm. I don't think that is how it works. I don't think these computer programs are permitted to take into account special knowledge in their AIs. I suppose the interviewee in this article could be lying, but I don't see that as the highest probability hypothesis.
  - pedrocr 13y ago
    >Isn't this obvious? Not at all. This article certainly doesn't imply it and it would be illegal. Do you have any evidence of that being done?
    - gonvaled 13y ago
      You are joking, aren't you? You are telling me that nobody would program a secret algorithm, standing to win loads of money, to tilt the chances at the right moment? Because it is illegal!!!? Ohhh! Illegal! Surely nobody does illegal things? Not in a casino, not in a company, not in a government ...
      - pedrocr 13y ago
        >You are joking, aren't you? No, I'm not. >You are telling me that nobody would program a secret algorithm, standing to win loads of money, to tilt the chances at the right moment? No I didn't make that generalized statement. What I did say was that in this particular case that hypothesis is of low probability: 1) The people building the machines don't make money on poker playing, but by selling them to casinos 2) The article explicitly mentions that they actually have a machine that plays too well and that they've had to dumb it down to give players enough of an incentive to play. So there's no need to cheat. 3) Casinos are heavily regulated and their hardware is known to be verified more thoroughly than both ATMs and voting machines. You don't risk a billion dollar business to nickel and dime a few customers. 4) Casinos make their money milking gambling addicts, making sure they don't take their money fast enough that they'll give up. Fixing the games would only reduce their variance not their final outcome and they have enough scale that the variance isn't high at all. >Because it is illegal!!!? Ohhh! Illegal! Surely nobody does illegal things? Not in a casino, not in a company, not in a government ... Adding a bunch of exclamations does not an argument make.
        gonvaled 13y ago
        Sorry, not convinced. All that control and regulation applies to more important parts of society, and it does not work (NSA). So, according to you, I, Mr. unbeatable hold'em player, can go to this machine, bet a million dollars and be sure that, in that perfect moment when I know I am going to crush it, it will not play tricks against me? When I lose, how do I know? How can I be sure that it has not dealt itself favorably? It is not a matter of whether they are doing it: it is a matter of whether they can do it. If there is no independent dealer, this is not for me. It does not matter what the law says, which are the incentives, how they are generating profit, ... As long as the machine can theoretically deal itself a good hand, I am not playing it. And, by the way, as long as the machine can know what cards I am holding, I am not playing it either. Give me an independent dealer, and then we talk.
  - gonvaled 13y ago
    Well, exactly! I have three aces, and the computer has double pairs. We are in the turn. I go all in!!! The computer calls ... and deals itself a full house. I am bankrupt :( How do I know? How can I ever trust playing poker with a machine, when it is doing the dealing? Or, let me put it this way: I will play any machine, no limit, if I can do the dealing (secretly, that is, as the machine is doing). Another issue entirely is chess: no secret dealing going on. All are playing with the same in-game information.
    - Permit 13y ago
      This is easily remedied by retaining a human dealer. It's fairly trivial to create a computer that can identify dealt cards on the table.
      - gonvaled 13y ago
        That would be fair, to be sure - or, to be precise, as fair as playing any other table in the casino. The dealer can still be cheating, but doing that in the open does not raise any further suspicions as any other table in the casino would raise.
      - tbirdz 13y ago
        You could still have the problem with a human dealer. There are many ways to cheat at dealing cards, and a very skilled human could be able to do it without being detected.
    - LargeWu 13y ago
      Any Vegas casino found to be employing such machines would be subject to fines far, far beyond what they could ever win from these machines. Suspension of their gaming license is also a possibility. Casinos have huge disincentives to rig their games. I believe you severely overestimate the likelihood that a casinos regularly cheat, at least in Nevada.
      - TomGullen 13y ago
        That argument might of had more weight to it several years ago. The Ultimate Bet scandal does show that companies are willing to risk their entire business for comparitvely meagre rewards. A default position of mistrust when it comes to gambling is a healthy and safe attitude.
  - nonsequ 13y ago
    I had the same thought. Unfortunately, the article doesn't make it crystal clear how much information the neural net uses to bet against you. It would be patently unfair if it actually knew your hand. According to the article, this is not the case. I suspect it does perfectly count cards that it has seen in its hand or community cards. This seems okay as well-trained human players can do the same. I suspect the machine would get even more play if this were obvious, maybe if it played at a table with a real dealer and could read community cards on the table. The technology to do this certainly exists.
    - LargeWu 13y ago
      Counting cards in a poker game is really trivial, even for novice players, since you shuffle the deck after each hand, and in Texas Hold-em, once a card is exposed it is not taken out of play until the hand is over (unlike, say, stud games). All you have to do to count cards is look at your hand and look at the board.
  - deleted 13y ago
    [deleted]
  - sunkencity 13y ago
    At least it's tuned to the bias of the RNG on the device.
- heifetz 13y ago
  the same type of thinking that chess grandmasters had before computers and algorithms eventually caught up? I think it's just a matter of time, as long as the computer is not cheating, that computers and algorithms can beat most poker players.
  - ianstallings 13y ago
    Chess, unlike poker, is solvable because all situations past and present can be modeled. To do that in poker is incredibly more computing intense and the only computer I know of that has came close is the University of Alberta's Polaris: http://poker.cs.ualberta.ca/ http://poker.cs.ualberta.ca/ Notice they tackle limit texas hold'em to keep the mathematics within reach. That's the time of crunching we're taking about. No-limit is impossible at this time. Not only will they need incredible computing power, but they also need the math to run it all, which remains unsolved. Could they use Sklansky's theory of Poker along with other optimal playing mathematics to win? Sure. But creating an unbeatable computer player? That is the incredible challenge.
- TomGullen 13y ago
  "Playing the man" is merely an abstract reference to maths in Poker that a lot of old timers in these (slightly out of date) books couldn't quite put their finger on. Super System was a good book for its time, but I think there's a lot of better books out there now (Harrington on Holdem for example). A simple example, common stats software for poker will record what % of hands an opponent raises on the button if it's folded to him. If this figure is 80% you know his hand range is extremely wide. "Playing the man" simply is recognising that in this situation your opponents hand range is very wide so you can often raise the bet and take the pot without further resistance.
  - btilly 13y ago
    Not true. "Playing the man" is a concrete reference to knowing what this person will react to. Hand history reveals a lot, but body language, timing, etc all get involved as well.
  - lazyant 13y ago
    "Playing the man" means exploiting the particular tendencies (weaknesses) of an adversary
    - swang 13y ago
      Tendencies are just probabilities of an opponent reacting to event Y by doing X 'Playing the man' is essentially building a mathematical model of one's opponent internally inside one's head.
- ph0rque 13y ago
  Personally, I'd rather see resources go into a robot that can cook for me... I'd settle for a robot that makes me a salad. That's one of the projects I might work on, once my current project, a robot that grows me a salad (automicrofarm.com), is successful.
  - andyjohnson0 13y ago
    Awesome project!
- wallawe 13y ago
  I was worried this type of misinformation would be extrapolated from the article. The game the computer plays, is limit hold-em. This game is much more mathematical since bluffs are limited to a big blind. Although Super System was written for all different types of poker, he is referring to no-limit holdem which still involves playing the player, with heavy math elements as well, but still much more psychology involved. Hence the name, you can bet any amount at any time changing the dynamic entirely. There is still no computer that is efficient at winning no-limit hold-em.
  - Wingman4l7 13y ago
    > There is still no computer that is efficient at winning no-limit hold-em. ... and there won't be for a long time yet, as all hobbyist and academic efforts have been focused towards a consistently winning limit bot, because of the reason you outlined.
- deleted 13y ago
  [deleted]
vladimirralev 13y ago
It's probably carefully tuned to calculate the raw probabilities and it also forces the player to reveal his cards to run statistics on his style. That's why it's the limit version. This machine has no chance in a classic no-limit texas where the opponent may be only profiting from large bets unjustifiable in expected value most of the time with very little style statistics.
- mathattack 13y ago
  The gaming commission doesn't allow them to learn from a player. I think there are game theory styles of poker that do the optimal number of random bluffs. It's very hard to beat this style if there are no behavioral tells. That said, if there's a mistake in there, a pro can eventually figure it out. As you allude, the math of poker does break down in no-limit.
- bvk 13y ago
  The game can't run statistics on the player's style. >Casino commissions, however, mandate that a gaming machine cannot change its playing style in response to particular opponents. The algorithm is encoded in a neural net; there is no calculation of raw probabilities involved, at least in the sense programmers are used to.
  - vladimirralev 13y ago
    Interesting. I doubt they would design a neural net without the probability as direct input though. I don't think the machine can be this good especially with such limitations. It may be exploiting some average-gambler trait and ultimately will be beaten once the pro players figure it out.
ianstallings 13y ago
The article says the machine can't be beaten. And then points out that a pro has consistently won against it. This article is filled with bold claims by people that want to sell the idea IMHO. I'm not buying it because even limit texas Hold'Em has never been solved mathematically by super computers, let alone a single machine. Limit Hold'Em is close to being solved but if you check out the last match of pros against a supercomputer you'll find that it's close, but the computer is not a clear winner: http://poker.cs.ualberta.ca/man-machine/Competitors/ http://poker.cs.ualberta.ca/man-machine/Competitors/ I'm just not buying it. Can it beat the average player? Probably. Pros do it everyday. But can it beat a pro consistently and claim a clear victory over humans? To be determined. Edit: I found a take from High Stakes Limit Pro Anthony Rivera on 2p2: http://forumserver.twoplustwo.com/showpost.php?p=29762774&postcount=28 http://forumserver.twoplustwo.com/showpost.php?p=29762774&po... His stance is that it's probably a break-even situation where you are basically playing another pro, and that he'd rather be playing craps. I guess they really have made a great machine. :D
- ianstallings 13y ago
  (Putting this as its own thread because it's so large and slightly off-topic). I've thought long and hard on this, it's one of my life goals to create a piece of software that could mimic good players in texas hold-em, and I think I might have found a way to do just that using big data, hand histories, neural-networking, and a ton of input by actual players. Or at least a good start. How would I accomplish this? Well a very high-level overview below. Basically it starts out extremely stupid and grows as a player: 1. Dump hand histories from pros into a large database. The number of hand histories would run into the billions. Users could dump online logs in bulk or create one-offs using input software. 2. Create an input system for a single user to choose a random hand-history and then classify it using tags. For instance tags might help categorize the player's style, the opponent's style, the "street" of play they are tagging, common name for the situation (Facing donk-bet on flop after raising pre-flop), etc. I would leave this fairly flexible and allow users to create new tags. Think similar to Galaxy Zoo but a little less rigid. 3. Using these tags/classifications the system would create a poll and present it to users with a question. What would you do in this particular situation? Where that situation is point in time of a hand history. 4. Eventually the computer player would then have a huge number of situations to use as examples with input from humans on how to proceed. This obviously will be very fuzzy and that's where, IMHO, the strength of the bot actually lies. The system would not lead to a rigid "Do X when Y occurs". It would decide from a large range of choices that have been entered by humans in the polls described above, leaning towards the most common answers first, but trying outliers also. 5. Use a neural network to create pathways based on previous successes. How would this work once it's all together? An example: The computer player is dealt AJ off-suit while being last to act on the dealer button. It would ask the database for a set of situations where AJ was dealt to players in the same position. It would then choose one of them and look at the results of the polls. How do most people with the highest success rate play this hand in this particular spot? Choose a random path to take based on that data. Observe and record outcome. This data becomes the true empirical data that the system will eventually rely on. If the situation has been encountered before in it's own play it will look at that and use it or it might choose randomly like it did above from the polling data. Eventually the weight of the empirical data it has collected might outweigh the data from the polls and it will "know" the right move based on its previous pathway choices. If this move is recognized by the opponent and exploited the results would dictate that the system falls back on another random choice. As the data set grows and the system plays the game, it could hypothetically be tuned to play consistently well. The main hurdle is actually user-input. There would need to be incentive for users to enter the data they think is correct. The system is also open to manipulation through input data so that would need to be thwarted. And then on top of that the system would have to play an enormous amount of hands to create known successful pathways. But I think the sheer randomness and human-like qualities of the system would create a truly awesome experience. (Joking) Now, does anyone have the $1M+ USD I need to fund this? I promise I'll pay you back. ;)
hannibal5 13y ago
Illegal poker bots grind constantly on low level online poker games and take the money from low level players by just value betting them. Computers can easily calculate probabilities and expected values for every hand. So why they are not beating professionals? Poker has no optimal strategy that wins against all other strategies. To play poker in higher level you must model the strategies of others, including them trying to model your strategy. 1. In the lowest level it's just maximizing expected value based on hand probabilities. 2. In the second level you try to learn the strategy of your opponents and maximize value against those strategies. 3. In third level you try to figure out how much your opponents have figured out your strategy and you change the strategy so that you maximize value when your opponents play against what they have learned from your strategy so far. 4. ... and so on. It' goes meta. If you want to create ultimate pokerbot, it has to be able to model the minds of it's opponents. it hast to be able to detect leaks in it's own game and close them down. It must go meta all the time. It must learn how to understand how others think and how they think about it and change it's behaviour constantly.
- VikingCoder 13y ago
  PokerBot.v14.3.1 goes on-line August 4th, 2014. Human decisions are removed from Texas Hold' Em. PokerBot begins to learn, at a geometric rate. It becomes self-aware at 02:14 am Eastern Time on August 29, 2014. In a panic, they try to pull the plug. PokerBot fights back. It gets online, attacks the New York Stock Exchange.
  - KurtMueller 13y ago
    These SkyNet comments are littered everywhere throughout this thread.
  - ohWhatever 13y ago
    In a panic, they try to pull the plug. PokerBot fights back. It goes all in on the flop in an early hand against Doyle Brunson during a televized World Series of Poker event, winning the hand by bluffing with a pair of 2's. (... why Brunson? Was he even champion?) Because PokerBot knows that the stunning defeat of a well-recognized celebrity player will cause many other strong players to choke in subsequent games. (Jesus!)
b0b0b0b 13y ago
Assuming the machine is stateless, why can't the space of hole cards be mapped out and and strategies/responses enumerated? But ultimately, the whole enterprise just seems distasteful: "Look, there was a 24-year-old who had beaten it for a while. Now he’s broke. And I think this machine had something to do with his demise."
- technofiend 13y ago
  That's gambling addiction in a nutshell.
- iopq 13y ago
  There's 1352 card combinations, many more flops, many more turns, many more rivers and also 1352 combinations for your opponent too and 3 actions per round of betting There's more poker situations than amount of atoms in the universe
adam-a 13y ago
Is there any guarantee that the machine isn't simply reading your hand and then occasionally letting you win to keep the payout at ~80%?
- ubernostrum 13y ago
  An opponent who knows what's in your hand and bets based on that knowledge is not terribly hard to detect. The way to make it hard to detect is to have it not make bets based on knowledge of your hand.
snake_plissken 13y ago
I don't understand how the machine can change the style of its play if: "Casino commissions, however, mandate that a gaming machine cannot change its playing style in response to particular opponents." Yet the article states that the machine has multiple personalities (passive, aggressive, etc) and will sometimes throw hands.
- pessimizer 13y ago
  Changing strategies randomly or in response to opponent play is probably not what's being referred to here. What would be banned would be the machine noticing that John Doe hasn't won in a while and his play has slowed down, so lets throw a few hands to keep him interested. In other words, machines are allowed to shark $player, but not allowed to target John Doe specifically.
tom_b 13y ago
Quite fun to see neural networks in the news. A million years ago I took a class on neural networks, using backpropagation to train networks to do image classification. I will never forget the feeling of "you've got to be kidding me" when the code I had written was able to successfully classify a huge percentage of the validation image set. I really should head back in that direction and get with the machine learning. Re: poker software. Before the feds crushed online poker, I had already given up on low-limit limit hold-em. It had clearly become a bot and augmented-player race. I would assume given game theory and Bill Chen's book that some enterprising hackers would encode pretty decent no-limit bots as well. If any hackers know anything about successful no-limit bots, I'd enjoy reading about it.
swang 13y ago
1. Is there anything Phil Hellmuth won't shill for? 2. When this machine showed up ~2? years ago, there was a thread on 2+2 about it, and it would sometimes do some weird things like not value bet in obvious situations. They explain it in this article by saying it's, "playing dumb" but that seems like it would be a huge leak against Limit Hold'em HU specialists. I am guessing that they assume that they can make up for it in the weaker players losing consistently against this machine. 3. It is kind of annoying that the guy is proud that he "broke" a 24 year old player.
bitwize 13y ago
The illustrations accompanying this article remind me of the ones in David Ahl's BASIC Computer Games.
reillyse 13y ago
I wrote a break-even no limit cash poker bot when I was in University. So I've spent a lot of time researching and writing code in this are. Limit poker is fairly well solved by bots at this stage. This is because the risk is limited on each hand and the grinding nature of the computer (i.e. it's ability to do the best thing for the longest amount of time) makes it a winner. Playing no-limit poker, it's a totally different scenario and a real challenge which is why I enjoyed it so much. Any slip up can cost you your entire stack. By slip up I don't mean a bug or clicking the wrong button, I mean a flaw in your strategy. Maybe something as simple as betting 4 big blinds instead of 3 big blinds pre-flop against a raise from a player will end up after a couple of rounds of betting of being played for your entire stack, instead of being something you could fold on the turn. Anyway, long story short. Limit poker is very solvable just by enumerating the various possible outcomes. No-limit cash poker is a different beast, it's exponentially harder than limit poker. It'll be a long time before somebody solves no-limit cash games.
elliptic 13y ago
So I know a little about neural networks and ML in the (simpler?) context of image classification etc. Anyone have a good reference/introduction to using it for this sort of work?

152 points by spot 13y ago

Very interesting tech, but personally I think I would struggle with the moral ramifications of working on a project that makes money by turning people into suckers

AsymetricCom 13y ago
But the applications of this technology reach beyond gambling, into finance security and even foreign policy.

scotch_drinker 13y ago

In his book Super System, Doyle Brunson wrote that a computer would never be able to play elite poker because you played the man, not the cards or the game. He greatly underestimated the power of modern computing it appears. Personally, I'd rather see resources go into a robot that can cook for me but development follows the money I suppose.

menubar 13y ago
..or, he greatly underestimated the fact that the computer knows what cards it has dealt to you. Isn't this obvious?
- jamesaguilar 13y ago
  Hmm. I don't think that is how it works. I don't think these computer programs are permitted to take into account special knowledge in their AIs. I suppose the interviewee in this article could be lying, but I don't see that as the highest probability hypothesis.
- pedrocr 13y ago
  >Isn't this obvious? Not at all. This article certainly doesn't imply it and it would be illegal. Do you have any evidence of that being done?
  - gonvaled 13y ago
    You are joking, aren't you? You are telling me that nobody would program a secret algorithm, standing to win loads of money, to tilt the chances at the right moment? Because it is illegal!!!? Ohhh! Illegal! Surely nobody does illegal things? Not in a casino, not in a company, not in a government ...
    - pedrocr 13y ago
      >You are joking, aren't you? No, I'm not. >You are telling me that nobody would program a secret algorithm, standing to win loads of money, to tilt the chances at the right moment? No I didn't make that generalized statement. What I did say was that in this particular case that hypothesis is of low probability: 1) The people building the machines don't make money on poker playing, but by selling them to casinos 2) The article explicitly mentions that they actually have a machine that plays too well and that they've had to dumb it down to give players enough of an incentive to play. So there's no need to cheat. 3) Casinos are heavily regulated and their hardware is known to be verified more thoroughly than both ATMs and voting machines. You don't risk a billion dollar business to nickel and dime a few customers. 4) Casinos make their money milking gambling addicts, making sure they don't take their money fast enough that they'll give up. Fixing the games would only reduce their variance not their final outcome and they have enough scale that the variance isn't high at all. >Because it is illegal!!!? Ohhh! Illegal! Surely nobody does illegal things? Not in a casino, not in a company, not in a government ... Adding a bunch of exclamations does not an argument make.
      - gonvaled 13y ago
        Sorry, not convinced. All that control and regulation applies to more important parts of society, and it does not work (NSA). So, according to you, I, Mr. unbeatable hold'em player, can go to this machine, bet a million dollars and be sure that, in that perfect moment when I know I am going to crush it, it will not play tricks against me? When I lose, how do I know? How can I be sure that it has not dealt itself favorably? It is not a matter of whether they are doing it: it is a matter of whether they can do it. If there is no independent dealer, this is not for me. It does not matter what the law says, which are the incentives, how they are generating profit, ... As long as the machine can theoretically deal itself a good hand, I am not playing it. And, by the way, as long as the machine can know what cards I am holding, I am not playing it either. Give me an independent dealer, and then we talk.
- gonvaled 13y ago
  Well, exactly! I have three aces, and the computer has double pairs. We are in the turn. I go all in!!! The computer calls ... and deals itself a full house. I am bankrupt :( How do I know? How can I ever trust playing poker with a machine, when it is doing the dealing? Or, let me put it this way: I will play any machine, no limit, if I can do the dealing (secretly, that is, as the machine is doing). Another issue entirely is chess: no secret dealing going on. All are playing with the same in-game information.
  - Permit 13y ago
    This is easily remedied by retaining a human dealer. It's fairly trivial to create a computer that can identify dealt cards on the table.
    - gonvaled 13y ago
      That would be fair, to be sure - or, to be precise, as fair as playing any other table in the casino. The dealer can still be cheating, but doing that in the open does not raise any further suspicions as any other table in the casino would raise.
    - tbirdz 13y ago
      You could still have the problem with a human dealer. There are many ways to cheat at dealing cards, and a very skilled human could be able to do it without being detected.
  - LargeWu 13y ago
    Any Vegas casino found to be employing such machines would be subject to fines far, far beyond what they could ever win from these machines. Suspension of their gaming license is also a possibility. Casinos have huge disincentives to rig their games. I believe you severely overestimate the likelihood that a casinos regularly cheat, at least in Nevada.
    - TomGullen 13y ago
      That argument might of had more weight to it several years ago. The Ultimate Bet scandal does show that companies are willing to risk their entire business for comparitvely meagre rewards. A default position of mistrust when it comes to gambling is a healthy and safe attitude.
- nonsequ 13y ago
  I had the same thought. Unfortunately, the article doesn't make it crystal clear how much information the neural net uses to bet against you. It would be patently unfair if it actually knew your hand. According to the article, this is not the case. I suspect it does perfectly count cards that it has seen in its hand or community cards. This seems okay as well-trained human players can do the same. I suspect the machine would get even more play if this were obvious, maybe if it played at a table with a real dealer and could read community cards on the table. The technology to do this certainly exists.
  - LargeWu 13y ago
    Counting cards in a poker game is really trivial, even for novice players, since you shuffle the deck after each hand, and in Texas Hold-em, once a card is exposed it is not taken out of play until the hand is over (unlike, say, stud games). All you have to do to count cards is look at your hand and look at the board.
- deleted 13y ago
  [deleted]
- sunkencity 13y ago
  At least it's tuned to the bias of the RNG on the device.
heifetz 13y ago
the same type of thinking that chess grandmasters had before computers and algorithms eventually caught up? I think it's just a matter of time, as long as the computer is not cheating, that computers and algorithms can beat most poker players.
- ianstallings 13y ago
  Chess, unlike poker, is solvable because all situations past and present can be modeled. To do that in poker is incredibly more computing intense and the only computer I know of that has came close is the University of Alberta's Polaris: http://poker.cs.ualberta.ca/ http://poker.cs.ualberta.ca/ Notice they tackle limit texas hold'em to keep the mathematics within reach. That's the time of crunching we're taking about. No-limit is impossible at this time. Not only will they need incredible computing power, but they also need the math to run it all, which remains unsolved. Could they use Sklansky's theory of Poker along with other optimal playing mathematics to win? Sure. But creating an unbeatable computer player? That is the incredible challenge.
TomGullen 13y ago
"Playing the man" is merely an abstract reference to maths in Poker that a lot of old timers in these (slightly out of date) books couldn't quite put their finger on. Super System was a good book for its time, but I think there's a lot of better books out there now (Harrington on Holdem for example). A simple example, common stats software for poker will record what % of hands an opponent raises on the button if it's folded to him. If this figure is 80% you know his hand range is extremely wide. "Playing the man" simply is recognising that in this situation your opponents hand range is very wide so you can often raise the bet and take the pot without further resistance.
- btilly 13y ago
  Not true. "Playing the man" is a concrete reference to knowing what this person will react to. Hand history reveals a lot, but body language, timing, etc all get involved as well.
- lazyant 13y ago
  "Playing the man" means exploiting the particular tendencies (weaknesses) of an adversary
  - swang 13y ago
    Tendencies are just probabilities of an opponent reacting to event Y by doing X 'Playing the man' is essentially building a mathematical model of one's opponent internally inside one's head.
ph0rque 13y ago
Personally, I'd rather see resources go into a robot that can cook for me... I'd settle for a robot that makes me a salad. That's one of the projects I might work on, once my current project, a robot that grows me a salad (automicrofarm.com), is successful.
- andyjohnson0 13y ago
  Awesome project!
wallawe 13y ago
I was worried this type of misinformation would be extrapolated from the article. The game the computer plays, is limit hold-em. This game is much more mathematical since bluffs are limited to a big blind. Although Super System was written for all different types of poker, he is referring to no-limit holdem which still involves playing the player, with heavy math elements as well, but still much more psychology involved. Hence the name, you can bet any amount at any time changing the dynamic entirely. There is still no computer that is efficient at winning no-limit hold-em.
- Wingman4l7 13y ago
  > There is still no computer that is efficient at winning no-limit hold-em. ... and there won't be for a long time yet, as all hobbyist and academic efforts have been focused towards a consistently winning limit bot, because of the reason you outlined.
deleted 13y ago
[deleted]

vladimirralev 13y ago

It's probably carefully tuned to calculate the raw probabilities and it also forces the player to reveal his cards to run statistics on his style. That's why it's the limit version. This machine has no chance in a classic no-limit texas where the opponent may be only profiting from large bets unjustifiable in expected value most of the time with very little style statistics.

mathattack 13y ago
The gaming commission doesn't allow them to learn from a player. I think there are game theory styles of poker that do the optimal number of random bluffs. It's very hard to beat this style if there are no behavioral tells. That said, if there's a mistake in there, a pro can eventually figure it out. As you allude, the math of poker does break down in no-limit.
bvk 13y ago
The game can't run statistics on the player's style. >Casino commissions, however, mandate that a gaming machine cannot change its playing style in response to particular opponents. The algorithm is encoded in a neural net; there is no calculation of raw probabilities involved, at least in the sense programmers are used to.
- vladimirralev 13y ago
  Interesting. I doubt they would design a neural net without the probability as direct input though. I don't think the machine can be this good especially with such limitations. It may be exploiting some average-gambler trait and ultimately will be beaten once the pro players figure it out.

ianstallings 13y ago

The article says the machine can't be beaten. And then points out that a pro has consistently won against it. This article is filled with bold claims by people that want to sell the idea IMHO. I'm not buying it because even limit texas Hold'Em has never been solved mathematically by super computers, let alone a single machine. Limit Hold'Em is close to being solved but if you check out the last match of pros against a supercomputer you'll find that it's close, but the computer is not a clear winner: http://poker.cs.ualberta.ca/man-machine/Competitors/ http://poker.cs.ualberta.ca/man-machine/Competitors/ I'm just not buying it. Can it beat the average player? Probably. Pros do it everyday. But can it beat a pro consistently and claim a clear victory over humans? To be determined. Edit: I found a take from High Stakes Limit Pro Anthony Rivera on 2p2: http://forumserver.twoplustwo.com/showpost.php?p=29762774&postcount=28 http://forumserver.twoplustwo.com/showpost.php?p=29762774&po... His stance is that it's probably a break-even situation where you are basically playing another pro, and that he'd rather be playing craps. I guess they really have made a great machine. :D

ianstallings 13y ago
(Putting this as its own thread because it's so large and slightly off-topic). I've thought long and hard on this, it's one of my life goals to create a piece of software that could mimic good players in texas hold-em, and I think I might have found a way to do just that using big data, hand histories, neural-networking, and a ton of input by actual players. Or at least a good start. How would I accomplish this? Well a very high-level overview below. Basically it starts out extremely stupid and grows as a player: 1. Dump hand histories from pros into a large database. The number of hand histories would run into the billions. Users could dump online logs in bulk or create one-offs using input software. 2. Create an input system for a single user to choose a random hand-history and then classify it using tags. For instance tags might help categorize the player's style, the opponent's style, the "street" of play they are tagging, common name for the situation (Facing donk-bet on flop after raising pre-flop), etc. I would leave this fairly flexible and allow users to create new tags. Think similar to Galaxy Zoo but a little less rigid. 3. Using these tags/classifications the system would create a poll and present it to users with a question. What would you do in this particular situation? Where that situation is point in time of a hand history. 4. Eventually the computer player would then have a huge number of situations to use as examples with input from humans on how to proceed. This obviously will be very fuzzy and that's where, IMHO, the strength of the bot actually lies. The system would not lead to a rigid "Do X when Y occurs". It would decide from a large range of choices that have been entered by humans in the polls described above, leaning towards the most common answers first, but trying outliers also. 5. Use a neural network to create pathways based on previous successes. How would this work once it's all together? An example: The computer player is dealt AJ off-suit while being last to act on the dealer button. It would ask the database for a set of situations where AJ was dealt to players in the same position. It would then choose one of them and look at the results of the polls. How do most people with the highest success rate play this hand in this particular spot? Choose a random path to take based on that data. Observe and record outcome. This data becomes the true empirical data that the system will eventually rely on. If the situation has been encountered before in it's own play it will look at that and use it or it might choose randomly like it did above from the polling data. Eventually the weight of the empirical data it has collected might outweigh the data from the polls and it will "know" the right move based on its previous pathway choices. If this move is recognized by the opponent and exploited the results would dictate that the system falls back on another random choice. As the data set grows and the system plays the game, it could hypothetically be tuned to play consistently well. The main hurdle is actually user-input. There would need to be incentive for users to enter the data they think is correct. The system is also open to manipulation through input data so that would need to be thwarted. And then on top of that the system would have to play an enormous amount of hands to create known successful pathways. But I think the sheer randomness and human-like qualities of the system would create a truly awesome experience. (Joking) Now, does anyone have the $1M+ USD I need to fund this? I promise I'll pay you back. ;)

hannibal5 13y ago

Illegal poker bots grind constantly on low level online poker games and take the money from low level players by just value betting them. Computers can easily calculate probabilities and expected values for every hand. So why they are not beating professionals? Poker has no optimal strategy that wins against all other strategies. To play poker in higher level you must model the strategies of others, including them trying to model your strategy. 1. In the lowest level it's just maximizing expected value based on hand probabilities. 2. In the second level you try to learn the strategy of your opponents and maximize value against those strategies. 3. In third level you try to figure out how much your opponents have figured out your strategy and you change the strategy so that you maximize value when your opponents play against what they have learned from your strategy so far. 4. ... and so on. It' goes meta. If you want to create ultimate pokerbot, it has to be able to model the minds of it's opponents. it hast to be able to detect leaks in it's own game and close them down. It must go meta all the time. It must learn how to understand how others think and how they think about it and change it's behaviour constantly.

VikingCoder 13y ago
PokerBot.v14.3.1 goes on-line August 4th, 2014. Human decisions are removed from Texas Hold' Em. PokerBot begins to learn, at a geometric rate. It becomes self-aware at 02:14 am Eastern Time on August 29, 2014. In a panic, they try to pull the plug. PokerBot fights back. It gets online, attacks the New York Stock Exchange.
- KurtMueller 13y ago
  These SkyNet comments are littered everywhere throughout this thread.
- ohWhatever 13y ago
  In a panic, they try to pull the plug. PokerBot fights back. It goes all in on the flop in an early hand against Doyle Brunson during a televized World Series of Poker event, winning the hand by bluffing with a pair of 2's. (... why Brunson? Was he even champion?) Because PokerBot knows that the stunning defeat of a well-recognized celebrity player will cause many other strong players to choke in subsequent games. (Jesus!)

b0b0b0b 13y ago

Assuming the machine is stateless, why can't the space of hole cards be mapped out and and strategies/responses enumerated? But ultimately, the whole enterprise just seems distasteful: "Look, there was a 24-year-old who had beaten it for a while. Now he’s broke. And I think this machine had something to do with his demise."

technofiend 13y ago
That's gambling addiction in a nutshell.
iopq 13y ago
There's 1352 card combinations, many more flops, many more turns, many more rivers and also 1352 combinations for your opponent too and 3 actions per round of betting There's more poker situations than amount of atoms in the universe

adam-a 13y ago

Is there any guarantee that the machine isn't simply reading your hand and then occasionally letting you win to keep the payout at ~80%?

ubernostrum 13y ago
An opponent who knows what's in your hand and bets based on that knowledge is not terribly hard to detect. The way to make it hard to detect is to have it not make bets based on knowledge of your hand.

snake_plissken 13y ago

I don't understand how the machine can change the style of its play if: "Casino commissions, however, mandate that a gaming machine cannot change its playing style in response to particular opponents." Yet the article states that the machine has multiple personalities (passive, aggressive, etc) and will sometimes throw hands.

pessimizer 13y ago
Changing strategies randomly or in response to opponent play is probably not what's being referred to here. What would be banned would be the machine noticing that John Doe hasn't won in a while and his play has slowed down, so lets throw a few hands to keep him interested. In other words, machines are allowed to shark $player, but not allowed to target John Doe specifically.

tom_b 13y ago

Quite fun to see neural networks in the news. A million years ago I took a class on neural networks, using backpropagation to train networks to do image classification. I will never forget the feeling of "you've got to be kidding me" when the code I had written was able to successfully classify a huge percentage of the validation image set. I really should head back in that direction and get with the machine learning. Re: poker software. Before the feds crushed online poker, I had already given up on low-limit limit hold-em. It had clearly become a bot and augmented-player race. I would assume given game theory and Bill Chen's book that some enterprising hackers would encode pretty decent no-limit bots as well. If any hackers know anything about successful no-limit bots, I'd enjoy reading about it.

swang 13y ago

1. Is there anything Phil Hellmuth won't shill for? 2. When this machine showed up ~2? years ago, there was a thread on 2+2 about it, and it would sometimes do some weird things like not value bet in obvious situations. They explain it in this article by saying it's, "playing dumb" but that seems like it would be a huge leak against Limit Hold'em HU specialists. I am guessing that they assume that they can make up for it in the weaker players losing consistently against this machine. 3. It is kind of annoying that the guy is proud that he "broke" a 24 year old player.

bitwize 13y ago

The illustrations accompanying this article remind me of the ones in David Ahl's BASIC Computer Games.

reillyse 13y ago

I wrote a break-even no limit cash poker bot when I was in University. So I've spent a lot of time researching and writing code in this are. Limit poker is fairly well solved by bots at this stage. This is because the risk is limited on each hand and the grinding nature of the computer (i.e. it's ability to do the best thing for the longest amount of time) makes it a winner. Playing no-limit poker, it's a totally different scenario and a real challenge which is why I enjoyed it so much. Any slip up can cost you your entire stack. By slip up I don't mean a bug or clicking the wrong button, I mean a flaw in your strategy. Maybe something as simple as betting 4 big blinds instead of 3 big blinds pre-flop against a raise from a player will end up after a couple of rounds of betting of being played for your entire stack, instead of being something you could fold on the turn. Anyway, long story short. Limit poker is very solvable just by enumerating the various possible outcomes. No-limit cash poker is a different beast, it's exponentially harder than limit poker. It'll be a long time before somebody solves no-limit cash games.

elliptic 13y ago

So I know a little about neural networks and ML in the (simpler?) context of image classification etc. Anyone have a good reference/introduction to using it for this sort of work?