9 ms·

Robot wars break out on poker sites

83 points by mootothemax 16y ago

uptown 16y ago
I'm surprised no poker site has launched a site that specifically geared towards algorithmic players. Seems like the kind of thing that might have a niche.
- die_sekte 16y ago
  It seems so. However, notice that the average poker bot programmer is a 20-something, highly intelligent hacker and that most of them don't have all that much money. Why do they write poker bots? (a) to make money. (b) to beat the system. (c) boredom. (a) doesn't work when all the other players are also good poker bots. Botting lives from the simple fact that most poker players are shit. (b) also doesn't apply, for obvious reasons. So, now you are left with the (c)s. Those will leave when they find something more interesting. Seemed to be a good idea. Well, it's probably not. (PS: There are poker bot competitions. The bigger ones have university teams competing, the smaller ones (the ones I found via Google) usually don't get enough entrants.)
  - uptown 16y ago
    I guess the stock market is the natural alternative for applying similar skills in a much bigger game.
- ig1 16y ago
  Betfair allows (i.e has a pubic API) algo players for some of their games, not sure if it covers their Poker site though.
  - daeken 16y ago
    Huh. I'm reading over the docs right now, and while I don't see anything about actually playing the game, they expose an interface for betting on hands. That's pretty interesting.
- noodle 16y ago
  i actually tried. i couldn't get enough interested parties and had a cofounder that wasn't very motivated. it would take your game history logs, analyze them, and let you know where your most common mistakes were based on context (i.e., ICM for SNGs) i still think its a good idea, for various reasons, i've just moved on.
- rograndom 16y ago
  "Good" players, including bot writers want, and need, very bad players in the games. The games need players who either don't know or don't care that they're making sub-optimal plays. That's what makes the money flow in poker. If everyone played perfectly, then only the house would win.
- chrisaycock 16y ago
  http://www.computerpokercompetition.org/ http://www.computerpokercompetition.org/ > At its heart, the Annual Computer Poker Competition aims to benefit artificial intelligence by promoting, aiding, and evaluating research in the challenging problems presented by the wide variety of poker games.
cryptoz 16y ago
> This score is based on a composite of a variety of factors including the length of their sessions, the standard deviation of those lengths, times played, and consistency of play schedule It seems that anyone able to write a poker bot is able to tell it to take breaks every now and then. This doesn't seem like a war the poker sites can win.
- rbanffy 16y ago
  > anyone able to write a poker bot is able to tell it to take breaks You can always go after those players who take breaks that are too random ;-)
  - uptown 16y ago
    The players still need to be mindful of how successful they are. It's easy to stand out when you're using illegal methods. Take a look at the plot from this article showing one poker cheater's win rate: http://simplecomplexity.net/catching-an-online-poker-cheater-with-data-mining/ http://simplecomplexity.net/catching-an-online-poker-cheater...
    - shasta 16y ago
      Your article is about actual cheating, not just automated "bot" play
      - uptown 16y ago
        I know. But presumably a bot is programmed to play consistently well ... otherwise there's no point in automating the process. I'm not saying a bot's performance would approach that of someone that's cheating, but if they can be consistently successful to the point where they can play a game unattended, I suspect their dot would fall towards (or beyond) the edges of most other players.
        shasta 16y ago
        Maybe if it was just playing heads up (1v1), but with technology where it is today, playing better than a group of good humans is apparently a hard problem.
mikek85 16y ago
If you can't beat a bot then poker is not the right game for you.
- bhousel 16y ago
  And what if several of them sit at your table and collude?
  - tomjen3 16y ago
    Would that actually be an advantage? I mean the bots are going to win more often that the other players, but on the other hand, they would also be at least twice as expensive to use.
    - daeken 16y ago
      If you have N seats, you have better odds for the best hand than you do with N-1 seats, yes, but what's even more important than that is that you know what cards the other players don't have. You have considerably more information, and this is absolutely essential. Collusion is the big problem with bots, not just smart logic.
    - chsonnu 16y ago
      I haven't played Hold 'em since fixed limit died (RIP party poker) but knowing even just two additional cards would be a tremendous advantage. You have an easy fold in small pots if you know you only have 5 outs vs 7 (overcard and gutshot draw). Hell you probably couldn't even be beat in the short run if you knew of 4 additional cards in addition to your hole cards.
    - icey 16y ago
      Collusion absolutely provides unfair advantages (which is why it's against the rules). Poker is a game of incomplete information. The more information you possess, the better your odds are of making the right decision. So, in the instance of 2 poker bots colluding on a table (the 4 suits are (c)lubs (d)iamonds (s)pades (h)earts): Bot 1 has Ac5h and discards pre-flop (no money spent) Bot 2 has KcQc and calls. Player 3 calls Player 4 calls All other players discard The flop comes up: 5c 10c 9c At this point, Bot 2 has an excellent hand (a King-high flush), but it doesn't have the best possible hand; it's vulnerable to a higher flush due to the Ace. However, the because bot 1 & 2 have colluded, it knows with 100% certainty that it has an unbeatable hand and can play accordingly. This is a pretty weak example, but it's the easiest I could think of without assuming any knowledge of poker. Having knowledge of cards that have been in play significantly increases a player's ability to make decisions in a way that give that player an unfair advantage.
bl4k 16y ago
This cat and mouse game is very interesting. I wrote a bot years ago that was easy to implement, since you could hook into the poker clients and listen for the window handler events. The more recent versions actually implement the console as graphics and the buttons as custom controls so that you can no longer do this. The modern bots use OCR to read the table play, so the sites responded again by changing it up. Having a single bot in a tournament is not bad per se, since they are easy to identify (they do not respond to player rage, they shove or call on hands that may be +EV isolated, but in tournament play are not, and they are the player who come final table sit all quiet and do not respond to deals) - the prob arises when bot networks join a single tournament and collude - especially on the double or nothing tournaments (where if you finish in the top half you get double your money, or nothing) Stars and FTP are very agressive with combating the problem. I receive refunds all the time because of previous games that have been investigated, so it is great that other players can rely on their work (and it is good to login and find that you have extra money because they are doing their job). These sites are making a lot of money on rake but they are definitely re-investing it to protect the players. They have hired some of the best security guys in the world to work for them (I know one of them, he should do an AMA because it would be super interesting).
- eclark 16y ago
  The profitable bots are a lot more than just shove bots. They take into account: Hand ranges, Nash Equilibrium (This is really cool stuff imo),EV of flop/turn/river, return from promotions, rake,reaction time of user, position, table image The last I saw the custom bots being run were buying up hand histories by the millions and using pre-flop agression and all the other factors to preform clustering. Then they clusters were used to choose a strategy. Stars and FTP are both pretty good about finding bots. But poker bot writers are learning from game bot makers. They are pretty well protected on the client end. So as long as the bots have some variance they are really hard to spot. And a lot of the variance comes from choosing how to play the person who is betting. Though it looks like some bot writers have not been putting enough variance into their bots. For a long time the big push was to find ways to make the input look like it came from a human. Moving the cursor to the exact middle of the button you want to push every time in <100ms is a dead give away. I don't really know if this was ever something that FTP and stars used though.
RickHull 16y ago
> The suit also accused Full Tilt pros Chris Ferguson and Andy Bloch of creating bots. ... Ferguson became a big tournament winner, with very little tournament experience, because of the edge he had. The Chris Ferguson? aka "Jesus"? I don't know which time period they are referring to, but he has been one of the most recognizable, well-known tournament players of the last 10 years. From http://en.wikipedia.org/wiki/Chris_Ferguson http://en.wikipedia.org/wiki/Chris_Ferguson > In 1994, he began playing in tournaments in California and in 1995, he entered his first World Series of Poker. Ferguson beat T. J. Cloutier at the main event of the 2000 WSOP to win the $1.5 million prize. So he became a winner of the WSOP after first entering 5 years previously. In what way is that "very little tournament experience"? I must be missing something...
- gojomo 16y ago
  WSOP is far from the only tournament; they're implying Ferguson was a tournament winner very soon after first starting to play in them. Do you believe otherwise?
  - deleted 16y ago
    [deleted]
gavingmiller 16y ago
There was a fantastic series of technical articles written a couple of years ago on building a poker bot: http://www.codingthewheel.com/archives/how-i-built-a-working-poker-bot http://www.codingthewheel.com/archives/how-i-built-a-working... The series of articles goes into detail on how to write a successful bot - Statistics, OCR, hooking dlls, etc. There is a ton of information that lays a solid ground work for doing this yourself (if you have the time.)
- patrickk 16y ago
  That link is superb; highly, highly recommended. Can't believe I didn't come across it before. You should submit it separately. I would except you should grab the karma (if you care about it) ;-)
deleted 16y ago
[deleted]
lmkg 16y ago
From the title, I was expecting this article to be about how the bots were evolving to play against each other. Sort of like HFT algorithms. Fortunately, it seems like they're still a small enough piece of the population that that's not a significant issue.
vinhboy 16y ago
argh.. every time I read these articles I wish my wife would let me play poker again.. =( .... I am so sad...
markbao 16y ago
If there's one thing I'm afraid of, it's the fact that robots are written to expect human input and respond to that input, but in practice, robots also unknowingly receive robot input, which results in weird, unexpected things happening. One example is high-frequency trading robots in the stock market, where one trade (or bug) could set off stuff like flash crashes (http://www.nytimes.com/2010/11/09/business/09flash.html http://www.nytimes.com/2010/11/09/business/09flash.html)
daeken 16y ago
I'm giving away a piece of data I planned to keep private and use for my own purposes, but here goes. Full Tilt Poker is now fully Qt. While they built their own slightly modified Qt binaries, it's not hard to get QtScript embedded in FTP. Once you have this, you can get the QObject corresponding to the window handle of a game table, and work backwards from the signals/slots there to find the model QObject. From here, you can enumerate all of the methods and events that the table model object exposes, giving you full access to all information that the client has, as well as allowing you to directly control the client. I built a framework that lets me use QtScript as a bridge from .NET to FTP, enabling me to trivially introspect and manipulate the game client. Never had time to actually hook bot logic up to it, sadly.
- eclark 16y ago
  Awesome stuff. Though I think it would be pretty hard to use. Last I saw the big sites were all monitoring dll loads and os hooks. So actually doing any actions would bring down their wrath.
  - daeken 16y ago
    It's amazingly easy to get around such things. Eventually I'll write up a blog post on how I've gone about it, and potential future ways of doing it. For the record, monitoring DLL loads is silly, because well, it only monitors DLL loads. If you load the DLL into memory yourself (essentially replicating the standard functionality), it's a non-issue, and that's not terribly hard.
    - nitrogen 16y ago
      load the DLL into memory yourself Does this work with data execution prevention?
      - daeken 16y ago
        Yes, DEP doesn't come into play if you properly set the protection on your code pages. The DLL specifies the protection for each section of the file, so that much is easy.
gyanguru 16y ago
These guys knew they had bot. PokerStars do a good job removing them. Full tilt knew they were there but kind of tolerated them (no idea why). The bot community says (people who made them) they had unwritten agreement of tolerance from FTilt. But now HR2267 passed which give high hopes of getting online poker regulated and all these sites would want a license so they are cleaning up the ship to get in good books. http://www.govtrack.us/congress/bill.xpd?bill=h111-2267 http://www.govtrack.us/congress/bill.xpd?bill=h111-2267 Also look at Lee Rousso case against Washington state which has a law making online poker play a felony. State Supreme court ruled the law as constitutional. Now PokerStars had moved out of Washington state for the same reason. If it was pre HR 2267 PokerStars wouldn't have done it.
VladRussian 16y ago
poker sites where bots are welcome http://www.nyse.com/ http://www.nyse.com/ and http://www.nasdaq.com/ http://www.nasdaq.com/
- jrockway 16y ago
  And you get paid whether you win or not.
- gigantor 16y ago
  not only are they welcome, they're the majority, participating in about 80% of the hands dealt
patrickk 16y ago
Betfair actually provide an API, so that you could easily write a bot for it: http://bdp.betfair.com/index.php?option=com_content&task=blogcategory&id=0&Itemid=78 http://bdp.betfair.com/index.php?option=com_content&task... They provide the API for the following: ‘Exchange Baccarat’, ‘Exchange Blackjack’ ‘Exchange Hilo’ and ‘Exchange Poker’. From reading the site, 'Exchange' seems to mean the games are 25% faster than standard. Haven't developed anything using their API, but it looks interesting. Also the polar opposite approach of PokerStars and FullTilt.
flipper 16y ago
Do any of the sites use captcha? That wouldn't stop a human nursing a PC running bots, but it would stop someone starting a bunch of bots, then going to sleep for 8 hours.

83 points by mootothemax 16y ago

I'm surprised no poker site has launched a site that specifically geared towards algorithmic players. Seems like the kind of thing that might have a niche.

die_sekte 16y ago
It seems so. However, notice that the average poker bot programmer is a 20-something, highly intelligent hacker and that most of them don't have all that much money. Why do they write poker bots? (a) to make money. (b) to beat the system. (c) boredom. (a) doesn't work when all the other players are also good poker bots. Botting lives from the simple fact that most poker players are shit. (b) also doesn't apply, for obvious reasons. So, now you are left with the (c)s. Those will leave when they find something more interesting. Seemed to be a good idea. Well, it's probably not. (PS: There are poker bot competitions. The bigger ones have university teams competing, the smaller ones (the ones I found via Google) usually don't get enough entrants.)
- uptown 16y ago
  I guess the stock market is the natural alternative for applying similar skills in a much bigger game.
ig1 16y ago
Betfair allows (i.e has a pubic API) algo players for some of their games, not sure if it covers their Poker site though.
- daeken 16y ago
  Huh. I'm reading over the docs right now, and while I don't see anything about actually playing the game, they expose an interface for betting on hands. That's pretty interesting.
noodle 16y ago
i actually tried. i couldn't get enough interested parties and had a cofounder that wasn't very motivated. it would take your game history logs, analyze them, and let you know where your most common mistakes were based on context (i.e., ICM for SNGs) i still think its a good idea, for various reasons, i've just moved on.
rograndom 16y ago
"Good" players, including bot writers want, and need, very bad players in the games. The games need players who either don't know or don't care that they're making sub-optimal plays. That's what makes the money flow in poker. If everyone played perfectly, then only the house would win.
chrisaycock 16y ago
http://www.computerpokercompetition.org/ http://www.computerpokercompetition.org/ > At its heart, the Annual Computer Poker Competition aims to benefit artificial intelligence by promoting, aiding, and evaluating research in the challenging problems presented by the wide variety of poker games.

cryptoz 16y ago

> This score is based on a composite of a variety of factors including the length of their sessions, the standard deviation of those lengths, times played, and consistency of play schedule It seems that anyone able to write a poker bot is able to tell it to take breaks every now and then. This doesn't seem like a war the poker sites can win.

rbanffy 16y ago
> anyone able to write a poker bot is able to tell it to take breaks You can always go after those players who take breaks that are too random ;-)
- uptown 16y ago
  The players still need to be mindful of how successful they are. It's easy to stand out when you're using illegal methods. Take a look at the plot from this article showing one poker cheater's win rate: http://simplecomplexity.net/catching-an-online-poker-cheater-with-data-mining/ http://simplecomplexity.net/catching-an-online-poker-cheater...
  - shasta 16y ago
    Your article is about actual cheating, not just automated "bot" play
    - uptown 16y ago
      I know. But presumably a bot is programmed to play consistently well ... otherwise there's no point in automating the process. I'm not saying a bot's performance would approach that of someone that's cheating, but if they can be consistently successful to the point where they can play a game unattended, I suspect their dot would fall towards (or beyond) the edges of most other players.
      - shasta 16y ago
        Maybe if it was just playing heads up (1v1), but with technology where it is today, playing better than a group of good humans is apparently a hard problem.

mikek85 16y ago

If you can't beat a bot then poker is not the right game for you.

bhousel 16y ago
And what if several of them sit at your table and collude?
- tomjen3 16y ago
  Would that actually be an advantage? I mean the bots are going to win more often that the other players, but on the other hand, they would also be at least twice as expensive to use.
  - daeken 16y ago
    If you have N seats, you have better odds for the best hand than you do with N-1 seats, yes, but what's even more important than that is that you know what cards the other players don't have. You have considerably more information, and this is absolutely essential. Collusion is the big problem with bots, not just smart logic.
  - chsonnu 16y ago
    I haven't played Hold 'em since fixed limit died (RIP party poker) but knowing even just two additional cards would be a tremendous advantage. You have an easy fold in small pots if you know you only have 5 outs vs 7 (overcard and gutshot draw). Hell you probably couldn't even be beat in the short run if you knew of 4 additional cards in addition to your hole cards.
  - icey 16y ago
    Collusion absolutely provides unfair advantages (which is why it's against the rules). Poker is a game of incomplete information. The more information you possess, the better your odds are of making the right decision. So, in the instance of 2 poker bots colluding on a table (the 4 suits are (c)lubs (d)iamonds (s)pades (h)earts): Bot 1 has Ac5h and discards pre-flop (no money spent) Bot 2 has KcQc and calls. Player 3 calls Player 4 calls All other players discard The flop comes up: 5c 10c 9c At this point, Bot 2 has an excellent hand (a King-high flush), but it doesn't have the best possible hand; it's vulnerable to a higher flush due to the Ace. However, the because bot 1 & 2 have colluded, it knows with 100% certainty that it has an unbeatable hand and can play accordingly. This is a pretty weak example, but it's the easiest I could think of without assuming any knowledge of poker. Having knowledge of cards that have been in play significantly increases a player's ability to make decisions in a way that give that player an unfair advantage.

bl4k 16y ago

This cat and mouse game is very interesting. I wrote a bot years ago that was easy to implement, since you could hook into the poker clients and listen for the window handler events. The more recent versions actually implement the console as graphics and the buttons as custom controls so that you can no longer do this. The modern bots use OCR to read the table play, so the sites responded again by changing it up. Having a single bot in a tournament is not bad per se, since they are easy to identify (they do not respond to player rage, they shove or call on hands that may be +EV isolated, but in tournament play are not, and they are the player who come final table sit all quiet and do not respond to deals) - the prob arises when bot networks join a single tournament and collude - especially on the double or nothing tournaments (where if you finish in the top half you get double your money, or nothing) Stars and FTP are very agressive with combating the problem. I receive refunds all the time because of previous games that have been investigated, so it is great that other players can rely on their work (and it is good to login and find that you have extra money because they are doing their job). These sites are making a lot of money on rake but they are definitely re-investing it to protect the players. They have hired some of the best security guys in the world to work for them (I know one of them, he should do an AMA because it would be super interesting).

eclark 16y ago
The profitable bots are a lot more than just shove bots. They take into account: Hand ranges, Nash Equilibrium (This is really cool stuff imo),EV of flop/turn/river, return from promotions, rake,reaction time of user, position, table image The last I saw the custom bots being run were buying up hand histories by the millions and using pre-flop agression and all the other factors to preform clustering. Then they clusters were used to choose a strategy. Stars and FTP are both pretty good about finding bots. But poker bot writers are learning from game bot makers. They are pretty well protected on the client end. So as long as the bots have some variance they are really hard to spot. And a lot of the variance comes from choosing how to play the person who is betting. Though it looks like some bot writers have not been putting enough variance into their bots. For a long time the big push was to find ways to make the input look like it came from a human. Moving the cursor to the exact middle of the button you want to push every time in <100ms is a dead give away. I don't really know if this was ever something that FTP and stars used though.

RickHull 16y ago

> The suit also accused Full Tilt pros Chris Ferguson and Andy Bloch of creating bots. ... Ferguson became a big tournament winner, with very little tournament experience, because of the edge he had. The Chris Ferguson? aka "Jesus"? I don't know which time period they are referring to, but he has been one of the most recognizable, well-known tournament players of the last 10 years. From http://en.wikipedia.org/wiki/Chris_Ferguson http://en.wikipedia.org/wiki/Chris_Ferguson > In 1994, he began playing in tournaments in California and in 1995, he entered his first World Series of Poker. Ferguson beat T. J. Cloutier at the main event of the 2000 WSOP to win the $1.5 million prize. So he became a winner of the WSOP after first entering 5 years previously. In what way is that "very little tournament experience"? I must be missing something...

gojomo 16y ago
WSOP is far from the only tournament; they're implying Ferguson was a tournament winner very soon after first starting to play in them. Do you believe otherwise?
- deleted 16y ago
  [deleted]

gavingmiller 16y ago

There was a fantastic series of technical articles written a couple of years ago on building a poker bot: http://www.codingthewheel.com/archives/how-i-built-a-working-poker-bot http://www.codingthewheel.com/archives/how-i-built-a-working... The series of articles goes into detail on how to write a successful bot - Statistics, OCR, hooking dlls, etc. There is a ton of information that lays a solid ground work for doing this yourself (if you have the time.)

patrickk 16y ago
That link is superb; highly, highly recommended. Can't believe I didn't come across it before. You should submit it separately. I would except you should grab the karma (if you care about it) ;-)

deleted 16y ago

[deleted]

lmkg 16y ago

From the title, I was expecting this article to be about how the bots were evolving to play against each other. Sort of like HFT algorithms. Fortunately, it seems like they're still a small enough piece of the population that that's not a significant issue.

vinhboy 16y ago

argh.. every time I read these articles I wish my wife would let me play poker again.. =( .... I am so sad...

markbao 16y ago

If there's one thing I'm afraid of, it's the fact that robots are written to expect human input and respond to that input, but in practice, robots also unknowingly receive robot input, which results in weird, unexpected things happening. One example is high-frequency trading robots in the stock market, where one trade (or bug) could set off stuff like flash crashes (http://www.nytimes.com/2010/11/09/business/09flash.html http://www.nytimes.com/2010/11/09/business/09flash.html)

daeken 16y ago

I'm giving away a piece of data I planned to keep private and use for my own purposes, but here goes. Full Tilt Poker is now fully Qt. While they built their own slightly modified Qt binaries, it's not hard to get QtScript embedded in FTP. Once you have this, you can get the QObject corresponding to the window handle of a game table, and work backwards from the signals/slots there to find the model QObject. From here, you can enumerate all of the methods and events that the table model object exposes, giving you full access to all information that the client has, as well as allowing you to directly control the client. I built a framework that lets me use QtScript as a bridge from .NET to FTP, enabling me to trivially introspect and manipulate the game client. Never had time to actually hook bot logic up to it, sadly.

eclark 16y ago
Awesome stuff. Though I think it would be pretty hard to use. Last I saw the big sites were all monitoring dll loads and os hooks. So actually doing any actions would bring down their wrath.
- daeken 16y ago
  It's amazingly easy to get around such things. Eventually I'll write up a blog post on how I've gone about it, and potential future ways of doing it. For the record, monitoring DLL loads is silly, because well, it only monitors DLL loads. If you load the DLL into memory yourself (essentially replicating the standard functionality), it's a non-issue, and that's not terribly hard.
  - nitrogen 16y ago
    load the DLL into memory yourself Does this work with data execution prevention?
    - daeken 16y ago
      Yes, DEP doesn't come into play if you properly set the protection on your code pages. The DLL specifies the protection for each section of the file, so that much is easy.

gyanguru 16y ago

These guys knew they had bot. PokerStars do a good job removing them. Full tilt knew they were there but kind of tolerated them (no idea why). The bot community says (people who made them) they had unwritten agreement of tolerance from FTilt. But now HR2267 passed which give high hopes of getting online poker regulated and all these sites would want a license so they are cleaning up the ship to get in good books. http://www.govtrack.us/congress/bill.xpd?bill=h111-2267 http://www.govtrack.us/congress/bill.xpd?bill=h111-2267 Also look at Lee Rousso case against Washington state which has a law making online poker play a felony. State Supreme court ruled the law as constitutional. Now PokerStars had moved out of Washington state for the same reason. If it was pre HR 2267 PokerStars wouldn't have done it.

VladRussian 16y ago

poker sites where bots are welcome http://www.nyse.com/ http://www.nyse.com/ and http://www.nasdaq.com/ http://www.nasdaq.com/

jrockway 16y ago
And you get paid whether you win or not.
gigantor 16y ago
not only are they welcome, they're the majority, participating in about 80% of the hands dealt

patrickk 16y ago

Betfair actually provide an API, so that you could easily write a bot for it: http://bdp.betfair.com/index.php?option=com_content&task=blogcategory&id=0&Itemid=78 http://bdp.betfair.com/index.php?option=com_content&task... They provide the API for the following: ‘Exchange Baccarat’, ‘Exchange Blackjack’ ‘Exchange Hilo’ and ‘Exchange Poker’. From reading the site, 'Exchange' seems to mean the games are 25% faster than standard. Haven't developed anything using their API, but it looks interesting. Also the polar opposite approach of PokerStars and FullTilt.

flipper 16y ago

Do any of the sites use captcha? That wouldn't stop a human nursing a PC running bots, but it would stop someone starting a bunch of bots, then going to sleep for 8 hours.