16 ms·

How A.I. Conquered Poker

248 points by jdkee 5y ago

hashberry 5y ago
> he opened a computer program called PioSOLVER, one of a handful of artificial-intelligence-based tools So I checked out this tool, and the team describes themselves as "programmers interested in algorithms"[0] ... what is the difference between A.I. and algorithms? [0] https://www.piosolver.com/pages/about-us https://www.piosolver.com/pages/about-us
- bluecalm 5y ago
  It just sounded like a good description. We never got any funding and started the business as a side project so no need for big words :)
- stu2b50 5y ago
  Depends on whether you use the traditional or vernacular definition of AI. Traditionally it was more about the how not the what. AI was an agent acting rationally in an environment. Pac-Man guided by Dijkstra? That's AI, there's an agent, an environment, and said agent is acting "rationally". In modern vernacular it has become a synonym for machine learning or sometimes specifically neural networks.
- buscoquadnary 5y ago
  The amount of VC funding you'll get.
- HALtheWise 5y ago
  The taxonomy I personally use is: * Every computer program has algorithms, it's an extremely general term for "the idea behind how the computer will solve the problem". Advanced algorithms are typically those that took a lot of human effort to come up with. * Machine Learning refers to a specific class of algorithms where the computer automatically figures out (part of) what it should do based on data. * Deep learning is the subset of ML that uses deep neural networks. * Artificial Intelligence is a marketing term, and is actually about the _problem_ being solved, not the technology being used to solve it. In particular, AI is any computer program that solves a problem people would previously expect can only be solved by a human applying creativity and/or intelligence, like playing a game, understanding natural language, or creating artwork. This definition is obviously a moving target as expectations change. Deep learning is currently the most powerful and general toolkit for solving AI problems, so the concepts tend to get mixed together pretty frequently, but I personally like using the definitions above to keep things straight.
  - hammock 5y ago
    Clearest and most useful descriptions I've seen by far. Thanks for sharing
  - JacobThreeThree 5y ago
    I like your framework. Would you put statistical analysis "algorithms" in the same category as "Machine Learning"?
    - HALtheWise 5y ago
      Sure, "classical" ML of various forms (knn, decision trees, etc.) are Machine Learning, but aren't Deep Learning. That's exactly why the distinction between those phrases is useful.
  - hashberry 5y ago
    I like your taxonomy. But just to clarify: is the cruise control on my car "artificial intelligence?" Because a lot of dumb 15-year-old teenagers don't know they have to increase power while going up a hill, while my cruise control algorithm does.
    - HALtheWise 5y ago
      My category for AI is based on "popular perception", not necessarily an actual comparison to what humans can or can't do. Most humans don't know how to multiply complex numbers, but that doesn't mean doing so demonstrates human-level intelligence and decision making. In the specific case of cruise control, I don't think maintaining speed accurately is typically viewed as "difficult" for a computer today, so it would be deceptive to refer to as AI. 100 years ago that may have been different.
- ModernMech 5y ago
  Algorithms are provably correct. If you've got no proof of correctness, the best you have is a heuristic.
neonate 5y ago
https://archive.is/rNhub https://archive.is/rNhub
marcodiego 5y ago
Is there any remaining logic game which AI hasn't become superhuman? IIRC Monte Carlo tree search has done wonders in this area.
- pclmulqdq 5y ago
  Contract Bridge is still untouched. I'm not sure if it's because of lack of resources, but it has a lot of difficult elements: cooperation, huge search trees due to imperfect information, and information theoretic aspects of bidding.
- btheshoe 5y ago
  Imperfect information games that require agents to cooperate with each other haven't been solved yet. (multiagent cooperation is still a hard problem) Hanabi is a big(ish) research area. Contract bridge has had some work done on it.
  - HWR_14 5y ago
    I'm surprised that Hanabi is such a huge research area. Compared to bridge or poker, it seems like a far more simple game.
    - ska 5y ago
      > I'm surprised ... it seems like a far more simple game. This probably shouldn't surprise you. If you are researching how to attack a new/difficult class of problems, typically you look for the simplest versions of that class first.
      - HWR_14 5y ago
        I would agree with that. I just tended to assume that Hanabi would be a solved problem by now.
        noambrown 5y ago
bluecalm 5y ago
I am the solver programmer mentioned in the article. I think it came out pretty decently. Maybe not all the details are right (I was late answering fact checking email) but I think it's a decent read and the author has done a good job!
- im3w1l 5y ago
  How perfect is the play that piosolver calculates? Do you make any simplifying assumptions? I noticed in the article that piosolver can incorporate a mixed strategy with multiple bet sizes. Pretty neat, is that a recent feature? As I rememember it, a few years back people would discuss what the optimal bet size was in various situations with the implicit assumption that there is just one.
  - bluecalm 5y ago
    Adding multiple betsizes was one of the first features we added in 2015 - so not very recent, no. The one true correct bet size is a seducing idea but that just doesn't exist for most situations. You can try finding the best one with the assumption you can only use one but it's not the best idea in my opinion. Main reasons are that that's the sizing most people will be most ready to play against and EV difference between possible bet sizes is minimal. What happened in chess is now happening in poker: at first people were all about the best theoretical move and then gradual shift to "not necessary the best just sound enough but less likely to be analyzed by the opponents" started happening. As to the first part of your question: the solutions are pretty much perfect. The only assumption is about possible bet sizes. As to accuracy we measure it in theoretical exploitability (how much a perfect adversary who knows our strategy exactly could win against us). You can easily go so low that even a theoretical perfect adversary wouldn't get close to beating the rake vs the solution, even at high stakes.
    - deleted 5y ago
      [deleted]
- ewhauser421 5y ago
  Congrats - PIOSolver is an amazing program. What are the future plans for it? It would be great if you didn't need a PhD to set it up though. I keep wondering if I'm actually training myself using the right solutions or not based on the setup :)
torginus 5y ago
Does this mean that online poker is effectively over?
- paxys 5y ago
  Chess has been "solved" by computers for a long time now, yet the online Chess community is bigger than ever. I don't see why it would be any different for Poker.
  - fdgsdfogijq 5y ago
    Big difference with poker is that its all about money, the game doesnt work if people arent risking something. And if computers are superior, the game breaks down.
    - robrenaud 5y ago
      You are worried that people will cheat in online poker, or what? For poker, people can still play face to face, and it being a game of mixed strategies means it's probably a lot less important that poker bots are better than people than in chess. Top chess players use the computers to set up preparation bombs on their opponents, with rebuttals to moves for specific positions created by the computers. That doesn't work in a game of imperfect information and mixed strategies.
      - JacobThreeThree 5y ago
        >You are worried that people will cheat in online poker, or what? The norm for online chess is to not play for money, whereas the norm for online poker is to play for money, often for large sums. Of course the game with higher stakes is more likely to attract cheaters.
        paxys 5y ago
        > whereas the norm for online poker is to play for money, often for large sums That is definitely not the norm. Most people are playing recreationally. Heck it is illegal to pay poker online with real money in 44/50 states in the US.
smallerfish 5y ago
I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.
- Nextgrid 5y ago
  Isn't that just algorithmic trading?
  - meetups323 5y ago
    Classic games tend to be more fun than New York horoscopes. For me at least, there's something fundamentally profound about solving a logical puzzle better than anyone in history, that just isn't there in guessing which way some lines are going to go.
- bloodyplonker22 5y ago
  Online poker has already been evolving in this direction. Any serious player who plays online will use player assistance tools (such as PIOsolver). Top level poker AI such as Pluribus has already proven that it can hold its own and beat the best online players such as linuslove. It is well known that online poker sites are full of poker bots now. There are programs to help users try to tell if they are playing against a bot by seeing how fast it reacts on each hand, for example if it always takes a fixed amount of time on each hand or if there is a pattern.
  - smallerfish 5y ago
    Yeah, that was the evolution of my thought process. Online poker is already full of bots, which will get more and more sophisticated over time and defeat bot detection hurdles -- so why not embrace it and have bot-only zones? You could have entire tournaments play out in less than a minute if all participants were software.
    - bloodyplonker22 5y ago
jstx1 5y ago
But it hasn't conquered it! I kept searching the article for some new recent breakthrough that I've missed but it's not there. Yes, solvers like Pio have been around for years and limit holdem has been essentially solved for a while but nobody plays limit holdem anyway. The two most popular games (no-limit Texas holdem and pot-limit Omaha) are still unsolved.
- protoax 5y ago
  Hasn't Facebook's Pluribus come pretty close to "solving" no-limit holdem? I don't know a lot about poker so I can't really assess their claim's validity.
  - ironrabbit 5y ago
    Not even mentioning Pluribus is just bad journalism
    - kuboble 5y ago
      The article is mostly about a change in the industry and adaptation of solver strategies by human players of all levels. Pluribus had approximately zero impact on that.
- evrydayhustling 5y ago
  There is still a lively academic community and major progress! Check out CMU's no limit results [1]. (I realize articles like this have to pick some angles to make it interesting, but it was weird to see only dated research mentioned.) But if you are rooting against the machines, don't worry: it is almost certainly impossible to calculate a full equilibrium policy for no limit multiplayer, so we will instead be debating over the virtues of various types of imperfection for a long time. And even if an Oracle gave us convenient access to equilibrium strategy, it would still not be the optimum at a table full of imperfect players. Your poker game is safe for a while! [1] https://www.nature.com/articles/d41586-019-02156-9 https://www.nature.com/articles/d41586-019-02156-9
soferio 5y ago
What teaching or training tools are out there for a very average player at no limit Texas hold’em who just wants to get at bit better to a respectable level at a modest time commitment , and does not need to be a pro-level player?
- jstx1 5y ago
  Books (there's very few good ones) Training videos 1-on-1 coaching PioSolver (and other similar software) For any of these to stick, you need to spend some amount of time studying by yourself; just consuming learning material and playing isn't enough.
  - Reflecticon 5y ago
    Could you recommend a couple of books for a start?
    - jstx1 5y ago
      The Mathematics of Poker, by Chen & Ankenman - this one doesn't age because it's really fundamental if you want to understand things from the ground up. But it won't seem that practical at first. Applications of No-Limit Hold 'em, by Matthew Janda - this one goes into specifics about how to think on each street, how to build a good strategy etc. Some of the details will be different from the latest theory because the book was released in 2013 but it's still a very solid read if you want to level up.
    - MaxfordAndSons 5y ago
      Modern Poker Theory by Michael Acevedo is considered a contemporary gold standard.
    - jdm386 5y ago
      Modern Poker Theory as mentioned by the other reply is good. I'd also suggest Play Optimal Poker by Andrew Brokos. The former outlines a lot of strategy in the game and talks a little about game theory and its implications. The latter gives less direct poker advice but uses "toy games" that share some similarities with poker but are simplified to demonstrate some of the implications that game theory has on poker. It does contain some direct poker advice but it's more of a book to teach you how to think properly about how game theory applies to poker and how you can use that to study with something like a solver and actually understand what you're seeing. I would suggest you pick up both, I think that the Brokos book assumes you know certain poker terms and doesn't contain a glossary whereas modern poker theory is almost like a textbook and has a section where it defines all of the terms it uses. Modern poker theory will have more actionable advice but the Brokos book is excellent to teach you how to think about game theory and requires more self-reflection and has more questions to the reader. Applications of no limit by Janda or mathematics of poker are recommended by a different reply here. I would caution that these are extremely academic and extremely dense texts that would be a very tough read for a newer player. Mathematics is less practical and more of a math book than a poker book and applications is Janda showing how to work out solver-like solutions before solvers existed and also contains a lot of math. I think the other two above are more practical and aren't going to lead you to put the book down a 10th of the way through.
RivieraKid 5y ago
For some time I've been pondering about creating a multiplayer poker AI that would (1) play reasonably well but not at a superhuman level (2) not require expensive hardware. Frustratingly, this seems to be an unsolved problem. Pluribus is only 6-max, requires fixed starting stack sizes and requires a lot of hardware.
- MaxfordAndSons 5y ago
  I mean, it may not have been publicly solved, but there are definitely bots out there in the wild that are beating human fields on online poker sites. But also I think Pokersnowie is a bot made with more traditional ML methods (i.e. training on hand histories rather than solving an entire game tree with CFR), which basically fits the description.
  - RivieraKid 5y ago
    Is there enough public info to reimplement PokerSnowie? I only know it uses neural nets in some way.
tromp 5y ago
I like how the article gives examples of the sort of entropy players draw on to make their random choices: > Koon will often randomly select which of the solver’s tactics to employ in a given hand. He’ll glance down at the second hand on his watch, or at a poker chip to note the orientation of the casino logo as if it were a clock face, in order to generate a percentage between 1 and 100.
- splonk 5y ago
  The watch face is the classic one, the chip orientation is new to me. I like it, it's less obvious that you're randomizing something, and given how much poker players like to fidget with their chips the orientation is probably reasonably random. Another classic is to use the suit of your cards, but that has problems with being correlated with the state of the board. It works fine preflop.
splonk 5y ago
(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization techniques and emotional control rather than the TV image of staring in someone's eyes and reading their soul. Probably the one biggest misconception people have is that pros have sick reading abilities since TV likes to emphasize staredowns, when the actual single biggest skill long term pros have is the ability to lose hand after hand for hours and still play their best game. Incidentally, the anecdote at the top of the article is pretty intuitive game theoretically. Basically on the river you need to bluff with some portion of your hands, or else nobody will ever call when you have a good hand. The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. On a board of Kc4c5c2d2c, 7d6d is quite likely the absolute worst hand you can hold given the action that's taken place, therefore it's the one you bluff with. (For some pot/stack sizes it's possible that you bluff so rarely that you have to choose between 7d6d/7h6h/7s6s, but that's getting into details.)
- deleted 5y ago
  [deleted]
- DennisP 5y ago
  Interesting, I read some Sklansky books about 15 years ago and he talked about semibluffs, where you're partly bluffing but you've also got some chance to win. Sounds like strategy has evolved into doing the exact opposite.
  - GoatOfAplomb 5y ago
    The distinction is that semi-bluffs are for when there are still cards to be dealt.
    - 5y ago
vvarp 5y ago
Worth to note that PioSolver isn’t the only tool of its kind - there’s also Jesolver (http://jeskola.net/jesolver_beta/ http://jeskola.net/jesolver_beta/), which is speed optimized variant, and cloud-based Deepsolver (https://deepsolver.com https://deepsolver.com) which leverages neural nets to do the job
achenatx 5y ago
<<But for Doug Polk, who largely retired from poker in 2017 after winning tens of millions of dollars, the change solvers have wrought is more existential. “I feel like it kind of killed the soul of the game,” Polk says, changing poker “from who can be the most creative problem-solver to who can memorize the most stuff and apply it.”>> I wrote probably the first online texas holdem game, played on IRC, around 1994. Back then in rec.gambling blackjack teams were forming and instead of joining a blackjack team I started to play poker, because it was more fun than counting cards. I wrote the game so I could play against other people and was immediately a winning player in low limit games of the time in lake charles and vegas. However even in the 90s, it was about statistics. I do regret not allowing people to play for money (people were constantly asking). Even back then the primitive bots were mostly killing human players.
- mikemcd 5y ago
  I'm pretty sure it was Vanessa Selbst who said that, not Doug Polk
sinuhe69 5y ago
I may be ignorant but I still don’t understand the fuss about this feat. I mean poker is all about probability, right? With uncertainty about bluffing and fraught with emotional control problems. So naturally, a machine is best suited for this kind of problem, but not human. The bluffing problem can also be solved with heuristic optimization techniques like GA. Thus, is it not obvious that machines/algorithms will beat humans in the long run?
- mym1990 5y ago
  The distinction is that the number of branches that manifest in games of incomplete information can become utterly massive, and then you add in 6-10 players at once, all playing at world class levels, and you have a real computation problem. I mean maybe this is a trivial thing to solve for you, in which case kudos, but its pretty neat for a lot of onlookers(and participants).
randomNumber7 5y ago
There is some misconception about the optimal strategy in the article. You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. You have to play an exploitable strategy yourself to exploit someone else. Think about playing rock paper scissors agains an opponent who always chooses rock. If you still play the optimal strategy (choosing randomly) you gain nothing...
- misja111 5y ago
  > You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. This is not true. If you play optimal strategy, you will win against any opponent except one that plays optimal as well, in which case you'll break even. But, of course you'll win a lot more if you are able to adapt your strategy to exploit any weaknesses that you have detected. And detecting weaknesses in your opponent(s) in live play is something that humans will remain better at than AI for quite a while. Because it requires not just careful analysis of your opponent's actions but some contextual information as well. E.g., how old is your opponent, is he experienced or not, is he drunk, have you experienced similar players like hime before, etc.
  - 3pt14159 5y ago
    > If you play optimal strategy, you will win against any opponent except one that plays optimal as well How does the example you're responding to not win 50% of the time? Rock v Rock = Tie Rock v Paper = Loss Rock v Scissors = Win The optimal game theoretic play of randomly choosing rock paper scissors is inferior play against this particular opponent. All that game theoretic perfect play gets you is the benefit of getting at least a tie. Possibly more, but not always more for non-perfect play.
    - kuboble 5y ago
      Poker isn't like this. Think rock, pair, scissors, crap where rules are that crap always loses. Now uniform strategy between rps will yield profit vs an opponent who plays crap sometimes. It's very easy to play crap in poker
rep_movsd 5y ago
This whole thing is a misconception about poker. Poker is never about calculating odds in your head, but bluffing convincingly and reading the opponents tells. Computers have no tells, and I doubt if any ML can learn to recognize tells of people, since they are very subtle. The statistics part anyone with basic math skills can figure in their head - that doesnt guarantee any win.

248 points by jdkee 5y ago

> he opened a computer program called PioSOLVER, one of a handful of artificial-intelligence-based tools So I checked out this tool, and the team describes themselves as "programmers interested in algorithms"[0] ... what is the difference between A.I. and algorithms? [0] https://www.piosolver.com/pages/about-us https://www.piosolver.com/pages/about-us

bluecalm 5y ago
It just sounded like a good description. We never got any funding and started the business as a side project so no need for big words :)
stu2b50 5y ago
Depends on whether you use the traditional or vernacular definition of AI. Traditionally it was more about the how not the what. AI was an agent acting rationally in an environment. Pac-Man guided by Dijkstra? That's AI, there's an agent, an environment, and said agent is acting "rationally". In modern vernacular it has become a synonym for machine learning or sometimes specifically neural networks.
buscoquadnary 5y ago
The amount of VC funding you'll get.
HALtheWise 5y ago
The taxonomy I personally use is: * Every computer program has algorithms, it's an extremely general term for "the idea behind how the computer will solve the problem". Advanced algorithms are typically those that took a lot of human effort to come up with. * Machine Learning refers to a specific class of algorithms where the computer automatically figures out (part of) what it should do based on data. * Deep learning is the subset of ML that uses deep neural networks. * Artificial Intelligence is a marketing term, and is actually about the _problem_ being solved, not the technology being used to solve it. In particular, AI is any computer program that solves a problem people would previously expect can only be solved by a human applying creativity and/or intelligence, like playing a game, understanding natural language, or creating artwork. This definition is obviously a moving target as expectations change. Deep learning is currently the most powerful and general toolkit for solving AI problems, so the concepts tend to get mixed together pretty frequently, but I personally like using the definitions above to keep things straight.
- hammock 5y ago
  Clearest and most useful descriptions I've seen by far. Thanks for sharing
- JacobThreeThree 5y ago
  I like your framework. Would you put statistical analysis "algorithms" in the same category as "Machine Learning"?
  - HALtheWise 5y ago
    Sure, "classical" ML of various forms (knn, decision trees, etc.) are Machine Learning, but aren't Deep Learning. That's exactly why the distinction between those phrases is useful.
- hashberry 5y ago
  I like your taxonomy. But just to clarify: is the cruise control on my car "artificial intelligence?" Because a lot of dumb 15-year-old teenagers don't know they have to increase power while going up a hill, while my cruise control algorithm does.
  - HALtheWise 5y ago
    My category for AI is based on "popular perception", not necessarily an actual comparison to what humans can or can't do. Most humans don't know how to multiply complex numbers, but that doesn't mean doing so demonstrates human-level intelligence and decision making. In the specific case of cruise control, I don't think maintaining speed accurately is typically viewed as "difficult" for a computer today, so it would be deceptive to refer to as AI. 100 years ago that may have been different.
ModernMech 5y ago
Algorithms are provably correct. If you've got no proof of correctness, the best you have is a heuristic.

neonate 5y ago

https://archive.is/rNhub https://archive.is/rNhub

marcodiego 5y ago

Is there any remaining logic game which AI hasn't become superhuman? IIRC Monte Carlo tree search has done wonders in this area.

pclmulqdq 5y ago
Contract Bridge is still untouched. I'm not sure if it's because of lack of resources, but it has a lot of difficult elements: cooperation, huge search trees due to imperfect information, and information theoretic aspects of bidding.
btheshoe 5y ago
Imperfect information games that require agents to cooperate with each other haven't been solved yet. (multiagent cooperation is still a hard problem) Hanabi is a big(ish) research area. Contract bridge has had some work done on it.
- HWR_14 5y ago
  I'm surprised that Hanabi is such a huge research area. Compared to bridge or poker, it seems like a far more simple game.
  - ska 5y ago
    > I'm surprised ... it seems like a far more simple game. This probably shouldn't surprise you. If you are researching how to attack a new/difficult class of problems, typically you look for the simplest versions of that class first.
    - HWR_14 5y ago
      I would agree with that. I just tended to assume that Hanabi would be a solved problem by now.
      - noambrown 5y ago

bluecalm 5y ago

I am the solver programmer mentioned in the article. I think it came out pretty decently. Maybe not all the details are right (I was late answering fact checking email) but I think it's a decent read and the author has done a good job!

im3w1l 5y ago
How perfect is the play that piosolver calculates? Do you make any simplifying assumptions? I noticed in the article that piosolver can incorporate a mixed strategy with multiple bet sizes. Pretty neat, is that a recent feature? As I rememember it, a few years back people would discuss what the optimal bet size was in various situations with the implicit assumption that there is just one.
- bluecalm 5y ago
  Adding multiple betsizes was one of the first features we added in 2015 - so not very recent, no. The one true correct bet size is a seducing idea but that just doesn't exist for most situations. You can try finding the best one with the assumption you can only use one but it's not the best idea in my opinion. Main reasons are that that's the sizing most people will be most ready to play against and EV difference between possible bet sizes is minimal. What happened in chess is now happening in poker: at first people were all about the best theoretical move and then gradual shift to "not necessary the best just sound enough but less likely to be analyzed by the opponents" started happening. As to the first part of your question: the solutions are pretty much perfect. The only assumption is about possible bet sizes. As to accuracy we measure it in theoretical exploitability (how much a perfect adversary who knows our strategy exactly could win against us). You can easily go so low that even a theoretical perfect adversary wouldn't get close to beating the rake vs the solution, even at high stakes.
  - deleted 5y ago
    [deleted]
ewhauser421 5y ago
Congrats - PIOSolver is an amazing program. What are the future plans for it? It would be great if you didn't need a PhD to set it up though. I keep wondering if I'm actually training myself using the right solutions or not based on the setup :)

torginus 5y ago

Does this mean that online poker is effectively over?

paxys 5y ago
Chess has been "solved" by computers for a long time now, yet the online Chess community is bigger than ever. I don't see why it would be any different for Poker.
- fdgsdfogijq 5y ago
  Big difference with poker is that its all about money, the game doesnt work if people arent risking something. And if computers are superior, the game breaks down.
  - robrenaud 5y ago
    You are worried that people will cheat in online poker, or what? For poker, people can still play face to face, and it being a game of mixed strategies means it's probably a lot less important that poker bots are better than people than in chess. Top chess players use the computers to set up preparation bombs on their opponents, with rebuttals to moves for specific positions created by the computers. That doesn't work in a game of imperfect information and mixed strategies.
    - JacobThreeThree 5y ago
      >You are worried that people will cheat in online poker, or what? The norm for online chess is to not play for money, whereas the norm for online poker is to play for money, often for large sums. Of course the game with higher stakes is more likely to attract cheaters.
      - paxys 5y ago
        > whereas the norm for online poker is to play for money, often for large sums That is definitely not the norm. Most people are playing recreationally. Heck it is illegal to pay poker online with real money in 44/50 states in the US.

smallerfish 5y ago

I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.

Nextgrid 5y ago
Isn't that just algorithmic trading?
- meetups323 5y ago
  Classic games tend to be more fun than New York horoscopes. For me at least, there's something fundamentally profound about solving a logical puzzle better than anyone in history, that just isn't there in guessing which way some lines are going to go.
bloodyplonker22 5y ago
Online poker has already been evolving in this direction. Any serious player who plays online will use player assistance tools (such as PIOsolver). Top level poker AI such as Pluribus has already proven that it can hold its own and beat the best online players such as linuslove. It is well known that online poker sites are full of poker bots now. There are programs to help users try to tell if they are playing against a bot by seeing how fast it reacts on each hand, for example if it always takes a fixed amount of time on each hand or if there is a pattern.
- smallerfish 5y ago
  Yeah, that was the evolution of my thought process. Online poker is already full of bots, which will get more and more sophisticated over time and defeat bot detection hurdles -- so why not embrace it and have bot-only zones? You could have entire tournaments play out in less than a minute if all participants were software.
  - bloodyplonker22 5y ago

jstx1 5y ago

But it hasn't conquered it! I kept searching the article for some new recent breakthrough that I've missed but it's not there. Yes, solvers like Pio have been around for years and limit holdem has been essentially solved for a while but nobody plays limit holdem anyway. The two most popular games (no-limit Texas holdem and pot-limit Omaha) are still unsolved.

protoax 5y ago
Hasn't Facebook's Pluribus come pretty close to "solving" no-limit holdem? I don't know a lot about poker so I can't really assess their claim's validity.
- ironrabbit 5y ago
  Not even mentioning Pluribus is just bad journalism
  - kuboble 5y ago
    The article is mostly about a change in the industry and adaptation of solver strategies by human players of all levels. Pluribus had approximately zero impact on that.
evrydayhustling 5y ago
There is still a lively academic community and major progress! Check out CMU's no limit results [1]. (I realize articles like this have to pick some angles to make it interesting, but it was weird to see only dated research mentioned.) But if you are rooting against the machines, don't worry: it is almost certainly impossible to calculate a full equilibrium policy for no limit multiplayer, so we will instead be debating over the virtues of various types of imperfection for a long time. And even if an Oracle gave us convenient access to equilibrium strategy, it would still not be the optimum at a table full of imperfect players. Your poker game is safe for a while! [1] https://www.nature.com/articles/d41586-019-02156-9 https://www.nature.com/articles/d41586-019-02156-9

soferio 5y ago

What teaching or training tools are out there for a very average player at no limit Texas hold’em who just wants to get at bit better to a respectable level at a modest time commitment , and does not need to be a pro-level player?

jstx1 5y ago
Books (there's very few good ones) Training videos 1-on-1 coaching PioSolver (and other similar software) For any of these to stick, you need to spend some amount of time studying by yourself; just consuming learning material and playing isn't enough.
- Reflecticon 5y ago
  Could you recommend a couple of books for a start?
  - jstx1 5y ago
    The Mathematics of Poker, by Chen & Ankenman - this one doesn't age because it's really fundamental if you want to understand things from the ground up. But it won't seem that practical at first. Applications of No-Limit Hold 'em, by Matthew Janda - this one goes into specifics about how to think on each street, how to build a good strategy etc. Some of the details will be different from the latest theory because the book was released in 2013 but it's still a very solid read if you want to level up.
  - MaxfordAndSons 5y ago
    Modern Poker Theory by Michael Acevedo is considered a contemporary gold standard.
  - jdm386 5y ago
    Modern Poker Theory as mentioned by the other reply is good. I'd also suggest Play Optimal Poker by Andrew Brokos. The former outlines a lot of strategy in the game and talks a little about game theory and its implications. The latter gives less direct poker advice but uses "toy games" that share some similarities with poker but are simplified to demonstrate some of the implications that game theory has on poker. It does contain some direct poker advice but it's more of a book to teach you how to think properly about how game theory applies to poker and how you can use that to study with something like a solver and actually understand what you're seeing. I would suggest you pick up both, I think that the Brokos book assumes you know certain poker terms and doesn't contain a glossary whereas modern poker theory is almost like a textbook and has a section where it defines all of the terms it uses. Modern poker theory will have more actionable advice but the Brokos book is excellent to teach you how to think about game theory and requires more self-reflection and has more questions to the reader. Applications of no limit by Janda or mathematics of poker are recommended by a different reply here. I would caution that these are extremely academic and extremely dense texts that would be a very tough read for a newer player. Mathematics is less practical and more of a math book than a poker book and applications is Janda showing how to work out solver-like solutions before solvers existed and also contains a lot of math. I think the other two above are more practical and aren't going to lead you to put the book down a 10th of the way through.

RivieraKid 5y ago

For some time I've been pondering about creating a multiplayer poker AI that would (1) play reasonably well but not at a superhuman level (2) not require expensive hardware. Frustratingly, this seems to be an unsolved problem. Pluribus is only 6-max, requires fixed starting stack sizes and requires a lot of hardware.

MaxfordAndSons 5y ago
I mean, it may not have been publicly solved, but there are definitely bots out there in the wild that are beating human fields on online poker sites. But also I think Pokersnowie is a bot made with more traditional ML methods (i.e. training on hand histories rather than solving an entire game tree with CFR), which basically fits the description.
- RivieraKid 5y ago
  Is there enough public info to reimplement PokerSnowie? I only know it uses neural nets in some way.

tromp 5y ago

I like how the article gives examples of the sort of entropy players draw on to make their random choices: > Koon will often randomly select which of the solver’s tactics to employ in a given hand. He’ll glance down at the second hand on his watch, or at a poker chip to note the orientation of the casino logo as if it were a clock face, in order to generate a percentage between 1 and 100.

splonk 5y ago
The watch face is the classic one, the chip orientation is new to me. I like it, it's less obvious that you're randomizing something, and given how much poker players like to fidget with their chips the orientation is probably reasonably random. Another classic is to use the suit of your cards, but that has problems with being correlated with the state of the board. It works fine preflop.

splonk 5y ago

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization techniques and emotional control rather than the TV image of staring in someone's eyes and reading their soul. Probably the one biggest misconception people have is that pros have sick reading abilities since TV likes to emphasize staredowns, when the actual single biggest skill long term pros have is the ability to lose hand after hand for hours and still play their best game. Incidentally, the anecdote at the top of the article is pretty intuitive game theoretically. Basically on the river you need to bluff with some portion of your hands, or else nobody will ever call when you have a good hand. The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. On a board of Kc4c5c2d2c, 7d6d is quite likely the absolute worst hand you can hold given the action that's taken place, therefore it's the one you bluff with. (For some pot/stack sizes it's possible that you bluff so rarely that you have to choose between 7d6d/7h6h/7s6s, but that's getting into details.)

deleted 5y ago
[deleted]
DennisP 5y ago
Interesting, I read some Sklansky books about 15 years ago and he talked about semibluffs, where you're partly bluffing but you've also got some chance to win. Sounds like strategy has evolved into doing the exact opposite.
- GoatOfAplomb 5y ago
  The distinction is that semi-bluffs are for when there are still cards to be dealt.
  - 5y ago

vvarp 5y ago

Worth to note that PioSolver isn’t the only tool of its kind - there’s also Jesolver (http://jeskola.net/jesolver_beta/ http://jeskola.net/jesolver_beta/), which is speed optimized variant, and cloud-based Deepsolver (https://deepsolver.com https://deepsolver.com) which leverages neural nets to do the job

achenatx 5y ago

<<But for Doug Polk, who largely retired from poker in 2017 after winning tens of millions of dollars, the change solvers have wrought is more existential. “I feel like it kind of killed the soul of the game,” Polk says, changing poker “from who can be the most creative problem-solver to who can memorize the most stuff and apply it.”>> I wrote probably the first online texas holdem game, played on IRC, around 1994. Back then in rec.gambling blackjack teams were forming and instead of joining a blackjack team I started to play poker, because it was more fun than counting cards. I wrote the game so I could play against other people and was immediately a winning player in low limit games of the time in lake charles and vegas. However even in the 90s, it was about statistics. I do regret not allowing people to play for money (people were constantly asking). Even back then the primitive bots were mostly killing human players.

mikemcd 5y ago
I'm pretty sure it was Vanessa Selbst who said that, not Doug Polk

sinuhe69 5y ago

I may be ignorant but I still don’t understand the fuss about this feat. I mean poker is all about probability, right? With uncertainty about bluffing and fraught with emotional control problems. So naturally, a machine is best suited for this kind of problem, but not human. The bluffing problem can also be solved with heuristic optimization techniques like GA. Thus, is it not obvious that machines/algorithms will beat humans in the long run?

mym1990 5y ago
The distinction is that the number of branches that manifest in games of incomplete information can become utterly massive, and then you add in 6-10 players at once, all playing at world class levels, and you have a real computation problem. I mean maybe this is a trivial thing to solve for you, in which case kudos, but its pretty neat for a lot of onlookers(and participants).

randomNumber7 5y ago

There is some misconception about the optimal strategy in the article. You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. You have to play an exploitable strategy yourself to exploit someone else. Think about playing rock paper scissors agains an opponent who always chooses rock. If you still play the optimal strategy (choosing randomly) you gain nothing...

misja111 5y ago
> You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. This is not true. If you play optimal strategy, you will win against any opponent except one that plays optimal as well, in which case you'll break even. But, of course you'll win a lot more if you are able to adapt your strategy to exploit any weaknesses that you have detected. And detecting weaknesses in your opponent(s) in live play is something that humans will remain better at than AI for quite a while. Because it requires not just careful analysis of your opponent's actions but some contextual information as well. E.g., how old is your opponent, is he experienced or not, is he drunk, have you experienced similar players like hime before, etc.
- 3pt14159 5y ago
  > If you play optimal strategy, you will win against any opponent except one that plays optimal as well How does the example you're responding to not win 50% of the time? Rock v Rock = Tie Rock v Paper = Loss Rock v Scissors = Win The optimal game theoretic play of randomly choosing rock paper scissors is inferior play against this particular opponent. All that game theoretic perfect play gets you is the benefit of getting at least a tie. Possibly more, but not always more for non-perfect play.
  - kuboble 5y ago
    Poker isn't like this. Think rock, pair, scissors, crap where rules are that crap always loses. Now uniform strategy between rps will yield profit vs an opponent who plays crap sometimes. It's very easy to play crap in poker

rep_movsd 5y ago

This whole thing is a misconception about poker. Poker is never about calculating odds in your head, but bluffing convincingly and reading the opponents tells. Computers have no tells, and I doubt if any ML can learn to recognize tells of people, since they are very subtle. The statistics part anyone with basic math skills can figure in their head - that doesnt guarantee any win.