Historic: AI System called DeepStack Beats 11 Pro Poker Players at Texas Hold'em

This latest discovery builds on an already impressive body of research findings about artificial intelligence and imperfect information games that stretches back to the creation of the University of Alberta’s Computer Poker Research Group in 1996. Bowling, who became the group’s principal investigator in 2006, has led the group to several milestones for artificial intelligence. He and his colleagues developed Polaris in 2008, beating top poker players at heads-up limit Texas hold’em poker. They then went on to solve heads-up limit hold’em with Cepheus, published in 2015 in Science.

DeepStack extends the ability to think about each situation during play–which has been famously successful in games like checkers, chess, and Go–to imperfect information games using a technique called continual re-solving. This allows DeepStack to determine the correct strategy for a particular poker situation without thinking about the entire game by using its “intuition” to evaluate how the game might play out in the near future.

“We train our system to learn the value of situations,” says Bowling. “Each situation itself is a mini poker game. Instead of solving one big poker game, it solves millions of these little poker games, each one helping the system to refine its intuition of how the game of poker works. And this intuition is the fuel behind how DeepStack plays the full game.”

Thinking about each situation as it arises is important for complex problems like heads-up no-limit hold’em, which has vastly more unique situations than there are atoms in the universe, largely due to players’ ability to wager different amounts including the dramatic “all-in.” Despite the game’s complexity, DeepStack takes action at human speed — with an average of only three seconds of “thinking” time–and runs on a simple gaming laptop with an Nvidia graphics processing unit.

To test the approach, DeepStack played against a pool of professional poker players in December, 2016, recruited by the International Federation of Poker. Thirty-three players from 17 countries were recruited, with each asked to play a 3000-hand match over a period of four weeks. DeepStack beat each of the 11 players who finished their match, with only one outside the margin of statistical significance, making it the first computer program to beat professional players in heads-up no-limit Texas hold’em poker.

Contacts and sources: