Jason Les and Libratus Creator Noam Brown Discuss the Bot

Jason Les took time during his shift of the Brains vs AI Challenge with the Libratus bot being run on a super computer. Noam Brown, the creator and coder of Libratus, joined Les on the Twitch stream to speak about the challenge and answer questions from viewers.

The Brains – Les, Dong Kim, Jimmy Chuo and Daniel McAulay – are roughly halfway through the 20-day challenge set for 120,000 hands. Each session against the AI begins with a 200 big blind stack. the players take turns playing heads-up and the group is collectively down $794,392.
 
“I’ve been pretty exhausted every day,” Les said. “Losing this big has been rough. It makes it so we’re not really playing our best, but I can’t say that it’s the difference between winning and losing.”

Kim has had the best success and is only down $20,940 compared to Les’ $308,324 hole; Chou is down $228,679 and McAulay is down $236,449.

The Brains meet each night to go over the day’s play and compare notes. Les felt the biggest challenge has been Libratus seems to be learning during the downtime. “It’s learning every night and there’s some insane stuff happening.”

A few minutes later Les also revealed that the bot is not playing the same against each player.

Brown designed the bot and coded the AI as part of a research project. “We haven’t solved heads-up No Limit, we’re just beating the humans,” Brown said. “The program uses reinforcement learning to learn how to play poker.”

“We taught it how to play and then it played itself trillions of times to develop a strategy,” said Brown. “There’s no human strategy programmed into it. It started randomly, has no historical data and slowly learned based on that information.”

The success Libratus has over the human players at this point in the competition comes as a surprise to Brown. “I’m thrilled the bot is beating the humans,” he said. “To be honest, I had no idea how it would do.”

Brown does intend to release information and data concerning the bot’s strategy following the competition. The exact hand histories won’t be disclosed to the public by request of the Brains. The competition runs through January 30th and has live Twitch streams live from River’s Casino in Pittsburgh each day from 11 am to 7 pm ET.

The Q & A finished up after around an hour of questions. Before Les signed off he said, “I can’t believe no one asked if the bot bluffs.”