Two Plus Two Publishing LLC
Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

Notices

News, Views, and Gossip For poker news, views, and gossip

Reply
 
Thread Tools Display Modes
Old 01-08-2015, 04:00 PM   #1
pocket_zeros
veteran
 
Join Date: Jun 2008
Posts: 2,252
Computers Conquer Texas Hold'em Poker for First Time

http://spectrum.ieee.org/tech-talk/c...for-first-time

Cliffs:
  • Limit-holdem only for now
  • New algorithm with a different "regret minimization technique" to select the best strategy at each step of the game
  • Reduced temporal averaging of algorithm to most recent hands vs all hands, which reduces computation and memory required
  • Used compression to reduced data set required to *just* 11 terabytes for counterfactual data and 6 terabytes for main strategy
  • Uses 200 computer nodes each consisting of 24 2.1Ghz AMD cores, 32GB of ram, and 1TB disk
  • Algorithm confirms that some basic tenants of optimal play hold up, including the advantage of position and aggression/raising, whereas some tenants were contradicted, like always bet-capping streets with the nuts and overly selective starting hand requirements out-of-position

Last edited by pocket_zeros; 01-08-2015 at 04:15 PM.
pocket_zeros is offline   Reply With Quote
Old 01-08-2015, 04:05 PM   #2
Larry Legend
Celtic Pride
 
Larry Legend's Avatar
 
Join Date: Jul 2009
Location: Kyrie's earth
Posts: 42,906
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
A new algorithm has taken the first big step in figuring out poker, the globally popular card game played by more than 150 million people, by solving a two-player version known as heads-up limit Texas hold’em.
.
Larry Legend is offline   Reply With Quote
Old 01-08-2015, 04:07 PM   #3
nedprz
grinder
 
nedprz's Avatar
 
Join Date: Mar 2013
Location: cold
Posts: 441
Re: Computers Conquer Texas Hold'em Poker for First Time

wouldn't really call that conquering
nedprz is offline   Reply With Quote
Old 01-08-2015, 04:09 PM   #4
theskillzdatklls
Pooh-Bah
 
theskillzdatklls's Avatar
 
Join Date: May 2008
Location: PM this account before trading w/me
Posts: 4,601
Re: Computers Conquer Texas Hold'em Poker for First Time

http://news.yahoo.com/self-taught-co...192346149.html

i saw that one but it doesn't sound that great. soon enough tho.
theskillzdatklls is offline   Reply With Quote
Old 01-08-2015, 04:10 PM   #5
Doc T River
Master of the Edit Line
 
Doc T River's Avatar
 
Join Date: Sep 2007
Location: Christian Player NEVER at Venetian
Posts: 12,890
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
The paper defines a lifetime of games as 200 games of poker an hour for 12 hours a day over the course of 70 years.
Do they say what constitutes a game?

Last edited by Doc T River; 01-08-2015 at 04:16 PM. Reason: Hooray, I learned how to quote from other sources.
Doc T River is offline   Reply With Quote
Old 01-08-2015, 04:14 PM   #6
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,671
Re: Computers Conquer Texas Hold'em Poker for First Time

It really bothers me when reporters don't know enough to explain something. In the article they mention that the computer rarely caps preflop with aces. But that misleads non poker players who weren't told that the bet size doubles on the turn. I doubt it would make this play otherwise.
David Sklansky is offline   Reply With Quote
Old 01-08-2015, 04:28 PM   #7
stealinpotatoes
banned
 
stealinpotatoes's Avatar
 
Join Date: Sep 2014
Posts: 577
Re: Computers Conquer Texas Hold'em Poker for First Time

Here's the real article:

Science 9 January 2015:
Vol. 347 no. 6218 pp. 145-149
DOI: 10.1126/science.1259433

It's not public access.

They reraise with a pair of threes.
stealinpotatoes is offline   Reply With Quote
Old 01-08-2015, 04:31 PM   #8
pocket_zeros
veteran
 
Join Date: Jun 2008
Posts: 2,252
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by David Sklansky View Post
It really bothers me when reporters don't know enough to explain something. In the article they mention that the computer rarely caps preflop with aces. But that misleads non poker players who weren't told that the bet size doubles on the turn. I doubt it would make this play otherwise.
On balance I think the article did a good job summarizing the project. In writing just as in information theory, it's impossible to distill the essence of a something without losing some detail.
pocket_zeros is offline   Reply With Quote
Old 01-08-2015, 04:36 PM   #9
bluffers
enthusiast
 
Join Date: May 2014
Posts: 92
Re: Computers Conquer Texas Hold'em Poker for First Time

What about algorithms such as "deep learning" being applied to poker.

Whats the problem with these new algorithms is, computer scientist say they can apply the same algorithm to many fields, such as a program designed to learn how to beat pong and diagnos medical data.

Whats amazing about A.I is it really seems to only be mostly computational power and data.
bluffers is offline   Reply With Quote
Old 01-08-2015, 04:43 PM   #10
gdsfather
grinder
 
gdsfather's Avatar
 
Join Date: Aug 2012
Location: earth and space
Posts: 591
Re: Computers Conquer Texas Hold'em Poker for First Time

Hasn't this been around for a while for limit?
gdsfather is offline   Reply With Quote
Old 01-08-2015, 04:43 PM   #11
syntackz
stranger
 
Join Date: Oct 2014
Posts: 4
Re: Computers Conquer Texas Hold'em Poker for First Time

http://poker.srv.ualberta.ca/

This is the university's project page. Its under some heavy load right now, but you can play against the bot and view its strategies.
syntackz is offline   Reply With Quote
Old 01-08-2015, 05:01 PM   #12
0desmu1
old hand
 
Join Date: May 2012
Posts: 1,557
HU limit poker finally solved

Or at least that's what this article says:

http://www.washingtonpost.com/news/s...ying-computer/

Looks like it's backed by science so I would assume there is some legitimacy to it.

It looks like it's solved HU limit poker by the way of Snowie where it just played against itself billion billions (not a typo) of times to find the most optimal line in every single scenario. It's not an intuitive program but rather a database recall. Will be interesting what comes from this.
0desmu1 is offline   Reply With Quote
Old 01-08-2015, 05:32 PM   #13
LektorAJ
Carpal \'Tunnel
 
LektorAJ's Avatar
 
Join Date: May 2014
Location: none
Posts: 7,519
Re: Computers Conquer Texas Hold'em Poker for First Time

^ This sounds like the stuff where we have to ask China to invade Taiwan 1 000 000 times so we can calibrate our military planning AI. It's fine for games but its a bit of a dead end for more real world stuff.
LektorAJ is offline   Reply With Quote
Old 01-08-2015, 05:35 PM   #14
darthsikle
adept
 
darthsikle's Avatar
 
Join Date: Oct 2008
Location: Perkasie, PA
Posts: 745
Re: Computers Conquer Texas Hold'em Poker for First Time

I heard after a few million test runs the computer complains that the damn fish call any two cards.
darthsikle is offline   Reply With Quote
Old 01-08-2015, 05:52 PM   #15
DougL
Too helpful for this post
 
DougL's Avatar
 
Join Date: Sep 2002
Location: Boulder, CO
Posts: 21,767
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by darthsikle View Post
I heard after a few million test runs the computer complains that the damn fish call any two cards.
No, it posts some sick graph up/to-the-right and claims to have solved poker.
DougL is offline   Reply With Quote
Old 01-08-2015, 06:00 PM   #16
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,671
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by pocket_zeros View Post
On balance I think the article did a good job summarizing the project. In writing just as in information theory, it's impossible to distill the essence of a something without losing some detail.
You are being to kind. If they are going to make a point about not capping with aces they should be competent enough to elaborate so as not to mislead. (Although the guys that were interviewed might be culpable as well.)
David Sklansky is offline   Reply With Quote
Old 01-08-2015, 06:22 PM   #17
LukeSilver
adept
 
Join Date: Jun 2010
Posts: 918
Re: Computers Conquer Texas Hold'em Poker for First Time

so does this program adapt, to players or just play an unexploitable game?

I found a number of spots in my poker variant with certain ranges etc that are 100% unexploitable what i found though is that i beat the players but lost to the rake, i had to vary from that line to strategys that were exploitable but exploited the weakness in weak opponents.

a bot that is unexploitable may potentially just lose to the rake but im guessing it will have enough of an edge on randoms with no concept to avoid that fate i dont know.
LukeSilver is offline   Reply With Quote
Old 01-08-2015, 06:24 PM   #18
Loctus
Carpal \'Tunnel
 
Loctus's Avatar
 
Join Date: Nov 2008
Posts: 10,031
Re: Computers Conquer Texas Hold'em Poker for First Time

Bloomberg article: http://www.bloomberg.com/news/2015-0...ong-moves.html

“We have a strategy that can guarantee a player won’t lose,” said Michael Bowling, a computer scientist from the University of Alberta, who led a team working on the program. “It’s going to be a break-even game. It’s only when someone makes a mistake that they could end up losing.”

In other words: they claim GTO


edit: It seems like they brute forced the solution, right? Just ran enough sims to the point where every random hand and random decision point had been up against every other random hand and decision point and from there on narrowed it down to the strategy with the most over-all +EV choice in every spot?

Last edited by Loctus; 01-08-2015 at 06:29 PM.
Loctus is offline   Reply With Quote
Old 01-08-2015, 06:33 PM   #19
DaycareInferno
Carpal \'Tunnel
 
DaycareInferno's Avatar
 
Join Date: Jun 2007
Posts: 18,279
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by gdsfather View Post
Hasn't this been around for a while for limit?
iirc they have been very close for a long time.
DaycareInferno is offline   Reply With Quote
Old 01-08-2015, 06:54 PM   #20
Conan776
Pooh-Bah
 
Conan776's Avatar
 
Join Date: Jul 2007
Location: Land of the Great Blue Hill
Posts: 3,685
Re: Computers Conquer Texas Hold'em Poker for First Time

Heads up limit? How long until it becomes self-aware and starts flinging actual poo?
Conan776 is offline   Reply With Quote
Old 01-08-2015, 07:03 PM   #21
FullyCompletely
enthusiast
 
Join Date: Mar 2009
Location: UofA CPRG member
Posts: 64
Re: Computers Conquer Texas Hold'em Poker for First Time

Hey! This is Mike Johanson, one of the authors on the paper. We're swamped right now keeping the website up, talking to the press, and running Twitter, so I only have a few seconds to spare, but I'll stop by 2+2 later to answer questions.

We're on twitter as @PolarisPoker . Check out my 30-tweet blitz from earlier today for a summary of what we've done.

We've been working towards this result for years, and have gotten closer and closer over time. Last year, we were down to 26 milli-big-blinds per game, or 1.3 BB/100, of exploitability: that's the most that a perfect counter-strategy / nemesis could win against our program. I've talked about that result on 2+2 a few times in the past, and the tools that we use to calculate that exploitability figure.

In October 2013, after I commented a few times in one 2+2 thread, our coauthor Oskari Tammelin got ahold of me. We'd worked with Oskari before, when he invented his PureCFR algorithm. He told us about two things he'd recently invented: CFR+, a new game solving algorithm based on our CFR algorithm that learns *incredibly* fast compared to what was then the state-of-the-art version of CFR, and a new compression technique that reduced the memory cost for solving holdem from 523 TB to 11 TB.

Oskari's two developments were hugely important, and opened the door to finally make solving HULHE possible. This paper is a joint effort between the U of A (me, Mike Bowling, Neil Burch) and Oskari Tammelin. It took us most of a year to write the code and tune it to be able to get the quality of result we wanted, at a memory limit that fit on our cluster, and at a CPU cost that was feasible. This was a tricky 3-way optimization. Actually running the computation used 4800 CPUs for about 70 days, with a total computation time of just under 1000 CPU-years.

The result is that our new program Cepheus is beatable for less than 1 milli-big-blind, or 0.05 BB/100. Even if you knew the perfect counter-strategy and could play it flawlessly, it'd take 60 million games to overcome the variance due to luck in order to actually have 95% confidence that you were winning. It's essentially solved: not quite perfect, but closer than any human could distinguish within a lifetime of play.

Our website is constantly going up and down right now due to traffic, but here's a link: http://poker.srv.ualberta.ca . The site will show you the preflop strategy (as a nice set of graphics), the strategy for any decision point in the game, and will also let you play against Cepheus. Although the "Play vs Cepheus" part is the part that is most swamped at the moment. It should get easier to get through in a few days.

I'll check back in on this thread tomorrow, when we're less hectic. If you have any questions about the result, I'll be happy to answer them then.
FullyCompletely is offline   Reply With Quote
Old 01-08-2015, 07:13 PM   #22
Loctus
Carpal \'Tunnel
 
Loctus's Avatar
 
Join Date: Nov 2008
Posts: 10,031
Re: Computers Conquer Texas Hold'em Poker for First Time

inb4 high stakes prop bets are made
Loctus is offline   Reply With Quote
Old 01-08-2015, 07:25 PM   #23
14756897412369
centurion
 
14756897412369's Avatar
 
Join Date: Jan 2015
Posts: 102
Re: Computers Conquer Texas Hold'em Poker for First Time

i assume game = hand?
14756897412369 is offline   Reply With Quote
Old 01-08-2015, 07:32 PM   #24
paletokio
old hand
 
paletokio's Avatar
 
Join Date: Nov 2009
Location: Microrollin
Posts: 1,236
Re: Computers Conquer Texas Hold'em Poker for First Time

Who plays limit holdem anyway...
paletokio is offline   Reply With Quote
Old 01-08-2015, 07:36 PM   #25
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by David Sklansky View Post
You are being to kind. If they are going to make a point about not capping with aces they should be competent enough to elaborate so as not to mislead. (Although the guys that were interviewed might be culpable as well.)
I'm one of the other authors (Neil Burch - webdocs.cs.ualberta.ca/~burch/two_plus_two.html for the skeptical) and I claim no responsibility for any poker errors in any media articles! I avoided any specific examples just because I figured I would mangle them. I'm not sure what Mike, Mike, or Oskari might have said, though...

Responding to others...
I will own up to the "200 games per hour" bit. Sorry. That probably should have been "200 hands per hour." From my point of view, as a game theorist and not a poker player, poker is a repeated game, but we tried to avoid that terminology specifically because it's confusing in the context of poker. We missed one, I guess. I _think_ I set it up so that it really is a single sided hand, not an average of playing both seats, but I'd have to double check the math to make sure.

I'm kind of happy to see a resounding "Meh." We hoped everyone with a bit of knowledge would be aware that we hadn't somehow ruined the game of poker by finding an (almost) game theoretically optimal strategy, but I wasn't sure exactly how much credit to give people... It's oddly reassuring to see a bunch of people say "who cares?"

Last edited by nburch; 01-08-2015 at 07:37 PM. Reason: making it clear the whole response is not to David Sklansky's comment
nburch is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 02:19 AM.


Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright © 2008-2020, Two Plus Two Interactive
 
 
Poker Players - Streaming Live Online