Two Plus Two Publishing LLC
Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

Notices

News, Views, and Gossip For poker news, views, and gossip

Reply
 
Thread Tools Display Modes
Old 01-09-2015, 03:32 PM   #126
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by 4-Star General View Post
Maybe I'm a bit to extreme... but hackers are cracking the biggest sites ever, aren't you fearing a such attack?
I mean, they could stealing your code and making a working bot without any problem I think.
While I do understand the legit reasons of posting your project, I think it is way to dangerous...

What do you think?
No fear!

The code to solve the game is open. You can go download it right now, and run it on your PC with a pile of 6TB drives. Have fun, see you in ten years from now (unless you've got your own research cluster - that IS possible...) It's always been a question of available resources. If you ignore practicality, you could have solved poker back in the 1950s. Koller's game theory work in the late 1990s made it interesting. 2007 and CFR made solving limit Hold'em seem out of reach, but not crazy. 2014 just sped things up.

As for being worried about someone breaking in and stealing the 11TB table of numbers that lists Cepheus' strategy. Really? Can't tell if serious...
nburch is offline   Reply With Quote
Old 01-09-2015, 03:35 PM   #127
4-Star General
adept
 
4-Star General's Avatar
 
Join Date: Feb 2014
Posts: 1,012
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by nburch View Post
No fear!

The code to solve the game is open. You can go download it right now, and run it on your PC with a pile of 6TB drives. Have fun, see you in ten years from now (unless you've got your own research cluster - that IS possible...) It's always been a question of available resources. If you ignore practicality, you could have solved poker back in the 1950s. Koller's game theory work in the late 1990s made it interesting. 2007 and CFR made solving limit Hold'em seem out of reach, but not crazy. 2014 just sped things up.

As for being worried about someone breaking in and stealing the 11TB table of numbers that lists Cepheus' strategy. Really? Can't tell if serious...

totally serious, but I'm not a tech savvy, so you know, no knowledge=stupid questions.

Ty for your time for answering
4-Star General is offline   Reply With Quote
Old 01-09-2015, 03:38 PM   #128
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by 4-Star General View Post
totally serious, but I'm not a tech savvy, so you know, no knowledge=stupid questions.

Ty for your time for answering
Okay, then have no fears about someone stealing Cepheus: even if you walked in with your own hard drive, and did a copy command, it would take more than a day. It's a BIG file.
nburch is offline   Reply With Quote
Old 01-09-2015, 03:52 PM   #129
punter11235
Carpal \'Tunnel
 
Join Date: Mar 2005
Location: solving poker
Posts: 8,007
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
I mean, they could stealing your code and making a working bot without any problem I think.
They steal the code... then they just need to find 4800 cores and a lot of disc space to run it for 2months. Then they need to find someone to play against in HU LHE and then they need to find a way to get assistance from their newly calculated solution while playing.
I mean... there are lower hanging fruits out there if you have this kind of resources and want to engineer a scam.

Quote:
The code to solve the game is open. You can go download it right now, and run it on your PC
Link please ? (Cepheus home page is unavailable right now)
punter11235 is offline   Reply With Quote
Old 01-09-2015, 03:57 PM   #130
Hood
Carpal \'Tunnel
 
Hood's Avatar
 
Join Date: Apr 2004
Location: 99 problems but a TT+ just ship pf
Posts: 6,866
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by fsn View Post
Too bad there's a flaw with Cepheus...

http://www.theguardian.com/sport/201...t-cepheus-flaw
Wow, i hope this is an A* troll. It is quite brilliant.
Hood is offline   Reply With Quote
Old 01-09-2015, 04:01 PM   #131
HIV
Pooh-Bah
 
HIV's Avatar
 
Join Date: Apr 2006
Location: bomb on his waist, mask on his face
Posts: 4,547
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by nburch View Post
We've done something that is hopefully one better: you can just directly check how Cepheus plays. Set up the board and the betting, and it will tell you exactly how it plays every hand, how likely it was to reach that situation with every hand, and how likely every opponent hand should be assuming they played like Cepheus.

We're aware that releasing Cepheus' decision could end up being a bit unpopular, but we tried to balance the requirements of both science and Science (results must be open), and the poker community. That's actually part of why we dump so much information at once: it's hopefully useful to someone who is legitimately learning something, while at the same time being too slow for someone to just sit there and use it to directly play a real game.

I guess the one exception we didn't think of was the online VLT-style poker boxes, which won't care if you take 10 seconds to make all your actions. We'll have to think about this one. Clean 'em out while you can, I guess (if you can't do it already...)


Either way, it's all irrelevant until we can keep the web site from repeatedly going down.
Let me dumb this down, because a lot of people are dumb like me but I think the gist of this is:

On this website http://poker.srv.ualberta.ca/strategy# (down right now) you can literally in real time ask it what to do in any hand of heads up limit holdem. Doesn't matter if you're playing $1500/$3000 stakes, as long as you have 10 second to make an action, you can play UNBEATABLE* at heads up limit holdem. Right now. On online poker. Any site.

How soon before sites like stars and full tilt take Holdem out of their mixed games rotation? Hell, how long before limit Holdem is taken off the board entirely?



*unbeatable = not necessarily the best strategy against all opponents, you could win more by exploiting obvious tendencies, and also if someone else is using the same strategy as you, you will both lose bad to the rake. but you will not get beat.
HIV is offline   Reply With Quote
Old 01-09-2015, 04:05 PM   #132
14756897412369
centurion
 
14756897412369's Avatar
 
Join Date: Jan 2015
Posts: 102
Re: Computers Conquer Texas Hold'em Poker for First Time

lol wow it has to be a troll. no "poker writer" would be so poker illiterate to actually write that and seriously believe it. the guardian got trolled imo. the bad thing is any non-poker players reading that and believing it.
14756897412369 is offline   Reply With Quote
Old 01-09-2015, 04:15 PM   #133
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,672
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by nburch View Post
Yeah, you're right - I was treating that specific thing as a more general case. We could have had special case code to handle drawing to the nuts.

The reason not to in my treating-it-as-the-general-case answer still holds: at least for constraining that policy that we learn, we tended to avoid special case rules after getting that wrong once. We don't even force it to not fold if it already HAS the nuts, and I'm pretty sure we have some strategies with some very small, but non-zero probability of doing that. I'm not 100% sure we ever even talked about drawing to the nuts, although I think we did.

Given a reluctance to add special cases, anything we did would have to have some expectation of moderate improvement in speed, space, or performance. Assuming I'm right that we did consider the case of drawing to the nuts, we would have seen it fail on all three, and dropped it.

The way we do things, it wouldn't save us any computation. It would actually be a (probably trivial) bit more computation.

It doesn't save us any space. It technically should save us space, because we have less decisions to make, but they're sprinkled across the game in a fashion that makes it hard for us to take advantage of.

It doesn't make or break our performance. Would it help? Sure. As long as it's possibly doing something stupid, forcing it not to has to be better. Would it help enough to make a statistically visible difference? Probably not: this should be the sort of thing that it's able to learn very quickly.

Should we make sure to do this before we do any other man vs. machine event/exhibition/whatever? Probably! It would certainly be embarrassing having to explain that it's a one in a million mistake that doesn't really affect the bottom line.
Thank you. I only brought this up because the casino heads up machine has been known to make this mistake. I assume this occurs because even with trillions of hands dealt, very specific situations may come up only a few thousand times and in those few thousand the computer did very slightly better by folding even though its EV was very slightly better if it called. Or something along those lines. (Am I right?) But in the case of drawing to the dead nuts with one to come it seems that a simple instruction (though maybe not that simple because of the unknown two cards) could guarantee avoiding an embarrassing (as far as the press is concerned) error.
David Sklansky is offline   Reply With Quote
Old 01-09-2015, 04:26 PM   #134
RonMexico
Pooh-Bah
 
RonMexico's Avatar
 
Join Date: Feb 2006
Posts: 4,323
Re: Computers Conquer Texas Hold'em Poker for First Time

How long did it take to get this through the review process at Science? I imagine there aren't too many people working on computational poker, so I'd be interested to know if any of the reviewers had any major hang-ups or demands for a revision.
RonMexico is offline   Reply With Quote
Old 01-09-2015, 04:35 PM   #135
cts
Carpal \'Tunnel
 
cts's Avatar
 
Join Date: Dec 2005
Posts: 7,738
Re: Computers Conquer Texas Hold'em Poker for First Time

Pretty amazing accomplishment, congrats! That Guardian article had be dying over here lol
cts is offline   Reply With Quote
Old 01-09-2015, 04:39 PM   #136
frinxor
adept
 
Join Date: Dec 2006
Location: SF
Posts: 723
Re: Computers Conquer Texas Hold'em Poker for First Time

Awesome work!

It does seem like only a matter of time until theres 3 4TB torrent files with the tables floating on the webs, and some software that lets you retrieve the numbers.

Set up some 6TB harddrives, adjust an existing bot, and then you've got Cepheus printing money (or Stars printing money for Cepheus vs Cepheus)

over/under 1 month?
frinxor is offline   Reply With Quote
Old 01-09-2015, 04:41 PM   #137
Lateralus
journeyman
 
Join Date: Nov 2005
Posts: 307
Re: Computers Conquer Texas Hold'em Poker for First Time

No money in HU LHE, computer's solid
Lateralus is offline   Reply With Quote
Old 01-09-2015, 04:42 PM   #138
14756897412369
centurion
 
14756897412369's Avatar
 
Join Date: Jan 2015
Posts: 102
Re: Computers Conquer Texas Hold'em Poker for First Time

its been pointed out many times. there isn't much money to be printed at online hulhe. certainly not worth the effort of going through all that to make a functional bot that can play on poker sites. and anyone stupid enough to play against it long enough is also probably stupid enough that one could easily beat them without the bot.
14756897412369 is offline   Reply With Quote
Old 01-09-2015, 04:42 PM   #139
kokiri
ffermwr moch
 
kokiri's Avatar
 
Join Date: Feb 2006
Location: not in Kansas anyore, Toto
Posts: 51,654
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by Lateralus View Post
No money in HU LHE, AI body's solid

fyp.
kokiri is offline   Reply With Quote
Old 01-09-2015, 04:42 PM   #140
odiggity
banned
 
odiggity's Avatar
 
Join Date: May 2005
Posts: 894
Re: Computers Conquer Texas Hold'em Poker for First Time

Leading quantum physics expert Sean Carroll wrote an article about this today. My bad if it's already been referenced itt http://www.preposterousuniverse.com/...tially-solved/
odiggity is offline   Reply With Quote
Old 01-09-2015, 04:43 PM   #141
frinxor
adept
 
Join Date: Dec 2006
Location: SF
Posts: 723
Re: Computers Conquer Texas Hold'em Poker for First Time

Whats next for the Cepheus team?
frinxor is offline   Reply With Quote
Old 01-09-2015, 05:05 PM   #142
Hood
Carpal \'Tunnel
 
Hood's Avatar
 
Join Date: Apr 2004
Location: 99 problems but a TT+ just ship pf
Posts: 6,866
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by RonMexico View Post
How long did it take to get this through the review process at Science? I imagine there aren't too many people working on computational poker, so I'd be interested to know if any of the reviewers had any major hang-ups or demands for a revision.
Really interested in this.

Quote:
Originally Posted by cts View Post
Pretty amazing accomplishment, congrats! That Guardian article had be dying over here lol
And also, this.
Hood is offline   Reply With Quote
Old 01-09-2015, 05:49 PM   #143
AllBlackDan
Pooh-Bah
 
AllBlackDan's Avatar
 
Join Date: Nov 2008
Location: The Moon
Posts: 5,491
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by fsn View Post
Too bad there's a flaw with Cepheus...

http://www.theguardian.com/sport/201...t-cepheus-flaw

Spoiler:
Luls
AllBlackDan is offline   Reply With Quote
Old 01-09-2015, 05:56 PM   #144
LektorAJ
Carpal \'Tunnel
 
LektorAJ's Avatar
 
Join Date: May 2014
Location: none
Posts: 7,521
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by paletokio View Post
Who plays limit holdem anyway...
If I was in the mood the gently rib Sklansky, I would say that computer scientists prefer to solve toy games like Kuhn poker and Limit Holdem because real world games like NLHE are too difficult to solve with present techniques.
LektorAJ is offline   Reply With Quote
Old 01-09-2015, 05:59 PM   #145
TheBryce2
stranger
 
Join Date: Jan 2015
Location: Forgot my PW.
Posts: 14
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by HIV View Post
Let me dumb this down, because a lot of people are dumb like me but I think the gist of this is:

On this website http://poker.srv.ualberta.ca/strategy# (down right now) you can literally in real time ask it what to do in any hand of heads up limit holdem. Doesn't matter if you're playing $1500/$3000 stakes, as long as you have 10 second to make an action, you can play UNBEATABLE* at heads up limit holdem. Right now. On online poker. Any site.

How soon before sites like stars and full tilt take Holdem out of their mixed games rotation? Hell, how long before limit Holdem is taken off the board entirely?


*unbeatable = not necessarily the best strategy against all opponents, you could win more by exploiting obvious tendencies, and also if someone else is using the same strategy as you, you will both lose bad to the rake. but you will not get beat.
A technical quibble here is that you can't just copy one of Cepheus' actions and consider it unexploitable. For the strategy to work EVERY decision you make has to be a duplicate of Cepheus', which includes taking actions with the correct randomly generated frequency when they aren't binary.
TheBryce2 is offline   Reply With Quote
Old 01-09-2015, 06:06 PM   #146
jaytorr
centurion
 
Join Date: May 2003
Posts: 127
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by David Sklansky View Post
Thank you. I only brought this up because the casino heads up machine has been known to make this mistake. I assume this occurs because even with trillions of hands dealt, very specific situations may come up only a few thousand times and in those few thousand the computer did very slightly better by folding even though its EV was very slightly better if it called. Or something along those lines. (Am I right?) But in the case of drawing to the dead nuts with one to come it seems that a simple instruction (though maybe not that simple because of the unknown two cards) could guarantee avoiding an embarrassing (as far as the press is concerned) error.

If I'm reading the CFR+ description correctly, it evaluates the entire game tree on every iteration, so it should learn to never fold the nuts on the end very quickly (after first iteration?). Folding (occasionally raising) hopeless hands on the end shouldn't take much longer. Once it learns a reasonable river strategy, the turn decision with the nut draw will follow since the algorithm evaluates the EV of every possible river card on every iteration.

So if I'm understanding it correctly, CFR+ is less susceptible to this type of error as Monte Carlo-based algorithms which may not sample some nodes frequently enough. Of course, if the decision is very close after accounting for implied odds, it might still get it wrong. But then it wouldn't be an obvious mistake.

nburch - Can you confirm? Also, congrats!
jaytorr is offline   Reply With Quote
Old 01-09-2015, 06:06 PM   #147
TheBryce2
stranger
 
Join Date: Jan 2015
Location: Forgot my PW.
Posts: 14
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by fsn View Post
Too bad there's a flaw with Cepheus...

http://www.theguardian.com/sport/201...t-cepheus-flaw

TheBryce2 is offline   Reply With Quote
Old 01-09-2015, 06:16 PM   #148
Jean
old hand
 
Join Date: Nov 2007
Location: Got house!
Posts: 1,585
Re: Computers Conquer Texas Hold'em Poker for First Time

The Guardian is the new Daily Mail.

Also congrats!

Quote:
Originally Posted by nburch View Post
It caps .06% of the time with 9T suited - most likely part of the noise that makes it "essentially" solved and not just solved.
If you mean that capping at all is noise:
Can't you solve the (ever so slightly) easier game where BTN is not allowed to cap, and when finding that the BTN's average gain is no lesser than the original gain, conclude that capping was useless?
Jean is offline   Reply With Quote
Old 01-09-2015, 06:16 PM   #149
ArtyMcFly
Carpal \'Tunnel
 
ArtyMcFly's Avatar
 
Join Date: Dec 2014
Location: Enchantment Under the Sea
Posts: 13,256
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by 14756897412369 View Post
lol wow it has to be a troll. no "poker writer" would be so poker illiterate to actually write that and seriously believe it. the guardian got trolled imo.
@ChrisKPHall: "Sadly I only got literally two hours to write, play and turn in the article and some parts got cut."

[x] Counterfactual regret minimization.
ArtyMcFly is offline   Reply With Quote
Old 01-09-2015, 06:25 PM   #150
Michman
veteran
 
Michman's Avatar
 
Join Date: Jul 2011
Posts: 2,255
Re: Computers Conquer Texas Hold'em Poker for First Time

I'd like to see it try to play against a real quality human player. My money is on the computer. No tell.
Michman is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 03:01 AM.


Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright 2008-2020, Two Plus Two Interactive
 
 
Poker Players - Streaming Live Online