Open Side Menu Go to the Top
Register
Computers Conquer Texas Hold'em Poker for First Time Computers Conquer Texas Hold'em Poker for First Time

01-09-2015 , 03:32 PM
Quote:
Originally Posted by 4-Star General
Maybe I'm a bit to extreme... but hackers are cracking the biggest sites ever, aren't you fearing a such attack?
I mean, they could stealing your code and making a working bot without any problem I think.
While I do understand the legit reasons of posting your project, I think it is way to dangerous...

What do you think?
No fear!

The code to solve the game is open. You can go download it right now, and run it on your PC with a pile of 6TB drives. Have fun, see you in ten years from now (unless you've got your own research cluster - that IS possible...) It's always been a question of available resources. If you ignore practicality, you could have solved poker back in the 1950s. Koller's game theory work in the late 1990s made it interesting. 2007 and CFR made solving limit Hold'em seem out of reach, but not crazy. 2014 just sped things up.

As for being worried about someone breaking in and stealing the 11TB table of numbers that lists Cepheus' strategy. Really? Can't tell if serious...
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 03:35 PM
Quote:
Originally Posted by nburch
No fear!

The code to solve the game is open. You can go download it right now, and run it on your PC with a pile of 6TB drives. Have fun, see you in ten years from now (unless you've got your own research cluster - that IS possible...) It's always been a question of available resources. If you ignore practicality, you could have solved poker back in the 1950s. Koller's game theory work in the late 1990s made it interesting. 2007 and CFR made solving limit Hold'em seem out of reach, but not crazy. 2014 just sped things up.

As for being worried about someone breaking in and stealing the 11TB table of numbers that lists Cepheus' strategy. Really? Can't tell if serious...

totally serious, but I'm not a tech savvy, so you know, no knowledge=stupid questions.

Ty for your time for answering
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 03:38 PM
Quote:
Originally Posted by 4-Star General
totally serious, but I'm not a tech savvy, so you know, no knowledge=stupid questions.

Ty for your time for answering
Okay, then have no fears about someone stealing Cepheus: even if you walked in with your own hard drive, and did a copy command, it would take more than a day. It's a BIG file.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 03:52 PM
Quote:
I mean, they could stealing your code and making a working bot without any problem I think.
They steal the code... then they just need to find 4800 cores and a lot of disc space to run it for 2months. Then they need to find someone to play against in HU LHE and then they need to find a way to get assistance from their newly calculated solution while playing.
I mean... there are lower hanging fruits out there if you have this kind of resources and want to engineer a scam.

Quote:
The code to solve the game is open. You can go download it right now, and run it on your PC
Link please ? (Cepheus home page is unavailable right now)
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 03:57 PM
Quote:
Originally Posted by fsn
Too bad there's a flaw with Cepheus...

http://www.theguardian.com/sport/201...t-cepheus-flaw
Wow, i hope this is an A* troll. It is quite brilliant.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:01 PM
Quote:
Originally Posted by nburch
We've done something that is hopefully one better: you can just directly check how Cepheus plays. Set up the board and the betting, and it will tell you exactly how it plays every hand, how likely it was to reach that situation with every hand, and how likely every opponent hand should be assuming they played like Cepheus.

We're aware that releasing Cepheus' decision could end up being a bit unpopular, but we tried to balance the requirements of both science and Science (results must be open), and the poker community. That's actually part of why we dump so much information at once: it's hopefully useful to someone who is legitimately learning something, while at the same time being too slow for someone to just sit there and use it to directly play a real game.

I guess the one exception we didn't think of was the online VLT-style poker boxes, which won't care if you take 10 seconds to make all your actions. We'll have to think about this one. Clean 'em out while you can, I guess (if you can't do it already...)


Either way, it's all irrelevant until we can keep the web site from repeatedly going down.
Let me dumb this down, because a lot of people are dumb like me but I think the gist of this is:

On this website http://poker.srv.ualberta.ca/strategy# (down right now) you can literally in real time ask it what to do in any hand of heads up limit holdem. Doesn't matter if you're playing $1500/$3000 stakes, as long as you have 10 second to make an action, you can play UNBEATABLE* at heads up limit holdem. Right now. On online poker. Any site.

How soon before sites like stars and full tilt take Holdem out of their mixed games rotation? Hell, how long before limit Holdem is taken off the board entirely?



*unbeatable = not necessarily the best strategy against all opponents, you could win more by exploiting obvious tendencies, and also if someone else is using the same strategy as you, you will both lose bad to the rake. but you will not get beat.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:05 PM
lol wow it has to be a troll. no "poker writer" would be so poker illiterate to actually write that and seriously believe it. the guardian got trolled imo. the bad thing is any non-poker players reading that and believing it.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:15 PM
Quote:
Originally Posted by nburch
Yeah, you're right - I was treating that specific thing as a more general case. We could have had special case code to handle drawing to the nuts.

The reason not to in my treating-it-as-the-general-case answer still holds: at least for constraining that policy that we learn, we tended to avoid special case rules after getting that wrong once. We don't even force it to not fold if it already HAS the nuts, and I'm pretty sure we have some strategies with some very small, but non-zero probability of doing that. I'm not 100% sure we ever even talked about drawing to the nuts, although I think we did.

Given a reluctance to add special cases, anything we did would have to have some expectation of moderate improvement in speed, space, or performance. Assuming I'm right that we did consider the case of drawing to the nuts, we would have seen it fail on all three, and dropped it.

The way we do things, it wouldn't save us any computation. It would actually be a (probably trivial) bit more computation.

It doesn't save us any space. It technically should save us space, because we have less decisions to make, but they're sprinkled across the game in a fashion that makes it hard for us to take advantage of.

It doesn't make or break our performance. Would it help? Sure. As long as it's possibly doing something stupid, forcing it not to has to be better. Would it help enough to make a statistically visible difference? Probably not: this should be the sort of thing that it's able to learn very quickly.

Should we make sure to do this before we do any other man vs. machine event/exhibition/whatever? Probably! It would certainly be embarrassing having to explain that it's a one in a million mistake that doesn't really affect the bottom line.
Thank you. I only brought this up because the casino heads up machine has been known to make this mistake. I assume this occurs because even with trillions of hands dealt, very specific situations may come up only a few thousand times and in those few thousand the computer did very slightly better by folding even though its EV was very slightly better if it called. Or something along those lines. (Am I right?) But in the case of drawing to the dead nuts with one to come it seems that a simple instruction (though maybe not that simple because of the unknown two cards) could guarantee avoiding an embarrassing (as far as the press is concerned) error.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:26 PM
How long did it take to get this through the review process at Science? I imagine there aren't too many people working on computational poker, so I'd be interested to know if any of the reviewers had any major hang-ups or demands for a revision.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:35 PM
Pretty amazing accomplishment, congrats! That Guardian article had be dying over here lol
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:39 PM
Awesome work!

It does seem like only a matter of time until theres 3 4TB torrent files with the tables floating on the webs, and some software that lets you retrieve the numbers.

Set up some 6TB harddrives, adjust an existing bot, and then you've got Cepheus printing money (or Stars printing money for Cepheus vs Cepheus)

over/under 1 month?
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:41 PM
No money in HU LHE, computer's solid
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:42 PM
its been pointed out many times. there isn't much money to be printed at online hulhe. certainly not worth the effort of going through all that to make a functional bot that can play on poker sites. and anyone stupid enough to play against it long enough is also probably stupid enough that one could easily beat them without the bot.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:42 PM
Quote:
Originally Posted by Lateralus
No money in HU LHE, AI body's solid

fyp.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:42 PM
Leading quantum physics expert Sean Carroll wrote an article about this today. My bad if it's already been referenced itt http://www.preposterousuniverse.com/...tially-solved/
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 04:43 PM
Whats next for the Cepheus team?
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 05:05 PM
Quote:
Originally Posted by RonMexico
How long did it take to get this through the review process at Science? I imagine there aren't too many people working on computational poker, so I'd be interested to know if any of the reviewers had any major hang-ups or demands for a revision.
Really interested in this.

Quote:
Originally Posted by cts
Pretty amazing accomplishment, congrats! That Guardian article had be dying over here lol
And also, this.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 05:49 PM
Quote:
Originally Posted by fsn
Too bad there's a flaw with Cepheus...

http://www.theguardian.com/sport/201...t-cepheus-flaw

Spoiler:
Luls
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 05:56 PM
Quote:
Originally Posted by paletokio
Who plays limit holdem anyway...
If I was in the mood the gently rib Sklansky, I would say that computer scientists prefer to solve toy games like Kuhn poker and Limit Holdem because real world games like NLHE are too difficult to solve with present techniques.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 05:59 PM
Quote:
Originally Posted by HIV
Let me dumb this down, because a lot of people are dumb like me but I think the gist of this is:

On this website http://poker.srv.ualberta.ca/strategy# (down right now) you can literally in real time ask it what to do in any hand of heads up limit holdem. Doesn't matter if you're playing $1500/$3000 stakes, as long as you have 10 second to make an action, you can play UNBEATABLE* at heads up limit holdem. Right now. On online poker. Any site.

How soon before sites like stars and full tilt take Holdem out of their mixed games rotation? Hell, how long before limit Holdem is taken off the board entirely?


*unbeatable = not necessarily the best strategy against all opponents, you could win more by exploiting obvious tendencies, and also if someone else is using the same strategy as you, you will both lose bad to the rake. but you will not get beat.
A technical quibble here is that you can't just copy one of Cepheus' actions and consider it unexploitable. For the strategy to work EVERY decision you make has to be a duplicate of Cepheus', which includes taking actions with the correct randomly generated frequency when they aren't binary.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 06:06 PM
Quote:
Originally Posted by David Sklansky
Thank you. I only brought this up because the casino heads up machine has been known to make this mistake. I assume this occurs because even with trillions of hands dealt, very specific situations may come up only a few thousand times and in those few thousand the computer did very slightly better by folding even though its EV was very slightly better if it called. Or something along those lines. (Am I right?) But in the case of drawing to the dead nuts with one to come it seems that a simple instruction (though maybe not that simple because of the unknown two cards) could guarantee avoiding an embarrassing (as far as the press is concerned) error.

If I'm reading the CFR+ description correctly, it evaluates the entire game tree on every iteration, so it should learn to never fold the nuts on the end very quickly (after first iteration?). Folding (occasionally raising) hopeless hands on the end shouldn't take much longer. Once it learns a reasonable river strategy, the turn decision with the nut draw will follow since the algorithm evaluates the EV of every possible river card on every iteration.

So if I'm understanding it correctly, CFR+ is less susceptible to this type of error as Monte Carlo-based algorithms which may not sample some nodes frequently enough. Of course, if the decision is very close after accounting for implied odds, it might still get it wrong. But then it wouldn't be an obvious mistake.

nburch - Can you confirm? Also, congrats!
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 06:06 PM
Quote:
Originally Posted by fsn
Too bad there's a flaw with Cepheus...

http://www.theguardian.com/sport/201...t-cepheus-flaw

Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 06:16 PM
The Guardian is the new Daily Mail.

Also congrats!

Quote:
Originally Posted by nburch
It caps .06% of the time with 9T suited - most likely part of the noise that makes it "essentially" solved and not just solved.
If you mean that capping at all is noise:
Can't you solve the (ever so slightly) easier game where BTN is not allowed to cap, and when finding that the BTN's average gain is no lesser than the original gain, conclude that capping was useless?
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 06:16 PM
Quote:
Originally Posted by 14756897412369
lol wow it has to be a troll. no "poker writer" would be so poker illiterate to actually write that and seriously believe it. the guardian got trolled imo.
@ChrisKPHall: "Sadly I only got literally two hours to write, play and turn in the article and some parts got cut."

[x] Counterfactual regret minimization.
Computers Conquer Texas Hold'em Poker for First Time Quote
01-09-2015 , 06:25 PM
I'd like to see it try to play against a real quality human player. My money is on the computer. No tell.
Computers Conquer Texas Hold'em Poker for First Time Quote

      
m