Two Plus Two Publishing LLC
Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

Notices

News, Views, and Gossip For poker news, views, and gossip

Reply
 
Thread Tools Display Modes
Old 01-10-2015, 10:48 PM   #251
ezdonkey
veteran
 
ezdonkey's Avatar
 
Join Date: Jun 2006
Location: MAKING STEEL
Posts: 2,955
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by David Sklansky View Post
Make the river bet 3 BB instead of 2 and they have to deal another 20 trillion hands.
Hmm, I don't understand why does changing river sizing would make the game tree any bigger, hence harder to calculate? Every street would still be capped at 4 bets and both players start with 8 small bets to cover pre+flop, 4 turn bets and 4 river bets = 24 small bets total. (Normal HU LHE game has 12 small bet cap.)

Or did you mean that players can opt to bet 2 or 3 on the river? However I don't know how that would work in general? Can you raise 2-lead on river with 3?

EDIT: Just remembered this topic: https://forumserver.twoplustwo.com/10...enario-869820/
Sounds like 20 trillion isn't really even a big increase fraction-wise to the original number?

Quote:
Put a preflop flop out where those five cards are dead and they have to deal more than 50 quintillion.
Rephrase please? Didn't quite catch your drift.

Last edited by ezdonkey; 01-10-2015 at 11:01 PM.
ezdonkey is offline   Reply With Quote
Old 01-10-2015, 11:01 PM   #252
TheBryce2
stranger
 
Join Date: Jan 2015
Location: Forgot my PW.
Posts: 14
Re: Computers Conquer Texas Hold'em Poker for First Time

I believe he means that if five cards were burned face up before the game then the game would have to be recomputed and the size would explode.
TheBryce2 is offline   Reply With Quote
Old 01-10-2015, 11:07 PM   #253
ezdonkey
veteran
 
ezdonkey's Avatar
 
Join Date: Jun 2006
Location: MAKING STEEL
Posts: 2,955
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by TheBryce2 View Post
I believe he means that if five cards were burned face up before the game then the game would have to be recomputed and the size would explode.
That actually sounds like a pretty cool game. I think even 3 would be good for starters. Dead Flop Hold'em.
ezdonkey is offline   Reply With Quote
Old 01-10-2015, 11:18 PM   #254
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,671
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by ezdonkey View Post
Hmm, I don't understand why does changing river sizing would make the game tree any bigger, hence harder to calculate? Every street would still be capped at 4 bets and both players start with 8 small bets to cover pre+flop, 4 turn bets and 4 river bets = 24 small bets total. (Normal HU LHE game has 12 small bet cap.)

Or did you mean that players can opt to bet 2 or 3 on the river? However I don't know how that would work in general? Can you raise 2-lead on river with 3?

EDIT: Just remembered this topic: https://forumserver.twoplustwo.com/10...enario-869820/
Sounds like 20 trillion isn't really even a big increase fraction-wise to the original number?


Rephrase please? Didn't quite catch your drift.
They have to start from scratch with different limits.

With a preflop flop they have 2.6 million strategies to figure out.
David Sklansky is offline   Reply With Quote
Old 01-10-2015, 11:25 PM   #255
ambientsilence
journeyman
 
Join Date: Oct 2014
Posts: 391
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by David Sklansky View Post
They have to start from scratch with different limits.

With a preflop flop they have 2.6 million strategies to figure out.
So how many decision points? I think someone mentioned 400 quads, so a factor of 1000 increase? Doesn't seem too far fetched with improved hardware and perhaps some tweaking of the algorithm. They would be finished before you could get a decent amount of people to start playing the game.

I think if you're going down this route, you should find a game that's definitely out of reach.

Even then the cat's out of the bag: the best way to compete against the most skilled players HU is by building smart AI and copying its strategy as much as possible.
ambientsilence is offline   Reply With Quote
Old 01-10-2015, 11:44 PM   #256
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by ezdonkey View Post
So I'm basically querying a supercomputer, no? A one which regular people don't have access to? Could someone grab the source codes and be able to build a similar system which can query the database almost real-time with a moderate investment? (Moderate in this case something in the range of 10-50k?)
Nah, no supercomputer for the query tool. It's running on a more-or-less standard desktop PC stuffed with a few extra drives. It's generating the strategy that takes a lot of computing resources - that wouldn't be a modest investment. The query tool is slow mostly because the strategy is compressed, and it has to read a fairly big chunk off disk to answer anything.
nburch is offline   Reply With Quote
Old 01-10-2015, 11:58 PM   #257
my_nameaintearl
adept
 
my_nameaintearl's Avatar
 
Join Date: Jul 2014
Posts: 751
Re: Computers Conquer Texas Hold'em Poker for First Time

Here is a some code from a file cepheus.js I haven't really read through it
Spoiler:
my_nameaintearl is offline   Reply With Quote
Old 01-11-2015, 12:09 AM   #258
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by ambientsilence View Post
So how many decision points? I think someone mentioned 400 quads, so a factor of 1000 increase? Doesn't seem too far fetched with improved hardware and perhaps some tweaking of the algorithm. They would be finished before you could get a decent amount of people to start playing the game.
If you sort the cards, and then ignore things that are same by re-labeling suits, there are 1755 different flops. If you publicly burned the cards before any actions, you would need to solve 1755 _independent_ games. Each of the games would be just a tiny bit bigger than HU LHE, because those burned cards mean you wouldn't get quite as much out of that same suit-relabeling in the rest of the game. Maybe 2000 times larger overall, and you don't actually have to solve the whole thing all at once, you can do it in 1755 separate pieces.

Quote:
I think if you're going down this route, you should find a game that's definitely out of reach.
If I haven't gotten that ~2000 number wrong, it's probably not a big enough jump.

But "kill the game" sort of feels like the wrong answer to preventing someone from cheating, especially for competitive play rather than open tables online :/
(not that I have a fantastic solution to offer...)
nburch is offline   Reply With Quote
Old 01-11-2015, 12:11 AM   #259
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by my_nameaintearl View Post
Here is a some code from a file cepheus.js I haven't really read through it
You're looking at the user interface code on the web page. There's not much poker knowledge to be gained there...
nburch is offline   Reply With Quote
Old 01-11-2015, 12:12 AM   #260
Spy_Lord
enthusiast
 
Join Date: Aug 2007
Posts: 90
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by nburch View Post
I'll happily go on record as saying heads-up NLHE 100b won't be solved in anyone's lifetime. Adding more players won't make that problem easier...

It's an interesting question of how small the stacks would need to be. We could certainly solve a 2 chip game. I need more free time :/
First of all, amazing work - both the latest publication and all your contributions to imperfect-information games over the years.

A question about some general stuff. Since it is one of possibly many (possibly infinite) Nash equilibria strategies, we would expect to see different convergence points if you ran the iterations many times. Are there any plans to run it again to see how different the next optimal play is? Obviously this may be a big use of resources for little in the way of scientific interest, but it would be very interesting from a poker perspective. Also, if you were to run the process and generate many different optimal strategies, it would be super interesting to see what the equilibria landscape looks like.

In this post you state HUNLHE won't be solved in our lifetimes. Are you sure about this? I've read somewhere that HUNLHE with 200bb stacks has something like 10^40 decision points. In the past 9 years you stated (and you have a chart which shows) that we have gone from solving order 10^6 games to HULHE at 10^14 states - so roughly increasing the exponent by 1.33 every year. At that rate of progress, I make it 19.5 years to hit 10^40. So I would estimate 20 years, give or take, before HUNLHE is a solved game.

Final question: are you accepting PhD students, and do you take people from a maths/machine learning background with no economics knowledge?
Spy_Lord is offline   Reply With Quote
Old 01-11-2015, 12:23 AM   #261
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,671
Re: Computers Conquer Texas Hold'em Poker for First Time

When does the speed of light start coming into play?
David Sklansky is offline   Reply With Quote
Old 01-11-2015, 01:30 AM   #262
ezdonkey
veteran
 
ezdonkey's Avatar
 
Join Date: Jun 2006
Location: MAKING STEEL
Posts: 2,955
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by David Sklansky View Post
When does the speed of light start coming into play?
Lay off da weed bro
ezdonkey is offline   Reply With Quote
Old 01-11-2015, 02:10 AM   #263
nburch
newbie
 
Join Date: Jan 2015
Posts: 47
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by Spy_Lord View Post
A question about some general stuff. Since it is one of possibly many (possibly infinite) Nash equilibria strategies, we would expect to see different convergence points if you ran the iterations many times. Are there any plans to run it again to see how different the next optimal play is? Obviously this may be a big use of resources for little in the way of scientific interest, but it would be very interesting from a poker perspective. Also, if you were to run the process and generate many different optimal strategies, it would be super interesting to see what the equilibria landscape looks like.
Yikes. No plans to run another HU LHE strategy. The project took a special resource request to the body that handles Canadian research computing resources. I don't know how we would write that up in a way that would make them want to give us those resources again for that...

It's an interesting question. We've tried looking at this in smaller games, just to see how variable the strategies are. There are definitely strategies that have different actions in a bunch of places, but looking at a bunch of randomly selected strategies has mostly just resulted in an unusable big list of changes in an even bigger sea of numbers. Even answering the question "how different are these strategies" seems to be hard to do well in a way that satisfies human intuition.

Quote:
In this post you state HUNLHE won't be solved in our lifetimes. Are you sure about this? I've read somewhere that HUNLHE with 200bb stacks has something like 10^40 decision points. In the past 9 years you stated (and you have a chart which shows) that we have gone from solving order 10^6 games to HULHE at 10^14 states - so roughly increasing the exponent by 1.33 every year. At that rate of progress, I make it 19.5 years to hit 10^40. So I would estimate 20 years, give or take, before HUNLHE is a solved game.
"Extrapolation bad!" Or so some people tell me

FullyCompletely lists a bunch of sizes in http://webdocs.cs.ualberta.ca/~games...rt-nl-size.pdf -- if you only care about the sizes, you can find them starting on page 10 for a few different HU NLHE games. You're looking for the one sided canonical, total values. The "Infosets" column tells you how many place you'll need to make a decision, and the "Actions" column tells you how many different numbers you'll need in a strategy.

So, for a 200bb game with 1 chip / 2 chip blinds, a strategy has about 1.8*10^47 values. That's a big table, and I don't see hard drives growing that quickly. That's out.

We now have safe methods for throwing away parts of the strategy, and then recovering them as needed, but they've got a big computation cost. Let's ignore that computation cost (bigger than HU LHE in this case) and split the game in half. Best case, for a balanced binary tree, we use the square root of the original space -- 4.2*10^23 values. Lets say we get the same great compression, and use less than a single bit per value -- about 5 * 10^22 bytes. That's still 10 billion TB. I guess that's a "maybe in my lifetime" amount, but it still leaves looking up an answering being a computation 10 billion times larger than HU LHE. So, that's out.

That leaves something unexpected, but it's a pretty surprising unexpected something. 10^47 is pretty big even for a perfect information game.

Quote:
Final question: are you accepting PhD students, and do you take people from a maths/machine learning background with no economics knowledge?
I'm a PhD student, and this is probably wildly off topic, more appropriate in an email to Mike Bowling.
nburch is offline   Reply With Quote
Old 01-11-2015, 02:43 AM   #264
chinagambler
veteran
 
chinagambler's Avatar
 
Join Date: Jun 2011
Location: Czech Republic
Posts: 2,562
Re: Computers Conquer Texas Hold'em Poker for First Time

Just out of curiosity, how long would it take to solve 20bb or 25bb deep HUNLHE?
chinagambler is offline   Reply With Quote
Old 01-11-2015, 03:38 AM   #265
RussianRoulette
newbie
 
Join Date: Dec 2014
Posts: 45
Re: Computers Conquer Texas Hold'em Poker for First Time

Surely if bet sizing were continuous, there would be a way to converge towards a GTO strategy without considering an infinite amount of bet sizes?
RussianRoulette is offline   Reply With Quote
Old 01-11-2015, 04:15 AM   #266
PLBlow
grinder
 
Join Date: Sep 2011
Posts: 442
Re: Computers Conquer Texas Hold'em Poker for First Time

backgammon has been solved for some time now, yet theres still plenty of (big) action to be had

the game (or other games) will not die out as a result of it being solved
PLBlow is offline   Reply With Quote
Old 01-11-2015, 04:27 AM   #267
12may2012
newbie
 
Join Date: May 2012
Posts: 49
Re: Computers Conquer Texas Hold'em Poker for First Time

to breakeven against a gto bot, doesnt we just have to go to the river everytime and bet allin?
12may2012 is offline   Reply With Quote
Old 01-11-2015, 04:37 AM   #268
chinagambler
veteran
 
chinagambler's Avatar
 
Join Date: Jun 2011
Location: Czech Republic
Posts: 2,562
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by 12may2012 View Post
to breakeven against a gto bot, doesnt we just have to go to the river everytime and bet allin?
I think we've just solved NLHE guys, time to quit playing.
chinagambler is offline   Reply With Quote
Old 01-11-2015, 07:30 AM   #269
Loctus
Carpal \'Tunnel
 
Loctus's Avatar
 
Join Date: Nov 2008
Posts: 10,031
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by PLBlow View Post
backgammon has been solved for some time now, yet theres still plenty of (big) action to be had

the game (or other games) will not die out as a result of it being solved
Is there big ONLINE action to be had in backgammon?
Loctus is offline   Reply With Quote
Old 01-11-2015, 12:08 PM   #270
Donkem
journeyman
 
Donkem's Avatar
 
Join Date: Feb 2014
Location: Close to donks
Posts: 359
Re: Computers Conquer Texas Hold'em Poker for First Time

People should stop mentioning GTO as it is supposedly refering to a possibly non exploitative strategy. This bot clearly uses the knowledge from past hands to decide how to play future hands, so it has an exploitative strategy.

What they claim here is that this bot applies a perfect exploitative strategy, or set of strategies, that exploits the bet/raise/call/check/fold ranges on every street, neglecting human tells {chat dialog, clicking speed}.

Basically they are claiming that Cepheus is the mathematical Nemesis for HU FLHE.
Donkem is offline   Reply With Quote
Old 01-11-2015, 01:13 PM   #271
bjsmith22
veteran
 
bjsmith22's Avatar
 
Join Date: Sep 2011
Location: Official PokerStars Shill Account
Posts: 2,235
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Originally Posted by Donkem View Post
People should stop mentioning GTO as it is supposedly refering to a possibly non exploitative strategy. This bot clearly uses the knowledge from past hands to decide how to play future hands, so it has an exploitative strategy.

What they claim here is that this bot applies a perfect exploitative strategy, or set of strategies, that exploits the bet/raise/call/check/fold ranges on every street, neglecting human tells {chat dialog, clicking speed}.

Basically they are claiming that Cepheus is the mathematical Nemesis for HU FLHE.
Yeah, I think you're onto something here.

What would a game theorist know about a game theory optimal strategy right?

Everyone, let's turn our attention to donkem here instead of listening to what these crackpot 'scientists' from the 'university' of 'alberta' have to say
bjsmith22 is offline   Reply With Quote
Old 01-11-2015, 01:39 PM   #272
punter11235
Carpal \'Tunnel
 
Join Date: Mar 2005
Location: solving poker
Posts: 8,006
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Just out of curiosity, how long would it take to solve 20bb or 25bb deep HUNLHE?
With one bet size after the flop (like half pot flop/turn, shove river) - about a week or two on your standard home quad CPU.
If you allow more bet sizes it depends what granularity you want to have. CFR is not really well suited for NL. You can use it to solve games with many bet sizes but you lose a lot of potentially useful information so it's "dumb brute force" approach (nevertheless it's feasible if you have a lot of hardware available).

Quote:
to breakeven against a gto bot, doesnt we just have to go to the river everytime and bet allin?
This is a good question. The thing is that Holdem is similar to rock paper scisors (when this approach work) but not quite the same - in poker there are objectively good and bad plays. Objectively good and bad bluffing hands and objectively good and bad bluff catchers. While your bluff every river approach will often work against a GTO it won't work often enough (because it works often enough with some hands - the ones containing blockers to calling range of your opponent and not often enough with random spews).
There are objectively good betting hands on the flop and objectively good calling hands so getting to the river in the first place without losing a lot of EV is not easy either.

Last edited by punter11235; 01-11-2015 at 01:45 PM.
punter11235 is offline   Reply With Quote
Old 01-11-2015, 01:46 PM   #273
PlsFold
grinder
 
Join Date: Nov 2009
Posts: 431
Re: Computers Conquer Texas Hold'em Poker for First Time

What a great thread. Congrats to team cepheus with this amazing achievement.
PlsFold is offline   Reply With Quote
Old 01-11-2015, 01:58 PM   #274
punter11235
Carpal \'Tunnel
 
Join Date: Mar 2005
Location: solving poker
Posts: 8,006
Re: Computers Conquer Texas Hold'em Poker for First Time

Quote:
Make the river bet 3 BB instead of 2 and they have to deal another 20 trillion hands.

Put a preflop flop out where those five cards are dead and they have to deal more than 50 quintillion.
I really like this idea for making holdem more resistant to solving. The thing is a lot of NL based games will fall in coming years (cap HU will be first, all the HU sngs second, then 50-100bb will fall as well, probably not to 0.001BB/hand but even something like 0.1BB/hand makes it unplayable with rake).
If you deal 5 dead cards before every hand and put them face-up on the table, then the searching space again will be way too big to attempt to solve the whole thing (at least for a while)
punter11235 is offline   Reply With Quote
Old 01-11-2015, 01:58 PM   #275
watergun7
veteran
 
Join Date: Jun 2012
Posts: 2,767
Re: Computers Conquer Texas Hold'em Poker for First Time

Going all in on the river against a gto bot is never that bad with a hand that has no showdown value tbh (unless it's spot where gto says to check your entire range, or to a lesser extent- when you should never have a shoving range).

Bluffing with hands with bad card removal can only be a small mistake when you consider how rare the spot is- you have to arrive at some river etc on a specific board run out.

NL holdem is much more complex than limit and I doubt it will ever be strongly solved. People already mentioned an infinite number of GTO lines you can take in limit (although it could just be noise with the limping preflop etc), but I'm pretty confident that there's going to be a crazy amount of GTO betsizing/ranges in NL.

There are just many spots where it seems to me impossible for there to be exactly 1 proper betsizing/range splitting (ofc wet boards prob means many hands will be indifferent, but even something like a dry AAx board in a 3bet pot I'm sure you can have a normal size cbet and a checking range, or you could go for very small cbets with a wide range since you are oop and I don't see how it one can be vastly superior to the other).
watergun7 is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 02:20 AM.


Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright 2008-2020, Two Plus Two Interactive
 
 
Poker Players - Streaming Live Online