Open Side Menu Go to the Top
Register
Heads Up Hold'em Solved? Heads Up Hold'em Solved?

10-14-2013 , 09:43 AM
Quote:
Originally Posted by joeschmoe
Does GTO follow the maxim, no risk, no gain?
If so, why would anyone adopt that as a poker strategy when the only objective is to win money?
GTO follows the maxim, don't lose (or rather, maximize expectation against a player that will exploit any mistake you make). Not losing doesn't necessarily imply breaking even. I think the GTO-breaks-even myth has to do with results from simple toy games (like rock-paper-scissors), where GTO can end up breaking even against a random opponent strategy. But real poker isn't like that. Random or clearly sub-optimal strategies get crushed by GTO.

For an example, look at HU LHE, where a GTO bot will open 80+% on the BTN and defend even wider in the BB (see the book The Intelligent Poker Player (Newall) for a discussion about GTO HU LHE bot strategy). Now pick a random opponent strategy to counter the bot. Since the opponent strategy is random, it will probably fold way too many BTN's and defend too little in the BB (otherwise it wouldn't be random), compared to the optimal strategy. As a result, it will get crushed by the GTO strategy because of preflop mistakes alone. And there are still tons of postflop mistakes a random strategy will make.

Last edited by ZenFish; 10-14-2013 at 09:58 AM.
Heads Up Hold'em Solved? Quote
10-14-2013 , 09:46 AM
Quote:
Originally Posted by joeschmoe
Does GTO follow the maxim, no risk, no gain?
If so, why would anyone adopt that as a poker strategy when the only objective is to win money?
If you guarantee yourself to lose the absolute minimum in every scenario, any mistakes your opponents make end up as profit for you. Nash guarantees in any two player zero sum game, that the worst possible scenario is that you break even. So if you consider a game like chess, if you played perfect nash for the whole game, do you see how powerful it would be that you can never lose, and only win if your opponent isn't as good as you are? It won't just try to break even though, if it sees a check mate is possible, it will take it, as long as there is no risk.

In poker (NLHE anyway) nash equilibrium is so complicated that everybody plays miles away from it. This means you would likely destroy any opponent no matter what he does and you don't even need to think about what he is doing. You also get the added benefit that you would have no leaks for your opponent to exploit on you and any adjustments he would try to make to exploit you would be completely wasted, and his only hope by doing that is to try and lose less money then he already is.

Nash in poker breaks down a little bit in poker due to its not technically a proper zero sum game because there is rake involved, and also 3+ players at the table players can collude or soft play (sometimes intentional but most likely accidental due to not being perfect as humans).
Heads Up Hold'em Solved? Quote
10-14-2013 , 10:00 AM
Quote:
Originally Posted by MouldyOnions
In poker (NLHE anyway) nash equilibrium is so complicated that everybody plays miles away from it. This means you would likely destroy any opponent no matter what he does and you don't even need to think about what he is doing.
I disagree with your conclusion. I think GTO vs most opponents would be a small winner. The better the opponent, the more superior a GTO strategy would be. Against fish an exploitative strategy is practically guaranteed to make more money.

The "RPS breakeven GTO" problem exists in some poker spots too. For example, suppose our river calling range is so balanced that the EV for our opponents bluffs is 0. Sure we are unexploitable, but we also stop our opponent from making mistakes on the river. If our opponent bluffs too much, we can exploit by calling very wide and sometimes raising as a bluff. If our opponent doesn't ever bluff in big spots, we are just leaking money every time we call.

An exploitative strategy will allow our opponent to make more mistakes that translate into EV for us.
Heads Up Hold'em Solved? Quote
10-14-2013 , 10:34 AM
I've read and am thinking about all the responses, but still can't get my head around the possibility that one can gain something with no risk of losing. It sounds so similar to the classical scam or con. No real life situations I can think of even offer this aspect of GTO.

Would GTO ever move all-in with less than the nuts? If so, it does take risks, and it can lose the game. And if it never does so, it would seem that opponents might easily exploit this "fear" of losing.
Heads Up Hold'em Solved? Quote
10-14-2013 , 10:38 AM
Quote:
Originally Posted by Adreno
I disagree with your conclusion. I think GTO vs most opponents would be a small winner. The better the opponent, the more superior a GTO strategy would be. Against fish an exploitative strategy is practically guaranteed to make more money.

The "RPS breakeven GTO" problem exists in some poker spots too. For example, suppose our river calling range is so balanced that the EV for our opponents bluffs is 0. Sure we are unexploitable, but we also stop our opponent from making mistakes on the river. If our opponent bluffs too much, we can exploit by calling very wide and sometimes raising as a bluff. If our opponent doesn't ever bluff in big spots, we are just leaking money every time we call.

An exploitative strategy will allow our opponent to make more mistakes that translate into EV for us.
At no point in my message did I say an unexploitable strategy is how to maximize your EV, so I am not entirely sure what you are disagreeing with. A nemesis strategy vs any opponent will always be a better choice then a nash strategy will, assuming you know what that nemesis strategy looks like. The problem there, is any strategy you use to exploit your opponent opens you up to become exploited.

Obviously if you are playing FishCake23 heads up, you should be playing a strategy that maximizes your profit. That strategy may involve also playing like a fish just to keep him playing.

Or even playing good opponents, everyone has some leaks you need to exploit. It is actually very important you exploit good opponents, because they will certainly exploit you.

I do however disagree with you when you say "I think GTO vs most opponents would be a small winner". If the best poker player in the world played a perfect GTO strategy bot at deep stacked poker, the player would get completely crushed for a very high win rate, in my opinion. You can feel free to disagree, but I am pretty confident there. You can try and play chess vs an estimated GTO bot, and see if it is only "a small winner" vs you.
Heads Up Hold'em Solved? Quote
10-14-2013 , 10:48 AM
Quote:
Originally Posted by joeschmoe
I've read and am thinking about all the responses, but still can't get my head around the possibility that one can gain something with no risk of losing. It sounds so similar to the classical scam or con. No real life situations I can think of even offer this aspect of GTO.

Would GTO ever move all-in with less than the nuts? If so, it does take risks, and it can lose the game. And if it never does so, it would seem that opponents might easily exploit this "fear" of losing.
Yes it absolutely would move all in with less then the nuts. It would be very well balanced in that respect between bluffing and shoving for value. Usually it will be in the general theme of betting your top hands for value, calling your middle strength hands, sometimes betting your weaker hands (as a bluff), and sometimes folding your weaker hands. That is a massive generalization there, but it should help to understand roughly how balancing works in GTO.

Don't worry about not being able to fully get your head around it. Many great poker players have also had the same issue. It is not a very intuitive thing to think about. There is an NVG thread somewhere with Hoss_TBF in the title (sorry I can't be more specific). This whole conversation we are about to have happened in there between Tom Dwan, Ike Haxton and a bunch of NVGers asking the same questions you are wondering now. It's a pretty good read overall and should help.
Heads Up Hold'em Solved? Quote
10-14-2013 , 11:07 AM
Now I think I do understand. GTO succeeds if allowed to operate over an infinite number of samples. Being a common mortal and possessed of a limited BR, I'm not especially attracted to it.

This is not to say GTO wouldn't be very useful in somewhat large samples such as certain online routines might offer, which I assume is the target.
Heads Up Hold'em Solved? Quote
10-14-2013 , 11:20 AM
Quote:
Originally Posted by joeschmoe
I've read and am thinking about all the responses, but still can't get my head around the possibility that one can gain something with no risk of losing. It sounds so similar to the classical scam or con. No real life situations I can think of even offer this aspect of GTO.

Would GTO ever move all-in with less than the nuts? If so, it does take risks, and it can lose the game. And if it never does so, it would seem that opponents might easily exploit this "fear" of losing.
A GTO strategy in poker is guaranteed to be >=0EV, not guaranteed to have only winning or break-even sessions. You're conflating the two.
Heads Up Hold'em Solved? Quote
10-14-2013 , 11:32 AM
Quote:
Originally Posted by Adreno
....

0:22 Hyperborean cc J2o on 8d8sJs, weak donk on turn 2, potdonk river J

This is funny in that it is exactly how many fish like to play J2. The line appears more "human-like" than I would expect. My issues with this hand from a GTO perspective:

Flop and turn sizings are small enough to allow the other person to call with drawing hands profitably. I would think the optimal sizing on flop & turn should be at least 1/2 pot, maybe 2/3 to 3/4 pot. Sizings are even more horrible if the bots use the same sizing for 8x, missing a ton of value from JX and giving really nice odds + implied odds for drawing hands. If that is not the case and the bots use a bigger sizing for 8x, then we have an issue of our range being capped with the small sizing and we are more vulnerable to bluffs.

...
Many of the recurring problems in your analysis stem from the fact that you're unaware of (or at least not acknowledging) the possibility of mixed strategies. Here, for example, the bots can partition their range into multiple sizings and apportion some mix of {nuts, good hands, draws, weak hands, air, etc.} among all, thus avoiding the problems you bring up.

Quote:
Originally Posted by Adreno
I also don't understand the need to "mix it up" with sizings with hands like K4o (sometimes 2.5BB sometimes 2BB). Preflop ranges are so wide anyway there's really no need for that. Find out which sizing is more optimal and use that 100% of the time.
Read the "optimal play" section in The Mathematics of Poker; it should clear up many of the things you don't understand, or at least give you a fundamental grounding from which to make better guesses.
Heads Up Hold'em Solved? Quote
10-14-2013 , 11:33 AM
Quote:
Originally Posted by joeschmoe
Now I think I do understand. GTO succeeds if allowed to operate over an infinite number of samples. Being a common mortal and possessed of a limited BR, I'm not especially attracted to it.

This is not to say GTO wouldn't be very useful in somewhat large samples such as certain online routines might offer, which I assume is the target.
You can say the same about any profitable strategy; it's only guaranteed to succeed in the long run. It really has nothing to do with sample size.

If you were to somehow be gifted with a GTO strategy, you'd be a fool not to use it because you'd be insta profiting in (almost) every single spot against every opponent. If you had the bankroll (or built it up using this strat) you could literally go sit against the 5 best hunl players and wipe the floor with them without having to put a second of thought into it.

I'd suggest reading the HSNL discussion about this (too lazy to find the link), or the HOSS NVG thread as someone mentioned. The key point is that GTO would figure out how to play so that at best, your opponent can break even vs you, and if they make any mistakes at all (which all humans do), they lose to you. How this edge manifests itself in terms of winnings is no different than to any other poker strategy: your theoretical winnings go up, and depending on whether variance is kind that day you win or lose.

Last edited by SmokeyQ123; 10-14-2013 at 11:40 AM.
Heads Up Hold'em Solved? Quote
10-14-2013 , 11:43 AM
Quote:
Originally Posted by SmokeyJ
You can say the same about any profitable strategy; it's only guaranteed to succeed in the long run. It really has nothing to do with sample size.

If you were to somehow be gifted with a GTO strategy, you'd be a fool not to use it because you'd be insta profiting in (almost) every single spot against every opponent. If you had the bankroll (or built it up using this strat) you could literally go sit against the 5 best hunl players and wipe the floor with them without having to put a second of thought into it.

I'd suggest reading the HSNL discussion about this (too lazy to find the link), or the HOSS NVG thread as someone mentioned. The key point is that GTO would figure out how to play so that at best, your opponent can break even vs you, and if they make any mistakes at all (which all humans do), they lose to you.
Aren't you overestimating its value a bit? MouldyOnions just said GTO will move in without the nuts. That alone exposes you to consecutive losses or a tight group of very substantial losses. Practically speaking a person could go broke before things have a chance to even out.

...and to me, "guaranteed to succeed in the long run" directly implies a large sample size.

I might do some reading, including that thread. There was also a long, NVG thread iirc.
Heads Up Hold'em Solved? Quote
10-14-2013 , 11:49 AM
Quote:
Originally Posted by joeschmoe
Aren't you overestimating its value a bit? MouldyOnions just said GTO will move in without the nuts. That alone exposes you to consecutive losses or a tight group of very substantial losses. Practically speaking a person could go broke before things have a chance to even out.

...and to me, "guaranteed to succeed in the long run" directly implies a large sample size.
.....this is what bankroll management is for! Any possible strategy could send you broke if aren't playing with enough buyins. What are you trying to get at?

If you wouldn't be tempted by an unbeatable strategy because it can expose you to consecutive losses, then you shouldn't be playing poker in the first place.
Heads Up Hold'em Solved? Quote
10-14-2013 , 12:00 PM
My problem with strict adherence to any strategy would be that it is inflexible.

I cannot know how many buyins will be "enough" so I shouldn't even start without a bottomless BR. Excellent br management might do nothing more than prolong the pain...

Look.. I'm not against GTO. It is a very amazing thing. I just don't see how it might be especially useful to me.

The tiny amount of reading I've done so far suggests "game theory" is not about playing games with an intent to win them. Instead it is about trying to accurately describe how games are played.

Last edited by joeschmoe; 10-14-2013 at 12:11 PM.
Heads Up Hold'em Solved? Quote
10-14-2013 , 12:12 PM
You would need a significantly smaller bankroll if you were to play perfect GTO. Your maximum downswing would in all likeliness be a fraction of what it is now. Also, with regards to not knowing what is enough of a bankroll... If you knew your exact strategy and how good it is compared to every player you play against, it would be significantly easier to figure out how much you would need. How do you figure out how much you would need right now? Surely you have the exact same issue you just spoke about, but with even more uncertainty.
Heads Up Hold'em Solved? Quote
10-14-2013 , 12:22 PM
This might be asking too much, but can someone fill in the blanks when they say "With X amount of a roll against a certain quality of player, over a certain amount of samples, you are guaranteed to win."
I am asking for a practical example of GTO in action.

got lots of errands to run.. no hurry.
Heads Up Hold'em Solved? Quote
10-14-2013 , 12:36 PM
Sick troll or incredibly stupid. GG joeschmoe, as usual you got quite a few people to waste time on you.
Heads Up Hold'em Solved? Quote
10-14-2013 , 12:51 PM
Quote:
Originally Posted by joeschmoe
This might be asking too much, but can someone fill in the blanks when they say "With X amount of a roll against a certain quality of player, over a certain amount of samples, you are guaranteed to win."
I am asking for a practical example of GTO in action.

got lots of errands to run.. no hurry.
Imagine for a second that you're a complete and perfect poker machine. If a person makes a single mistake you know how to and do instantly exploit it. Now for some crazy reason I decide to tell you exactly how I'm going to play every single hand and situation I could ever end up in online poker. You give me an exhaustive list of places you'd exploit my play at. I correct those leaks, potentially introducing more in doing so, and we repeat the process. This continues on until we finally reach a point where I tell you exactly how I'm going to play and ask you how you'd exploit me - try as you might, you can no longer find a single exploitable spot in my strategy. This strategy would be considered GTO.

I'm never going to change my strategy regardless of what you do so it's going to look a bit strange at times. For instance I'm going to be perfectly balanced for you deciding to open shove 200bb. Yet my range for calling isn't going to change even if you then decide to open shove the next 50 hands in a row as that would open me up to exploitation since if I you discover I'm going to adjust once you open shove 5 times in a row - then you can revert to a different strategy on the 6th hand and suddenly you're exploiting me.

A lot of my lines are going to look very strange. My betsizes will stupefy. Ironically I'll probably be able to get all the action I could ever want, because no good human would ever play this way. That's no problem though - I'm no human.
Heads Up Hold'em Solved? Quote
10-14-2013 , 02:03 PM
Quote:
In summary, both bots appeared super unbalanced and very exploitable. They sometimes do random stuff that doesn't appear to make sense and they mimick human fish in other ways too. Not convinced of the end of poker.
From what I've heard their postflop strategies are exploitable for a ****load of money (more than 20bb/100). I hope authors of those could clarify it because I only heard it from 2nd hand.
When you look at strategies very close to 0 exploitability (in games without many bet size options but without any card abstraction) they suddenly make perfect sense and there are 0 plays which you would identify as spews.
Heads Up Hold'em Solved? Quote
10-14-2013 , 02:03 PM
Quote:
Originally Posted by SmokeyJ
Sick troll or incredibly stupid. GG joeschmoe, as usual you got quite a few people to waste time on you.
.
Heads Up Hold'em Solved? Quote
10-14-2013 , 02:42 PM
Quote:
Originally Posted by MouldyOnions
(..)
There is an NVG thread somewhere with Hoss_TBF in the title (sorry I can't be more specific). This whole conversation we are about to have happened in there between Tom Dwan, Ike Haxton and a bunch of NVGers asking the same questions you are wondering now. It's a pretty good read overall and should help.
said thread

Quote:
Originally Posted by joeschmoe

I am asking for a practical example of GTO in action.
here you has it, nash push-fold chart for heads-up play. you won't obviously neither win every hand nor any match, and there are opponents you are better off playing an exploitative (i.e. exploitable) strategy, but you cannot lose longterm.

Last edited by franxic; 10-14-2013 at 03:06 PM.
Heads Up Hold'em Solved? Quote
10-14-2013 , 03:11 PM
Great thread!

A question for FullyCompletely:

Say you were to make the breakthroughs needed to possibly eclipse the top holdem players of today within a couple of years. What do you think would happen to the bot? How is it protected?

Since such a bot's strategy would beat any game it's obviously capable of making a tremendous amount of money, and surely it would be nothing but human for people working on the bot to sell it once the right offers start to trickle into the team.

Basically, how worried should we be about the effects this will have on the online games?

Thanks in advance.
Heads Up Hold'em Solved? Quote
10-14-2013 , 03:21 PM
Quote:
Originally Posted by franxic
here you has it, nash push-fold chart for heads-up play. you won't obviously neither win every hand nor any match, and there are opponents you are better off playing an exploitative (i.e. exploitable) strategy, but you cannot lose longterm.
I hate to be "that guy", but the chart you are referring to would do pretty badly in the real world. Even on small stack sizes playing shove or fold is not the optimal line to take. So it isn't the best of examples.

Also this is a little nit picky, but I will mention it anyway. even if the game you played specifically only allowed you to play shove or fold, that chart still isn't perfect GTO. This is because depending on your stack sizes and whether you are the SB or BB, your expected value on the hand changes. This means a perfect GTO strategy would need to take future hands into account, and that chart does not.
Heads Up Hold'em Solved? Quote
10-14-2013 , 03:46 PM
In the game the chart refers to the only moves are shove and fold. In that game if you follow that chart you cannot lose long term. That is a pretty decent practical example of GTO, which is what was asked for.
Heads Up Hold'em Solved? Quote
10-14-2013 , 04:14 PM
It looks like this kinda derailed while I was away over the (Canadian) Thanskgiving holiday, and I need to get back to focussing on writing my PhD thesis: I have a deadline to finish a draft by the end of the month if I'm going to be able to graduate in January. This will probably be my last post for a while until I can wrap that up. Thanks to everyone for the great questions. This was fun, and that made it way too easy to avoid writing long thesis chapters by instead spending my time here writing long posts. :^D

GraceForDrowning: I don't think there's much danger of our bots leaking. We know the new grad students joining our group very well by the time they get access to the codebase or the bots, and we wouldn't give access to anyone where we suspected that'd be a danger. Every time we've ever had a bot on a play money site (where it's always clearly marked as one of our bots) like the old Poker Academy Online site (which closed down a while ago), we've never sent out a copy of the bot; instead, we connected from our end. Because of the potential for a man-in-the-middle attack, we're careful about only cooperating with play money sites we trust, too. A lot of the Poker Academy Online people had previously been part of our team, which is why we partnered up with them in the past.

To be honest, the codebase or the bots isn't nearly as valuable as the experience or knowledge that our people have. But so far, everyone that's graduated from our group over the last decade or so has either stayed in academia, gone to work for an online casino, or gone to other high-paying software industry jobs. I don't think anyone would have been tempted by botting or selling a copy.
Heads Up Hold'em Solved? Quote
10-14-2013 , 04:17 PM
The big question is : why the bot has a 23x opening range !!!! (w/ AK,69o) (cf the other thread)

1. we are all ******ed and this is optimal

2. It's the sampling order in the MonteCarlo CFR (lot of 69x flops in the begining of the training)

3 ???

4. The convergence is not finished, the CFR has not run long enough


PokerStars Hand #1743273479442: Hold'em No Limit ($0.50/$1 USD) - 2013/07/17 15:13:24 ET
Table 'table' 9-max Seat #2 is the button
Seat 1: hyperborean_iro ($200 in chips)
Seat 2: slumbot ($200 in chips)
slumbot: posts small blind $0.50
hyperborean_iro: posts big blind $1
*** HOLE CARDS ***
slumbot: calls $0.50
hyperborean_iro: raises $22 to $23
slumbot: folds
Uncalled bet ($22) returned to hyperborean_iro
hyperborean_iro collected $2 from pot
*** SUMMARY ***
Total pot $2 | Rake $0
Seat 1: hyperborean_iro mucked [9h 6s]
Seat 2: slumbot mucked [2h 8h]


PokerStars Hand #1743273479666: Hold'em No Limit ($0.50/$1 USD) - 2013/07/17 15:13:24 ET
Table 'table' 9-max Seat #2 is the button
Seat 1: hyperborean_iro ($200 in chips)
Seat 2: slumbot ($200 in chips)
slumbot: posts small blind $0.50
hyperborean_iro: posts big blind $1
*** HOLE CARDS ***
slumbot: calls $0.50
hyperborean_iro: raises $22 to $23
slumbot: folds
Uncalled bet ($22) returned to hyperborean_iro
hyperborean_iro collected $2 from pot
*** SUMMARY ***
Total pot $2 | Rake $0
Seat 1: hyperborean_iro mucked [6d 9s]
Seat 2: slumbot mucked [5c 9c]

PokerStars Hand #174327347919964: Hold'em No Limit ($0.50/$1 USD) - 2013/07/17 15:13:24 ET
Table 'table' 9-max Seat #2 is the button
Seat 1: hyperborean_iro ($200 in chips)
Seat 2: slumbot ($200 in chips)
slumbot: posts small blind $0.50
hyperborean_iro: posts big blind $1
*** HOLE CARDS ***
slumbot: calls $0.50
hyperborean_iro: raises $22 to $23
slumbot: folds
Uncalled bet ($22) returned to hyperborean_iro
hyperborean_iro collected $2 from pot
*** SUMMARY ***
Total pot $2 | Rake $0
Seat 1: hyperborean_iro mucked [6d 9s]
Seat 2: slumbot mucked [9c 5s]

PokerStars Hand #174327347920876: Hold'em No Limit ($0.50/$1 USD) - 2013/07/17 15:13:24 ET
Table 'table' 9-max Seat #2 is the button
Seat 1: hyperborean_iro ($200 in chips)
Seat 2: slumbot ($200 in chips)
slumbot: posts small blind $0.50
hyperborean_iro: posts big blind $1
*** HOLE CARDS ***
slumbot: calls $0.50
hyperborean_iro: raises $22 to $23
slumbot: folds
Uncalled bet ($22) returned to hyperborean_iro
hyperborean_iro collected $2 from pot
*** SUMMARY ***
Total pot $2 | Rake $0
Seat 1: hyperborean_iro mucked [9h 6s]
Seat 2: slumbot mucked [4c Kc]

PokerStars Hand #174327347926554: Hold'em No Limit ($0.50/$1 USD) - 2013/07/17 15:13:24 ET
Table 'table' 9-max Seat #2 is the button
Seat 1: hyperborean_iro ($200 in chips)
Seat 2: slumbot ($200 in chips)
slumbot: posts small blind $0.50
hyperborean_iro: posts big blind $1
*** HOLE CARDS ***
slumbot: calls $0.50
hyperborean_iro: raises $22 to $23
slumbot: folds
Uncalled bet ($22) returned to hyperborean_iro
hyperborean_iro collected $2 from pot
*** SUMMARY ***
Total pot $2 | Rake $0
Seat 1: hyperborean_iro mucked [6d 9s]
Seat 2: slumbot mucked [6c 5h]

Last edited by ohsosick; 10-14-2013 at 04:25 PM.
Heads Up Hold'em Solved? Quote

      
m