Two Plus Two Publishing LLC Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > > >

Notices

Stoxpoker.com Sponsored by StoxPoker.com

This forum is now read only. Please visit the new CardRunners / Stoxpoker forum!

 
 
Thread Tools Display Modes
Old 07-09-2008, 02:01 AM   #1
The Bryce
Stoxpoker Sponsored Forum
 
The Bryce's Avatar
 
Join Date: Oct 2005
Location: Edmonton
Posts: 4,245
Understanding Game Theory and Hold'em

I tend to get a few PMs about this article per month. So due to popular demand I'll be temporarily hosting the article here until we're able to get it up on Stoxpoker.


---------


Understanding Game Theory and Hold’em, by Bryce Paradis and Douglas Zare

Game theory has become a popular, if somewhat misunderstood, topic for hold’em discussion. This article is intended to give you a fundamental understanding of what game theory optimal strategy is, how it works, and what its impact is on hold’em play. Before we begin on the article proper, however, we will start by reviewing some key definitions. These definitions are not necessarily the same as those used by all others.

Optimal Exploitive Strategy:
A strategy which yields the highest possible EV against your opponent’s strategy. For example, if in a game of rock-paper-scissors your opponent’s strategy is to choose rock every single time your optimal exploitive strategy is to pick paper every single time. The same is true if your opponent’s strategy is rock 50%, paper 25%, and scissors 25%.

Suboptimal Strategy: A strategy which performs worse than an optimal exploitive strategy. For example, if your opponent’s strategy is to choose rock every single time, choosing paper 50% and rock 50% is still a winning strategy. The EV of the paper-and-rock strategy, however, is less than that of the paper-only strategy. Therefore the paper-and-rock strategy is suboptimal.

Game Theory Optimal (GTO): A strategy that yields the highest possible EV (or: “is optimal”) if your opponent always chooses the best possible counter-strategy. In a game of rock-paper-scissors the GTO strategy is to choose randomly from an equal distribution of paper, scissors, and rocks. If you play rock less often than paper, you will have less than ½ equity against an all scissors strategy. Similarly, you must play paper at least as often as you play scissors, and scissors at least as often as you play rock. As a result, you must play paper, scissors, and rocks with equal frequency to guarantee ½ equity against all strategies. So long as your opponent always chooses the optimal counter-strategy to whatever strategy you choose no strategy on your part can have a higher EV than this.

Exploitive Strategy: Any strategy which has a higher EV than GTO strategy against a particular opponent.

Exploitable Strategy: A strategy which has less EV against some exploitive strategies than GTO strategy. All non-GTO strategies are exploitable.

When analyzing optimal, exploitive strategies, we treat an opponent’s strategy as a known. For example: “my opponent always chooses rock.” In reality, our opponent’s strategy is an unknown, and we often act on assumptions and observations in order to determine what we will treat our opponent’s strategy as. To determine a GTO strategy, we assume that our opponent always chooses the optimally exploitive counter to whichever strategy we try, rather than playing a fixed strategy.

Hold’em is a much more complicated game than rock-paper-scissors, and until the game is solved by computers no one will ever play against an opponent who always chooses a GTO (or: “unexploitable”) strategy. This is an important point, as a GTO strategy is not necessarily the strategy with the highest possible EV. For example, if our opponent’s strategy is rock-only then the GTO strategy of choosing randomly from an equal distribution of paper, scissors, and rocks has less EV than that of the paper-only strategy.

GTO play, however, still plays an important role in hold’em strategy. Even though a GTO strategy may have less EV an exploitive strategy, understanding what the GTO strategy is and being able to identify how our opponents’ strategy deviates from it can help you to better exploit your opponents. Further, understanding GTO strategy can also allow to be able to create balanced strategies which are difficult to exploit. These strategies can be used as a defense against tough opponents looking for an exploitive edge.

In hold’em, as in many simple games such as rock-paper-scissors, a GTO strategy is often identifiable by finding an indifference point. What this means is that the GTO strategy will often distribute your actions in such a way that your opponent is indifferent to choosing between two actions. As a result your strategy is unexploitable.

Although hold’em has not been solved, many half-street and full-street mini-games which model real hold’em situations have been solved. By understanding where the indifference points lie in different hold’em scenarios, you can identify your opponent’s deviations from GTO play and exploit your opponent maximally. At its most basic conceptual level hold’em is still a very simply game: rather than playing with a distribution of paper, scissors and rocks we play with a distribution of bluffs and not-bluffs. By understanding even just the simplest mini-games you can greatly improve your play.

A common example of a half-street game would be one where we either hold hands that always win, or always lose if we see a showdown, and can either bet or check, and our opponent may only call or fold. If he calls, there is a showdown. This is often analogous to a river-betting scenario in real hold’em play where our opponent’s range is narrow and ours is polarized. By solving the mini-game we can see that the GTO strategy is to bluff an amount proportionate to the price we are laying our opponent on his call. For example, if we bet $1 into a $2 pot we are laying 3:1 by betting, and the GTO strategy is to bluff 25% of the time that we bet. Our opponent will be indifferent to calling or folding. As a result, we know that if we deviate from this strategy our opponent can exploit us by either always calling if we bluff more, or always folding if we bluff less.

Conversely, in this scenario the pot is laying us 2:1 on our bluffs, and so we become indifferent to betting or checking with our bluffs if our opponent calls 67% of the time. This is our opponent’s GTO strategy. If our opponent deviates from this strategy we can exploit him by always bluffing if he calls less, or by never bluffing if he calls more.

If our opponent deviates from GTO strategy in the previous example, the optimal exploitive strategies of either always folding or always bluffing have higher EV than any exploitive strategies which involve bluffing or folding less than 100% of the time. Weak opponents are weak not only because they choose exploitable strategies so often, but because we can also make such large deviations from indifference points without them adapting to exploit us.

Not all GTO decisions involve finding an indifference point. For example, say we are playing a variant of rock-paper-scissors where there is a fourth option to choose dynamite, which beats everything. The GTO strategy is to choose dynamite-only. Your opponent, however, may still select a dominated strategy by choosing either paper, scissors, or rock. Similar circumstances arise in hold’em, for example, when the nuts is such a large portion of our total range that we are unable to bluff often enough to make our opponent indifferent to calling or folding.

What this means is that while a GTO strategy can never be exploited, and can therefore never be a losing strategy in hold’em (if there is no rake), your opponents can still make dominated strategy decisions which will cause them to lose, and you to win. Therefore, while GTO strategies in hold’em are often suboptimal, the prospect of these “invincible strategies” still hold some exciting implications for a savvy student of game theory, particularly at the highest levels of play.

A tough opponent is only tough, after all, because he or she chooses makes far fewer suboptimal strategy decisions than soft opponent. An extraordinarily tough opponent will have an extremely refined capacity for dynamic play. If you choose a strategy of rock-only, he or she will quickly recognize it and choose paper-only, and so on. Such players will quickly identify trends in your play, or even make pre-emptive assumptions about your play, which may allow them to exploit your non-GTO strategies with unnerving frequency and accuracy.

It is appealing to think that by selecting a GTO strategy, our opponents could only lose. However, even the strongest opponents have exploitive (and therefore potentially-exploited) strategies in their play, and hold’em is, after all, a game of incomplete information. If you are playing against an extremely tough opponent who you know uses a strategy analogous to paper 33%, scissors 20%, and rock 47%, you would be foolish to attempt a strategy of paper-only. By definition of your opponent’s toughness, your opponent will quickly adapt to exploit you. By understanding where the indifference points lie, however, and by making small deviations from them, you can still play exploitatively. Even the toughest, most cut-throat opponents are not clairvoyant, after all, and if you elect an exploitive strategy of paper 40%, scissors 30%, rock 30% how are they to know?

Last edited by The Bryce; 07-09-2008 at 02:20 AM.
The Bryce is offline  
Old 07-09-2008, 06:47 PM   #2
mvdgaag
veteran
 
mvdgaag's Avatar
 
Join Date: Dec 2006
Location: Delft, the Netherlands
Posts: 2,336
Re: Understanding Game Theory and Hold'em

Thanks a lot!
mvdgaag is offline  
Old 07-09-2008, 07:26 PM   #3
mvdgaag
veteran
 
mvdgaag's Avatar
 
Join Date: Dec 2006
Location: Delft, the Netherlands
Posts: 2,336
Re: Understanding Game Theory and Hold'em

I have a game theory question about how to apply it to my real game. I put a simple headsup NL/potlimit holdem game here as an example.


This game has either a nuts/nothing or 0..1 distribution. This doesn't really matter.
All bets are pot sized. A can bet or give up (in which case B wins the pot) and B can call or fold.

I found that if we give up directly after being called we can easily find an overall strategy for each street individually as long as we know we can make our opponent indifferent between his actions on the next street.

A bets 100% of the time. He gives up a% of the time when called.
B calls b% of the time and folds the rest.

If A gives up B wins the pot plus the bet = 2p.
If A doesn't give up, B should also be indifferent on the next street we play, so he loses this bet = -1p.

For B to be indifferent on this street A can give up 33% of his hands and bet the other 66% on the next street.

Since A is risking p to win p, B should call a minimum of p/2p = 50% of the time, for A's bet not to be directly profitable.

Now if we get to the river, B has a much stronger range than A and for B to be indifferent here, A should be having the winning hand 66% of the time when he gets called. But A still has 16/27 of his starting hands, while B has made a tighter selection of 1/8 and will optimally only call half. So he will have far more equity than the 33% A can allow him to have and make him indifferent here. The assumption that on the next street B can be made indifferent is only true if we replicate the same trick. Not if there's an actual showdown.

So to apply this to an overall strategy in real holdem, without getting too specific, we should be working back from the river. We need the best hand 66% of the time, so on the previous street we only need expect to have the best hand at the river 4/9 of the time, the street before that 8/27, etc.

I solve a lot of these mini games when I'm travelling to work, since I got plenty of time in the train then, but I never figured out how to translate these ideas and concepts to practical plays in more specific situations. How do I go about and figure out where to put this agression, on what boards and with which hands?
mvdgaag is offline  
Old 07-22-2008, 07:41 AM   #4
Nichlemn
Carpal \'Tunnel
 
Nichlemn's Avatar
 
Join Date: Oct 2007
Location: The water hemisphere
Posts: 9,407
Re: Understanding Game Theory and Hold'em

I'd like to note that one of the reasons GTO strategies are unnecessary (except for what you mention about understanding how much your opponents deviate from it) is the employment of table selection. If you and your opponent play perfectly, the GTO strategy is best, yes, but you'll both lose to the rake. Only if your opponent was able to play some parts of the game perfectly but had significant leaks in others would you ever want to play a Nash equilibrium.
Nichlemn is offline  
Old 07-22-2008, 12:07 PM   #5
mvdgaag
veteran
 
mvdgaag's Avatar
 
Join Date: Dec 2006
Location: Delft, the Netherlands
Posts: 2,336
Re: Understanding Game Theory and Hold'em

Quote:
Originally Posted by Nichlemn View Post
I'd like to note that one of the reasons GTO strategies are unnecessary (except for what you mention about understanding how much your opponents deviate from it) is the employment of table selection. If you and your opponent play perfectly, the GTO strategy is best, yes, but you'll both lose to the rake. Only if your opponent was able to play some parts of the game perfectly but had significant leaks in others would you ever want to play a Nash equilibrium.
Sometimes you don't know how your opponent plays in certain spots. It's never wrong to choose near optimal plays in those spots while you can always make a big mistake if you choose your standard exploititive play.
mvdgaag is offline  
Old 07-22-2008, 02:01 PM   #6
The Bryce
Stoxpoker Sponsored Forum
 
The Bryce's Avatar
 
Join Date: Oct 2005
Location: Edmonton
Posts: 4,245
Re: Understanding Game Theory and Hold'em

Nice breakdown mv, at a glance it looks solid. That sort of analysis can tell you a lot about 3 barrel bluffing. You would know that if your opponent makes it to the river with a hand he intends to call with less than 50% of the time HU you could exploit him/her by bluffing more, for example.
The Bryce is offline  
Old 07-24-2008, 07:51 AM   #7
BetaPro
adept
 
Join Date: Apr 2008
Location: Montreal
Posts: 1,031
Re: Understanding Game Theory and Hold'em

Thanks so much. I asked you for this article.
BetaPro is offline  
Old 07-25-2008, 12:41 AM   #8
JarnoV
adept
 
Join Date: Nov 2007
Location: generating rake
Posts: 960
Re: Understanding Game Theory and Hold'em

I just wanted to make a couple of minor remarks. First is that GTO doesn't always make the opponent completely indifferent for the actions, but rather it makes indifferent at the thresholds of the regions. Therefore, it's not only dominated strategies that lose value against unexploitive strategy, but those deviations too that aren't at the indifference thresholds.

Second, while this is just a short article and cannot go into depths discussing GTO, I think it's still important enough idea that it should be mentioned that the bluffing frequency is always proportional to the value-betting/raising frequency. If you would throw a X-sided dice at river to decide whether to bluff or not with a hand that has little chance of winning the showdown otherwise, it would not guarantee a suitable frequency for bluffing. It would be pretty difficult to exploit, but it wouldn't be completely balanced.
JarnoV is offline  

 
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 11:40 AM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright © 2008-2017, Two Plus Two Interactive
 
Poker Players - Streaming Live Online