Open Side Menu Go to the Top
Register
The Equivalence of Line EVs in Mixed Strategies. The Equivalence of Line EVs in Mixed Strategies.

09-22-2017 , 03:42 PM
After recently reading Matthew Janda’s excellent latest book, I wrestled with a statement claiming mixed strategies must contain lines of equivalent EV and mixed strategies occur because pure strategies can at times be exploitable. I rather instinctively rejected this since in theory, I felt, we could construct mixed strategies with distinct EVs which improve upon pure strategies (since both strategies will often face distinct counter strategies).

Anyhow, in my ignorance, I wasn’t aware that GTO churns out only EV-equivalent mixed strategy lines and, in fact, tonnes of them. Many hands such as AQo, are revealed to have the same call and raise EV in some spots (as ArtyMcfly kindly pointed out). This absolute equality strikes me as improbable in some natural sense. Of course there can be no natural EV, except perhaps, in some GTO utopia (or rather dystopia :-)): the EV of a hand is contextual - adjustments to our range and strategy as well opponents’ will constantly shift the EV of hands and render lines once equal, now different. **

So why the mass of mixed strategies with equal line-EVs? Why with such innate complexity are so many lines of hands tied to the same EV? *

It seems possible (to me) that rather than GTO forming an optimal strategy with innumerable mixed strategies from all these EV equivalent lines; the natural development of an optimal GTO strategy is to implicitly create as many mixed strategies and thus equivalent lines as possible. So perhaps it is implicit in its optimisation to calibrate the myriad of naturally interdependent line frequencies to produce lines with equal EVs. This seems an improbable task, but of course there are a vast number of parameters available.

Hands with some scenario specific GTO pure strategy presumably can never be calibrated to hold distinct lines with an equivalent EV*. So for example in some situation where we always three-bet Aces, even a disguised flat-calling frequency of 1% isn’t sufficient to boost EV to the 3-bet threshold: so no 99:1 mixed strategy.

Anyhow, I thought I’d at least ask the question: why must (if it is the case) mixed strategy lines in theory be equal? Secondly, given the complexity of the game why, theoretically, do so many lines have equivalent EVs? Why again in theory would a solution (if it is the case) containing more mixed strategies be better?

So in relation to the last question, suppose we invented a game and ran two separate programs to find the best GTO in some limited time period. If one returned with 80:20 ratio of pure to mix and the other 70:30, *would we bet on one strategy being better than the other, if so why?

An additional comment on GTO and mixed strategies. Assuming I am not taking Mr Janda out of context, Matthew mentioned we have mixed strategies in GTO to avoid exploitation. Now, I assume, a GTO strategy is (exclusively) optimal against a GTO strategy and not optimal against any other strategy. Suppose we have a GTO bot, Alpha-GTO and we create another bot to play against it, Beta-bot. Now Beta-bot should eventually develop an identical GTO strategy itself, with mixed strategies. But why will it develop mixed strategies if doesn’t run the risk of exploitation against a non-adaptive opponent? This would seem to suggest that the EVs of lines in mixed strategies are functions of the frequencies - so we can’t be indifferent about the frequency with which we deploy EV equivalent lines against a non-adapting opponent - if we are then Beta-bot won’t need to develop the same unexploitable GTO strategy as Alpha. *

Thanks for reading

{I should add I’ve not used PokerSnowie et al yet, but will do shortly and also not heavily immersed in poker theory, so there is a large dose of ignorance in the post.}

* Or possibly the price to re-calibrate other lines to generate an EV for a line which can be part of a mixed strategy is too high.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-22-2017 , 05:22 PM
You are asking a whole bunch of good questions. Maybe we should start with your most basic question.

To make it easiest, consider a two-person zero-sum game. Suppose further that there is no pure strategy Nash Equilibrium. Then Nash proved that there must be a Nash Equilibrium in mixed strategies (actually von Neumann proved this many years before Nash). We'll come back to what this all means below.

Then it is straightforward to demonstrate that if each player is playing his/her optimal strategy, then all the EV's of all the pure strategies comprising their mixed strategy versus the opponent's optimal mixed strategy will be equal. If any strategy has a higher EV, then, of course, that player would play that pure strategy (and therefore that pair of mixed strategies cannot be jointly optimal). The point is that this "Equal EV" phenomenon is predicated on both players playing their optimal strategies.

As a simple example, suppose two players called Hero and Villain are playing the following game. Hero chooses between two pure strategies called Top and Bottom, while Villain chooses between two pure strategies called Left and Right.

The table below presents the payoffs to the two players called the payoff matrix (the first number is what Hero receives and the second number is what Villain receives).

StrategyLeftRight
Top
90,10
20,80
Bottom
30,70
60,40

It should be clear that there is no "pure strategy" Nash Equilibrium. If Hero plays Top, Villain would play Right. If Villain plays Right, Hero would play Bottom. If Hero plays Bottom, Villain would play Left. And if Villain plays Left, Hero would play Top. So there is no single best pure strategy for either player.

But there is a Nash Equilibrium in mixed strategies. It should be easy to show that the optimal (non-exploitable) strategy for Hero is to play Top 30% and Bottom 70%. Similarly, it should be easy to show that the optimal (non-exploitable) strategy for Villain is to play Left 40% and Right 60%. The "solution" to this game (the expected outcome if both players play their optimal strategies) is seen to be a payoff of 48 to Hero and 52 to Villain.

It is easy to show that if Villain is playing her optimal mixed strategy of 40% Left and 60% Right, then Hero is "indifferent" between playing Top or Bottom. Equivalently, if Villain is playing her optimal mixed strategy, then Hero's EV of playing Top will equal his EV of playing Bottom. (If this is not true, then Villain's strategy cannot be optimal.)

Similarly, if Hero is playing his optimal mixed strategy of 30% Top and 70% Bottom, then Villain's respective EV's of playing either Left or Right will be equal. (If this is not true, then Hero's strategy cannot be optimal.)

Since poker is fairly complicated, most people are convinced that there are no "pure strategy" Nash Equilibria. Of course, that means that there must be one or more mixed strategy Nash Equilibria. In each mixed strategy poker Nash Equilibrium, the "Equal EV" phenomenon must hold.

Anyway, this only scratches the surface of answering your questions. But I thought it might be a good idea if we start with some basics of game theory so we have a common starting point.

Last edited by whosnext; 09-22-2017 at 05:33 PM.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-23-2017 , 06:34 AM
Why is GTO is mess of mixed strategies? Why do so many lines have identical EV?

-GTO strategies come in "optimal pairs"

-A defining characteristic of optimal pairs is that they cannot be improved further. They are maximally exploiting each other, and unexploitable themselves.

-Pure strategies in poker are often exploitable. Therefore mixed strategies are usually necessary to be unexploitable.

-GTO employs mixed strategies if, and only if, each of the lines have identical EV. Otherwise, there is still room to improve by choosing the higher EV line 100% of the time.

So basically GTO has to contain mixed strategies, and the EVs of those different lines have to be identical, or it's not GTO.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-23-2017 , 12:41 PM
Here's a clip from a post I made in one of the limit holdem forums a while back regarding when to mix it and when to pure it:

Quote:
If he plays a pure strategy then we best exploit that by playing a pure strategy of our own; mixing is both unnecessary and non exploitive.

If he plays a properly mixed strategy then we must play a mixed strategy of our own; mixing begets mixing; mixing is both necessary and max exploitive.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-23-2017 , 03:36 PM
Quote:
Originally Posted by AbsurdHero
Why with such innate complexity are so many lines of hands tied to the same EV? *
I think some of this can easily be explained by a large majority of time the pot ends up the same size and our equity in the pot is approximately the same vs an opponent that is value betting and bluffing correctly vs hero's strategy that is also value betting and bluffing correctly.

So for example if we bet and get called by our opponent then if instead we check and then call a similar sized bet by the opponent then we are essentially in the same spot in the next part of the game tree.

Quote:
Originally Posted by AbsurdHero
It seems possible (to me) that rather than GTO forming an optimal strategy with innumerable mixed strategies from all these EV equivalent lines; the natural development of an optimal GTO strategy is to implicitly create as many mixed strategies and thus equivalent lines as possible. So perhaps it is implicit in its optimisation to calibrate the myriad of naturally interdependent line frequencies to produce lines with equal EVs. This seems an improbable task, but of course there are a vast number of parameters available.
I am not sure I completely follow what you are trying to say here but if I do it almost sounds like you're not familiar with how the strategies are derived via computer modeling.

Most models just simulate play in some form or another to maximize EV (or minimize regret). They start with naive strategies and perform the simulation until the EV for a hand changes less than some given amount (say .001bb).

So what I guess I am trying to say it's all about EV and nothing about taking strategic options and just mixing them to mix them.


Quote:
Originally Posted by AbsurdHero
Anyhow, I thought I’d at least ask the question: why must (if it is the case) mixed strategy lines in theory be equal?
The definition of a Nash Equilibrium means that no player in the game can choose an alternate strategy to increase his or her individual EV.

Imagine you had 2 strategic options and one produced a higher EV than the otherone. If you chose the lower EV strategy or a mix of the 2 strategies the game would not be at equilibrium by definition as you could always choose the higher EV strategy with 100% frequency and improve your EV.

So the fact that these computer strategies are showing up with mixed strategies means that the EV of those strategies is equivalent vs the other part of the strategy (i.e. the strategy from the other position).


Quote:
Originally Posted by AbsurdHero
Secondly, given the complexity of the game why, theoretically, do so many lines have equivalent EVs?
It's likely a combination of what I said above, the probabilistic nature of the game, as well as the hidden information component of the game (i.e. players don't know all the information at each decision point due to hidden cards).

Quote:
Originally Posted by AbsurdHero
Why again in theory would a solution (if it is the case) containing more mixed strategies be better?
I think whosnext's post covered this well. Just to add on to his post about why poker is likely not full of pure strategies is again because of the hidden information and stochastic elements.

As an extreme example AA is the best hand you can be dealt preflop in Texas holdem. So let's say as a strategy everytime you get AA you want to either get all in or have your opponent be all in by the river.

Intuitively we know this strategy likely isn't best because sometimes by pure chance (probabilistic nature of the game) we know AA will no longer be the best hand at showdown. So we can amend our strategy to sometimes be all in by the river and sometimes not. So just in this general example we can see it's difficult to construct a pure strategy for easily the most profitable hand in hold'em.

Now introduce the fact that because we don't know our opponent's hand (hidden information) we can simultaneously have the best hand and the worst hand on the river some percentage of the time (i.e. villain can have hands we beat and hands we don't in the same betting line) and you can see why pure strategies don't exist in hold'em (or poker more generally)

Quote:
Originally Posted by AbsurdHero
So in relation to the last question, suppose we invented a game and ran two separate programs to find the best GTO in some limited time period. If one returned with 80:20 ratio of pure to mix and the other 70:30, *would we bet on one strategy being better than the other, if so why?
It would depend on the methodology of each simulation, how long each ran, and how small of change in the EV prior to the simulation terminating. All of those things would effect it.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-23-2017 , 04:23 PM
Q: When an NFL coach is looking at 4th and goal on the one yard line, why doesn't he call the same running play every single time?
A: Because a pure strategy is exploitable (too predictable) and mixed strategies have a higher EV in the long run.

You can score a touchdown by running or passing. A half back dive can score six points, but so can an endzone fade, or a quick slant, or a quarterback sneak. The opposing team has to be able to defend against all these possibilities. If you call each play at its optimal frequency you'll be unexploitable... and score more points than anyone.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-27-2017 , 03:06 PM
Wow tons of great stuff in here. You guys are all great posters!

Does the human preoccupation with bet size ~= pot size create a lot of equal EV decisions? We even set up the solvers this way. The AI computers dont give a hoot about avoiding overbetting multiples of pot. I would guess that THOSE lines are more pure and involve less duality.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-27-2017 , 07:00 PM
Quote:
Originally Posted by AbsurdHero
After recently reading Matthew Janda’s excellent latest book, I wrestled with a statement claiming mixed strategies must contain lines of equivalent EV and mixed strategies occur because pure strategies can at times be exploitable. I rather instinctively rejected this since in theory, I felt, we could construct mixed strategies with distinct EVs which improve upon pure strategies (since both strategies will often face distinct counter strategies).

Anyhow, in my ignorance, I wasn’t aware that GTO churns out only EV-equivalent mixed strategy lines and, in fact, tonnes of them. Many hands such as AQo, are revealed to have the same call and raise EV in some spots (as ArtyMcfly kindly pointed out). This absolute equality strikes me as improbable in some natural sense. Of course there can be no natural EV, except perhaps, in some GTO utopia (or rather dystopia :-)): the EV of a hand is contextual - adjustments to our range and strategy as well opponents’ will constantly shift the EV of hands and render lines once equal, now different. **

So why the mass of mixed strategies with equal line-EVs? Why with such innate complexity are so many lines of hands tied to the same EV? *

It seems possible (to me) that rather than GTO forming an optimal strategy with innumerable mixed strategies from all these EV equivalent lines; the natural development of an optimal GTO strategy is to implicitly create as many mixed strategies and thus equivalent lines as possible. So perhaps it is implicit in its optimisation to calibrate the myriad of naturally interdependent line frequencies to produce lines with equal EVs. This seems an improbable task, but of course there are a vast number of parameters available.

Hands with some scenario specific GTO pure strategy presumably can never be calibrated to hold distinct lines with an equivalent EV*. So for example in some situation where we always three-bet Aces, even a disguised flat-calling frequency of 1% isn’t sufficient to boost EV to the 3-bet threshold: so no 99:1 mixed strategy.

Anyhow, I thought I’d at least ask the question: why must (if it is the case) mixed strategy lines in theory be equal? Secondly, given the complexity of the game why, theoretically, do so many lines have equivalent EVs? Why again in theory would a solution (if it is the case) containing more mixed strategies be better?

So in relation to the last question, suppose we invented a game and ran two separate programs to find the best GTO in some limited time period. If one returned with 80:20 ratio of pure to mix and the other 70:30, *would we bet on one strategy being better than the other, if so why?

An additional comment on GTO and mixed strategies. Assuming I am not taking Mr Janda out of context, Matthew mentioned we have mixed strategies in GTO to avoid exploitation. Now, I assume, a GTO strategy is (exclusively) optimal against a GTO strategy and not optimal against any other strategy. Suppose we have a GTO bot, Alpha-GTO and we create another bot to play against it, Beta-bot. Now Beta-bot should eventually develop an identical GTO strategy itself, with mixed strategies. But why will it develop mixed strategies if doesn’t run the risk of exploitation against a non-adaptive opponent? This would seem to suggest that the EVs of lines in mixed strategies are functions of the frequencies - so we can’t be indifferent about the frequency with which we deploy EV equivalent lines against a non-adapting opponent - if we are then Beta-bot won’t need to develop the same unexploitable GTO strategy as Alpha. *

Thanks for reading

{I should add I’ve not used PokerSnowie et al yet, but will do shortly and also not heavily immersed in poker theory, so there is a large dose of ignorance in the post.}

* Or possibly the price to re-calibrate other lines to generate an EV for a line which can be part of a mixed strategy is too high.
- What would a pure strategy be refering to in a poker context anyway? Given different pre flop ranges, bet sizing, different board textures, flop turn and river. I can understand how one could use a pure strategy with a specific hand i.e. bet 100% x sizing with AA on xxx flop. Which piosolver does recommend in some spots given pre flop range and fixed bet sizing, but I don't give piosolver the option of every bet size because it would crash my computer if it's even possible to do so, so maybe it would have a 100% mixed strategy for every hand in every possible scenario in regards to sizing i.e. it would never bet one size with one hand in one spot?
- "Many hands such as AQo, are revealed to have the same call and raise EV in some spots ". This doesn't mean a strategy will be mixed between call and raise though does it? The call and raise strategy will be based off of the range as a whole. I.e. if your check range is strong one might want to add AQ (assumed value hand) in to the cbet range instead to make it stronger and balance it/allow more bluffs.
- Maybe I am lacking understand but I don't understand what you mean by "Why must (if it is the case) mixed strategy lines in theory be equal?"
Equal in what way? and equal to what? all other mixed strategy lines?
- "Secondly, given the complexity of the game why, theoretically, do so many lines have equivalent EVs?"
If you mean, checking AQ 50% and betting it 50%, then its not because checking and betting AQ is equal in EV if opponent plays perfect strategy is it? It's because checking that value hand in some spots protects your checking whole range from being weak.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 07:09 AM
Quote:
Originally Posted by robert_utk
Wow tons of great stuff in here. You guys are all great posters!

Does the human preoccupation with bet size ~= pot size create a lot of equal EV decisions? We even set up the solvers this way. The AI computers dont give a hoot about avoiding overbetting multiples of pot. I would guess that THOSE lines are more pure and involve less duality.
I would say that allowing different betsizes would change the range composition of some lines but likely wouldn't change the EV by much.

Also I would say that there is a limit to how big bets could be. A lot of the time Nash Equilibrium strategies seem to be conservative in the sense that they seem to have lines that protect EV as opposed to greedily maximizing it.

In that way I would think incredibly large over bets (like >2*pot) would be easily exploitable by an opponent, especially since there are a fixed number of bluffing candidates and the larger you bet the more you would need to get a call from a bluff catcher.

Sent from my SM-G900R4 using Tapatalk
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 07:29 AM
Quote:
Originally Posted by Desultory
- What would a pure strategy be refering to in a poker context anyway? Given different pre flop ranges, bet sizing, different board textures, flop turn and river. I can understand how one could use a pure strategy with a specific hand i.e. bet 100% x sizing with AA on xxx flop. Which piosolver does recommend in some spots given pre flop range and fixed bet sizing, but I don't give piosolver the option of every bet size because it would crash my computer if it's even possible to do so, so maybe it would have a 100% mixed strategy for every hand in every possible scenario in regards to sizing i.e. it would never bet one size with one hand in one spot?
If you look at the solutions provided by solvers if you look at a certain point in the game tree you will see recommendations to bet different sizing or check the same hand and that will be listed for almost all hands.

By pure strategy I think op meant "why at certain points in the game tree are we not 100% bet with 1 sizing or 100% fold with individual hands?"

Quote:
Originally Posted by Desultory
- "Many hands such as AQo, are revealed to have the same call and raise EV in some spots ". This doesn't mean a strategy will be mixed between call and raise though does it? The call and raise strategy will be based off of the range as a whole. I.e. if your check range is strong one might want to add AQ (assumed value hand) in to the cbet range instead to make it stronger and balance it/allow more bluffs.
Yes if 2 lines have equivalent EV in a certain spot then the solver will likely mix between the lines so you can get to the same point in the hand and take 2 different actions.

Your second part is an easy way for humans to understand and use mixing but Nash Equilibrium strategies by computers are highly complex about mixing certain actions with equivalent EVs. The computer models don't care about ranges they just look directly at EV.

Quote:
Originally Posted by Desultory
- Maybe I am lacking understand but I don't understand what you mean by "Why must (if it is the case) mixed strategy lines in theory be equal?"
Equal in what way? and equal to what? all other mixed strategy lines?
I think OP meant if I get to a point where a solver uses a mixed strategy, why is it that the mixed strategies at that point must be equal in EV?

Quote:
Originally Posted by Desultory
- "Secondly, given the complexity of the game why, theoretically, do so many lines have equivalent EVs?"
If you mean, checking AQ 50% and betting it 50%, then its not because checking and betting AQ is equal in EV if opponent plays perfect strategy is it? It's because checking that value hand in some spots protects your checking whole range from being weak.
That is incorrect. If the computer models choose to take a mixed action it is because the 2 lines have equivalent EV.

The solvers are modeling what is called a Nash Equillibrium. A basic definition of a Nash Equillibrium is that when a game is at equillibrium, no player in the game can change his or her strategy to improve his or her individual EV.

So if an equillibrium strategy includes mixed actions, it means that both the actions have equivalent EV. If they did not have equal EV it means that one of them yields more EV so the solution cannot be in equillibrium by definition because the player could chose to change his or her strategy to the pure strategy with the higher EV thus violating the definition of equillibrium.



Sent from my SM-G900R4 using Tapatalk
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 12:54 PM
Quote:
Originally Posted by just_grindin
I would say that allowing different betsizes would change the range composition of some lines but likely wouldn't change the EV by much.

Also I would say that there is a limit to how big bets could be. A lot of the time Nash Equilibrium strategies seem to be conservative in the sense that they seem to have lines that protect EV as opposed to greedily maximizing it.

In that way I would think incredibly large over bets (like >2*pot) would be easily exploitable by an opponent, especially since there are a fixed number of bluffing candidates and the larger you bet the more you would need to get a call from a bluff catcher.

Sent from my SM-G900R4 using Tapatalk


Very interesting, thanks jg.
Overbets would/are highly polar for sure. They also involve pulling bottom value range and turning those into potential bluffs. However, I do believe that mathematical Nash frequency can be derived for these types of bets/bluffs and there would potentially be a ton of profit from humans overfolding (or calling with the wrong value) against this agression. Variance would be huge, and bankroll would need to be appropriate.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 02:57 PM
Quote:
Originally Posted by Desultory
If you mean, checking AQ 50% and betting it 50%, then its not because checking and betting AQ is equal in EV if opponent plays perfect strategy is it? It's because checking that value hand in some spots protects your checking whole range from being weak.
Two sides of the same coin. Where a mixed strategy is used, the two options do have exactly the same EV, and while you could say "Checking stops your range being too weak" (it prevents villain from exploiting you by bluffing you off your weak range), you could also say "It stops your betting range being too strong" (which villain could exploit by folding to your overly value-heavy betting range).
If you did one option all the time, you become exploitable, because one of your ranges will be "too strong" and the other will be "too weak". If you balance/mix correctly, your betting range maximises its EV, but so does your checking range. The hands that fit into both groups will have the same EV with each option.
Note: Do not assume that the mixing always happens at a 50:50 ratio. Some hands might check as little as 1% of the time, if that's all they need to put enough "strength" into the checking range.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 04:11 PM
Quote:
Originally Posted by just_grindin
Yes if 2 lines have equivalent EV in a certain spot then the solver will likely mix between the lines so you can get to the same point in the hand and take 2 different actions.
Ok that makes sense from a logical perspective.
The point I was making was that the solver would factor the whole range and consider the weights of acting with AQ even if the AQ had 50:50 EV for bet:check? It wouldn't just make them bet 50% check 50% because the EV is equal? as I supposed op was asking.

If AQ had 0.75EV for betting and 0.25EV for checking what happens? What to do with AQ here will depend on the EV of the whole range right and AQ action will be weighted accordingly?

Does a solver ever have 100% betting or check call frequencies with a specific hand?

Quote:
Originally Posted by just_grindin
That is incorrect. If the computer models choose to take a mixed action it is because the 2 lines have equivalent EV.
When you mention lines here we are talking about whole balanced ranges line (bet,x,bet) not just a line with AQ?

Quote:
Originally Posted by just_grindin
So if an equillibrium strategy includes mixed actions, it means that both the actions have equivalent EV. If they did not have equal EV it means that one of them yields more EV so the solution cannot be in equillibrium by definition because the player could chose to change his or her strategy to the pure strategy with the higher EV thus violating the definition of equillibrium.
If this is a near to a summation of what op wrote, then I misunderstood what he wrote, and my questions are probably almost off topic sorry.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 04:38 PM
Quote:
Originally Posted by Desultory
Ok that makes sense from a logical perspective.
The point I was making was that the solver would factor the whole range and consider the weights of acting with AQ even if the AQ had 50:50 EV for bet:check? It wouldn't just make them bet 50% check 50% because the EV is equal? as I supposed op was asking.
Correct. The percentages are calibrated as part of the response to the other strategy in the equillibrium and are not completely arbitrary.

However let's say you have a spot where you bet AQ 75% and check it 25% and you bet AJ 50% and x/c AJ 50%. As long as your opponent is no longer adjusting and simply playing the equilibrium strategy as is, you could bet AQ 100% and bet AJ 25% of the time and check AJ 75% of the time because both lines are equivalent in EV for each of those hands at that point in the equillibrium AND we have fixed your opponent's strategy to prevent this exploitation. Notice we also didn't change our overall betting and checking frequencies.



Quote:
Originally Posted by Desultory
If AQ had 0.75EV for betting and 0.25EV for checking what happens? What to do with AQ here will depend on the EV of the whole range right and AQ action will be weighted accordingly?
My understanding is the solvers would choose the highest EV option every time. Again see the definition of Nash Equilibrium for an explaination. As far as I know ranges are simply a natural result of taking the same action with individual hands. The solvers aren't trying to build ranges they are maximizing EV of individual hands.

Quote:
Originally Posted by Desultory
Does a solver ever have 100% betting or check call frequencies with a specific hand?
At individual nodes in the game tree I would imagine that you will see 100% strategies like raise preflop with AA every time, but over the entire tree as hand values change based on chance and unknown information it's likely you'll see mixed strategies.


Quote:
Originally Posted by Desultory
When you mention lines here we are talking about whole balanced ranges line (bet,x,bet) not just a line with AQ?
Well both. The solvers come up with lines for individual hands, but many hands will have the same betting line. But again it's all about EV and not necessarily about composing ranges, though you get multiple hands that take the same betting line.

Basically the solvers answer the question "What is the most profitable line I can take with this individual hand?" And then play every possible scenario with that hand to determine the answer.






Sent from my SM-G900R4 using Tapatalk
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 07:06 PM
Thanks very much for the great replies (I’m a little behind in responding to later posts). First of all, apologies for the superfluous asterisks in the opening post, a curse of the 2+2 editor. There is one purposeful asterisk at the end of the sentence ‘…………..lines with an equivalent EV*.


Just_grindin: I am not sure I completely follow what you are trying to say here but if I do it almost sounds like you're not familiar with how the strategies are derived via computer modeling.

Yes, this is partly true, I have some limited experience in OR and linear programming some years ago.

Most models just simulate play in some form or another to maximize EV (or minimize regret). They start with naive strategies and perform the simulation until the EV for a hand changes less than some given amount (say .001bb).

Instinctively, I would say, though, that finding solutions exclusively this way is problematic: it is very easy to wind up on a local maximum. Most of us experience this frustrating state while witnessing profitable plays which would never reward us. These desired actions are optimised within the framework of a distinct overarching strategy, which may on balance be better or worse than our own. Finding other possible optimal strategies is not always a straightforward process - I imagine there would be a requirement to simultaneously seed multiple different lines to break out of local maximums since testing one at a time will likely always lead back to the same solution. From recollection this is certainly one of the challenges of numerical methods.

You’re right though it wasn't very clear, so I’ll try again:

One possible mind-model we possess when envisaging the optimisation process is where a whole bunch of hands with EVs determined by the line, player positions, stack etc are utilised; drill down sufficiently and we’ll find an exact EV in some specific circumstance set to a frequency of our choosing with other lines of coincidentally equivalent EVs. The model in my mind, though, is one where the mix determines the EV - where were we to start adjusting the line-frequencies of a number of hands in some strategy (which are naturally interdependent) the EVs will change. And where, for example, the EVs of two lines are equal at a 70:30 mix, they won’t be at say at 50:50 or even while maintaining the 70:30 but adjusting other line-frequencies within the range. This seems sensible: the value of bluffing our flush draw, say, will be predicated on the hands we can likely represent, which of course will be determined by other line-frequencies.

So in essence rather trying to (in effect) optimise the distributions of a mixed strategy from a set of lines with equal EVs, the program through the optimisation process is led to specific instances where lines of EV are equal because they occur at the maxima and are specific to the line-frequencies. So for example let’s say two lines at 60:40 have EV’s at 0.8 and 0.85 respectively (combined 0.82), but at 50:50 it’s 0.825, say. Now, why, if my suggestion is the case, are the EVs of mixed strategies with unequal EV-lines always inferior to those with equal EVs? Why say in the case of the previous example is the equilibrium not 0.815, thus allowing the selection of the mixed strategy with unequal line-EVs?

I’m guessing it would need to be demonstrated for some class of functions f and g that:

max {f(p) + g(1-p)} for p∈(0,1) = X when f(p) = g(1-p)

Perhaps it is implicit if the functions are continuous. So in the above example where we have a 60:40 mixed strategy with respective EVs of 0.85 and 0.80 if it can be shown that

EV(0.60 + ε, 0.40 - ε) > EV( 0.60, -.40) for some very small ε

then it could be proven that the overall EV is maximised when the EVs of the lines are equal.

The above inequality makes intuitive sense since we are marginally increasing the distribution of the line with an EV of 0.85 and marginally decreasing distribution of the line with an EV of 0.80. Effectively swapping in an epsilon’s worth of 0.85 EV and swapping out an epsilon’s worth of 0.80EV. But it needs proving. The obstacle though is that if the EV of the line decreases as its frequency increases then we also lose some EV while also gaining elsewhere from the swap. So if we say that EV(Line1:0.60) =0.85 but the EV(Line1:0.60+ε) = 0.85 - δ1 (where δ1 is small) then we’d need to show that

(0.6 * 0.85) + 0.4*0.80) < (0.6+ε) *(0.85 - δ1) + *(0.4+ε) *(0.80 - δ2)

(I’m sure Whosnext has a proof for the result needed)


Fold Once: -Pure strategies in poker are often exploitable. Therefore mixed strategies are usually necessary to be unexploitable.

-GTO employs mixed strategies if, and only if, each of the lines have identical EV. Otherwise, there is still room to improve by choosing the higher EV line 100% of the time.

Hi Fold Once, thanks for your comment. The dual claims of the exploitability of (some) pure strategies and the improvability of mixed strategies with unequal EVs through the full deployment of the higher EV line are on appearance inconsistent. If a pure strategy is exploitable then by implication the line takes an EV-hit when exploited. So we can’t simply take the best EV-line in the mixed strategy, jack it up to 100% and assume it is an improvement. Were the opponent’s strategy unaltered by any change in distribution, then of course this claim would be true but in such cases the line isn’t being any more or less exploited as a pure strategy than it is when part of a mixed one.


Hi Bob148: If he plays a pure strategy then we best exploit that by playing a pure strategy of our own; mixing is both unnecessary and non exploitive.

If he plays a properly mixed strategy then we must play a mixed strategy of our own; mixing begets mixing; mixing is both necessary and max exploitive.


Except in current theory if the opponent is employing a non-adaptive mixed strategy (as in GTO) then you just take the most profitable line as you’re not worried about an exploitative counter-strategy. So it would follow that the optimal strategy against a GTO would be a bunch of pure strategies yet by definition the optimal strategy against GTO is GTO - a mass of mixed strategies (I believe). This may suggests that mixing is not simply about avoiding exploitation but rather about creating synergies.

Who’sNext: Thanks for your insightful reply.

Then it is straightforward to demonstrate that if each player is playing his/her optimal strategy, then all the EV's of all the pure strategies comprising their mixed strategy versus the opponent's optimal mixed strategy will be equal. If any strategy has a higher EV, then, of course, that player would play that pure strategy (and therefore that pair of mixed strategies cannot be jointly optimal). The point is that this "Equal EV" phenomenon is predicated on both players playing their optimal strategies.

I’m not sure what is meant here in particular this line:

then all the EV's of all the pure strategies comprising their mixed strategy versus the opponent's optimal mixed strategy will be equal.

Do you mean the EV’s of the lines within the mixed strategy? I’m not clear on what is summed on both sides to create this equality.

In your example the EVs of the two lines ‘top and bottom’ are functions of the frequency of the lines within the mixed strategy. In order to maximise the EV of the strategy the line-frequencies are solved for lines with equal EVs. Repeating the earlier point I suggest it is the same in the poker GTO mixed strategies: the lines are calibrated to be of equal EV. Two lines have equal EV at some specific mix, subtly adjust the mix, though, and they are no longer equal. This would explain why we have what appears to be so many lines of equal-EV. But to say, I would suggest, that in some situation A5s has the same EV whether we call or raise is most likely false - it is only at the specific distribution the strategy churns out where they are found to be equal.

Once again thanks for all the contributions.

Last edited by AbsurdHero; 09-28-2017 at 07:12 PM.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-28-2017 , 08:22 PM
I presented the absolutely simplest example above so that there should be no confusion on the underlying "equal EV" principle. And if there is any confusion, anybody can work through the example in less than a minute on the back of an envelope (what I do below).

If Villain plays her optimal strategy of 40% Left and 60% Right, then what is Hero's EV of each of his pure strategies?

Hero's EV of playing Top (when Villain is playing 40/60):
= (.4)*90 + (.6)*20
= 48

Hero's EV of playing Bottom (when Villain is playing 40/60):
= (.4)*30 + (.6)*60
= 48

48 = 48 meaning that the two EV's of Hero playing his respective pure strategies vs Villain's optimal mixed strategy are equal.

It can also be easily seen that each of the EV's of Villain's pure strategies vs. Hero's optimal mixed strategy of 30% Top and 70% Bottom are equal @ 52 (.3*10+.7*70 = 52 = .3*80+.7*40).

People in the thread have explained why this must be true if both players are playing their optimal strategies. (Some people utilize this indifference principle as a way to derive the respective weights in each player's optimal mixed strategy.)

Hopefully, this is clear to all.

Last edited by whosnext; 09-28-2017 at 08:29 PM.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 05:20 AM
I presented the absolutely simplest example above so that there should be no confusion on the underlying "equal EV" principle. And if there is any confusion, anybody can work through the example in less than a minute on the back of an envelope (what I do below).

Hi whosnext

I think you misunderstood me, the example was perfect in demonstrating the equality of EVs; I thought you also had a proof at hand from Nash/Von Neumann proving why it will always be the case. Your example also reinforced the argument I was making that the EVs of all the lines in GTO 'are made to be equal' rather than as a result of some quirk or property of the game: only at the precise distributions within the mixed strategy are the EVs equal - if we alter the mix, they're no longer equal. We can see that very clearly in the case you highlighted.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 07:11 AM
Quote:
Originally Posted by AbsurdHero
only at the precise distributions within the mixed strategy are the EVs equal - if we alter the mix, they're no longer equal. We can see that very clearly in the case you highlighted.
That's simply wrong. If an action pair is mixed 50-50 at equilibrium it doesn't matter if you stay at 50-50 or pick any mix in between 0 and 100, the ev against the nemesis will always be the same. The nemesis strategy is fixed, it doesn't adjust, it just guarantees a certain EV.

What is true is that if you pick a mix that differs from optimal you leave open the door to being exploited, but that's a totally different subject. We're talking about optimal strategy pairs here.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 10:12 AM
Quote:
Originally Posted by getmeoffcompletely
That's simply wrong. If an action pair is mixed 50-50 at equilibrium it doesn't matter if you stay at 50-50 or pick any mix in between 0 and 100, the ev against the nemesis will always be the same. The nemesis strategy is fixed, it doesn't adjust, it just guarantees a certain EV.

What is true is that if you pick a mix that differs from optimal you leave open the door to being exploited, but that's a totally different subject. We're talking about optimal strategy pairs here.
I may have misunderstood you but as I don't see how you've debunked my assertion. My simple observation is that if you adjust in anyway the frequencies of the lines in whosnext's example then the lines no longer yield the same EV. The second point is that I believe this is also true of the mixed strategies in the software solutions where lines have equal EVs: they are only equal at precisely those frequencies. If some line is 97% fold, 3%raise then that distribution is the only point they are equal (unless we were to fix other lines on the overall strategy in some inefficient way, then we would likely find different mixes).

The point I believe you are making is if, for example, my opponent has some non-adaptive strategy and my two actions do equally well against it then the mix doesn't matter. Which is of course true. But the mixed strategy example whosnext provided and the formation of GTO strategies aren't assuming a non-adaptive opponent.

Apologies if I've misunderstood your point.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 11:23 AM
Yes, I think you are missing the point.

The concept of a Nash Equilibrium (GTO in poker parlance) relies upon the notion that each player considers varying his own strategy while all the strategies of his opponents are fixed and static (non-varying).

This is the heart of a Nash Equilibrium. This is the heart of GTO. This is the heart of the "equal EV" principle.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 01:19 PM
QUOTE=whosnext;52907471]Yes, I think you are missing the point.

The concept of a Nash Equilibrium (GTO in poker parlance) relies upon the notion that each player considers varying his own strategy while all the strategies of his opponents are fixed and static (non-varying).

This is the heart of a Nash Equilibrium. This is the heart of GTO. This is the heart of the "equal EV" principle.[/QUOTE]

Obviously the Nash solution is predicated on the presence of an adaptive opponent not a static one. What you appear to describe is an iterative, turn-taking process: if I make this move, does it improve against my current opponents strategy? If yes then await for the counter strategy to see if we maintain the change or retreat from it. But of course there is no assumption that any improvement made against your opponent's frozen-strategy is held once the adversary is free to readapt. Nash is not optimised or developed against some fixed strategy unless presumably it happens to be a GTO.

To suggest that lines retain the EVs in mixed strategies whatever the distribution makes little sense - which is my point. They are only every equal at precisely the distribution the GTO provides (as in your example). It is inconsistent with the property of pure strategies being exploitable that we can retain the EVs regardless of the frequency distribution of lines within the mix. It is also obvious that the EV of a line is a function of all the other lines and their distributions - so if we adjust those we will clearly affect the EV of the lines we are paying attention to, once of course our opponent is allowed to adapt. Which is, dare I say, the heart of the Nash assumption: developing an unexploitable strategy against an adaptive opponent.

Sure, as in the example you provide if you fix the opponent's strategy to 40% left 60% right, then the top/bottom distribution will retain equivalent line EVs for any mix. But allow the opponent to adapt and there will only be a unique mix where the EVs are equivalent. So if for some GTO line we have 96:4 distribution then this will be a unique point where the line-EVs are equal unless we fix our opponent's strategy. If changed to 93:7 against a frozen strategy then obviously the EV of the mix-strategy and individual lines will be identical to the 96:4, but once we allow the opponent to adapt to the altered mix then the EVs of the lines in the 93:7 would become unequal. So in your example we can go 90% top and 10% bottom against the 40:60 left right and still retain the same EVs for top and bottom as 30:70. But obviously once the opponent adapts then the EVs of top and bottom at a 90:10 mix won't be equal.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 01:41 PM
Quote:
Originally Posted by Desultory
If AQ had 0.75EV for betting and 0.25EV for checking what happens? What to do with AQ here will depend on the EV of the whole range right and AQ action will be weighted accordingly?
If one option has a higher EV than another at equilibrium, you should always do it. e.g. If placing a cross in the middle box at the start of a game of tic tac toe gives you a better chance of winning the game than putting a cross in the bottom right, go for the middle: It has the highest EV.
Quote:
Originally Posted by Desultory
Does a solver ever have 100% betting or check call frequencies with a specific hand?
Yes. Usually (but not always), the pure strategies are associated with the most 'obvious' valuebets and the clearest bluffs.
This visual example from Snowie (which hasn't solved poker, but is somewhat useful for approximations), might also help OP:



BTN opens 2.25x pre, BB calls. There is 5bb in the middle. Flop comes AT3r and it's checked to BTN. With AQo on this board, Snowie uses a mixed strategy. It checks back 76%, and bets half pot 24% of the time. Its database tells it that it had an EV of 4.96bb with both lines, which is why both choices are viable.

With some other hands, pure strategies are used. All the sets and two pair combos are so valuable (and want to build big pots) that Snowie suggests betting them at 100% frequency).



By betting all the nutted hands, BTN can also profitably bluff a decent number of combos, since the EV of the bluffing combos is increased if you have the nuts in your range. (For this spot, Snowie would balance the value hands by having pure 100% betting frequencies with a number or airballs, like Q8s, and literally every hand in its range that is worse than J7s).
To pad out the betting and checking ranges, it would put various mid-strength hands (one pairs, various Kx/Qx) into each 'bucket', mixing their frequencies until the EV of the betting range is maximised, but the checking range is good too.

Bonus: Here's a Pio solution from an entirely different spot, where there are three raise sizes to choose from, but checking is also possible (it's for the BB facing a limp in a HUSNG, IIRC). Some hands have pure strategies in this spot (e.g. Q7s is solid green, meaning 100% check), but others take each option at a different frequency.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 02:03 PM
Quote:
Originally Posted by Desultory
If AQ had 0.75EV for betting and 0.25EV for checking what happens? What to do with AQ here will depend on the EV of the whole range right and AQ action will be weighted accordingly?
Does a solver ever use a lower EV line such as the imaginary 0.25EV check instead of the imaginary 0.75EV bet for a specific hand in order to satisfy nash equilibrium for it's entire/opponents range?

I can imagine this scenario possible in small vs small ranges?
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 02:40 PM
Quote:
Originally Posted by AbsurdHero
Obviously the Nash solution is predicated on the presence of an adaptive opponent not a static one. What you appear to describe is an iterative, turn-taking process: if I make this move, does it improve against my current opponents strategy? If yes then await for the counter strategy to see if we maintain the change or retreat from it. But of course there is no assumption that any improvement made against your opponent's frozen-strategy is held once the adversary is free to readapt. Nash is not optimised or developed against some fixed strategy unless presumably it happens to be a GTO.

To suggest that lines retain the EVs in mixed strategies whatever the distribution makes little sense - which is my point. They are only every equal at precisely the distribution the GTO provides (as in your example). It is inconsistent with the property of pure strategies being exploitable that we can retain the EVs regardless of the frequency distribution of lines within the mix. It is also obvious that the EV of a line is a function of all the other lines and their distributions - so if we adjust those we will clearly affect the EV of the lines we are paying attention to, once of course our opponent is allowed to adapt. Which is, dare I say, the heart of the Nash assumption: developing an unexploitable strategy against an adaptive opponent.

Sure, as in the example you provide if you fix the opponent's strategy to 40% left 60% right, then the top/bottom distribution will retain equivalent line EVs for any mix. But allow the opponent to adapt and there will only be a unique mix where the EVs are equivalent. So if for some GTO line we have 96:4 distribution then this will be a unique point where the line-EVs are equal unless we fix our opponent's strategy. If changed to 93:7 against a frozen strategy then obviously the EV of the mix-strategy and individual lines will be identical to the 96:4, but once we allow the opponent to adapt to the altered mix then the EVs of the lines in the 93:7 would become unequal. So in your example we can go 90% top and 10% bottom against the 40:60 left right and still retain the same EVs for top and bottom as 30:70. But obviously once the opponent adapts then the EVs of top and bottom at a 90:10 mix won't be equal.
I guess the concept of Nash Equilibrium may be difficult for people to fully grasp. A lot of good information on Nash Equilibrium can be found in any introductory game theory course or textbook. I even imagine that there is information on it available on the internet.
The Equivalence of Line EVs in Mixed Strategies. Quote
09-29-2017 , 03:36 PM
Quote:
Originally Posted by whosnext
I guess the concept of Nash Equilibrium may be difficult for people to fully grasp. A lot of good information on Nash Equilibrium can be found in any introductory game theory course or textbook. I even imagine that there is information on it available on the internet.
Thanks I'm sure I will read more. I'm not sure why you are assuming I'm puzzled by the Nash Equilibrium, as what I've recently stated hasn't resulted from a misunderstanding of it.

I was initially very curious about the presence of so many lines with equal EV as this seemed to be a very unlikely occurrence. I now realise that is still probably the case in practice, just as it would be in the example you provided - if you took suboptimal strategies. The Nash solution provides lines of equal EV - if you adjust the frequency distribution and allow the opponent to adapt the lines won't be of equal EV (or certainly not all of them). Lines of equivalent EVs are the product of the Nash solution which is defined by its mixed strategy frequencies.

If we threw a couple of randomised strategies against each other which were fairly close to the Nash equilibrium, I'd be very surprised if we found any two lines which were of equivalent EV. That lines of equal EV are typically rare but are a very common product of optimal solutions is really the explanation I was looking for.
The Equivalence of Line EVs in Mixed Strategies. Quote

      
m