Hi Everyone:

This is from the “Win Rate” section in the Pre-flop Play chapter

After playing 100,000 hands, players start to get an idea of what their actual win rate looks like, but a sample of at least 1,000,000 hands is required to be statistically significant”

All comments welcome.

Mason

I saw some post a long time ago that was about some player who ran rather differently during his second 100k run and that comes to my mind here.

I prefer to look at the tracking site stats (cash, tourneys) to give me an idea of what to expect in this or that form and format per month, year, years.

Some web sites have softwares that give math/simulated results.

I would say he's both right and wrong.

100k hands is not enough to know how you're doing. Unless your win rate is Very High you might not even know if you're a winning player long term.

But it's also laughable to think that you're the same player after 1 million hands as you were at hand 0. So you probably shouldn't "count" some number of your first hands. How many should you discount? Or can you include them but with some kind of adjustment factor?

Not to mention that if you're any good at all you won't be playing the same players (or the same stakes) at hand 1 million as at hand zero, so not only have you changed, but your opposition has also

Hi Rusty:

Besides what you state being accurate, suppose you're a very good player and are in a 9-handed hold 'em game, limit or no-limit, you raise UTG and the unknown player on your immediate left calls with king-ten offsuit. How many hands do you need to play to know you have a big advantage (at least against this player)?

Best wishes,
Mason

Does the author attempt to prove this mathematically in the book, or are we just expected to take his word for it?

(Sorry, I don’t have this book..)

Although I think I see what you're getting at, I'm not 100% sure it's apropos to the quote from your OP.

It is possible to know that you're a big winner vs someone at your table, and still be a losing player at your stakes. I think that's probably a fairly uncontroversial statement.

But I think it's possible to be a winner against many or even most of the players at your table, and still be a loser at your stakes. This could be due to several intersecting reasons:
* table/game selection - you may be a winner at some tables and not at others
* you may win against many opponent but lose against others, and the amount you lose may be more than what you win
* at the lowest stakes, your edge may not be enough to overcome rake, even against bad players - this is a really common problem, some "types" of bad players will lose badly over time, but you won't necessarily profit from it
* it is very common to be able to recognize that you're "better" than another player without being able to assign a long term EV value to your superiority. By design poker games include a lot of randomness and very loose players can do better than most "good" players expect. Your anticipated EV against a loose aggressive player who is the tiniest bit wily can disappear faster than most people expect.

This last one I have seen personally a lot. There are a lot of well known players who are loose, aggressive, and just a bit canny, who are under rated by their opponents. I think a good TAG player can have a small advantage against these players, but rate themselves as having a *large* advantage. They can also lose their edge easily against these players via tilt or underestimation. I personally learned a lot about some of the advantages a LAG player can have by dropping down several levels, playing very LAG, and watching people play their hands face up against me.

^ i like what rusty said. in this game we often talk about win rates as if the ecosystem is just a big static variable, but winning in poker is about succeeding VS individuals, and each hand/game is a unique event in which our edge varies dramatically

In the time it takes to play 1M hands the ecosystem itself will change to some degree, changing our win rate with it...additionally our skill will increase across this timeline (as rusty said). Its almost like the author is saying you cannot gauge the skill of a highschool athlete until they have played 4 seasons, but in those four years the player will grow, and the teams/players he faces will also change

Hi Rusty:

I’ve only read about one-fourth of the book, and where I came across this it’s just stated.

Best wishes,
Mason

The required samples size depends on the player’s standard deviation, the confidence level and the acceptable deviation. Relying on the Central Limit Theorem for normality of a sample mean you can show that a sample size of about 1.2 million is required to have 90% confidence of the mean being within 1bb of the true value if the SD is 85bb/100. However, it is less than half of that at 525,000 if the player accepts a deviation of 1.5bb from the true value.

At 80% confidence, the required sample sizes are about half of those stated above. The confidence level to use along with the deviation specified will naturally depend on how the result will be used. I agree with the concern about the likely change in the playing environment.

Hi Everyone:

Statmanhal has this right. For an estimate to be statistically significant within a certain amount we need to know the standard deviation and the level of significance we’ll be measuring at. A statement like that given in my initial post doesn’t make any sense.

Also, as Stat uses in his example, the standard deviation of an arbitrary number, 1 BB in this case, is zero. But if he wanted to test whether the difference between two estimates, which could also be 1BB, it’s a different formula and the standard deviation of the difference would be larger than the standard deviation of either individual estimate.

Best wishes,
Mason

Sorry, but I think this is an attempt to create agreement (over a single sentence) that a book by another author and from another publisher isn't good enough. We know you don't like Z&X Publishing or Lonathan Jittle.

IMO, there was no need to even make this thread. It seems sour grapes to me, especially since the author was simply pointing out that you need an extremely large sample before you can say anything with any degree of certainty about your winrate.

It's obvious this book was not written to be a poker statistician's companion. I have found very little heavy math computation and zero actual statistical computation. Instead, I believe it is meant for an audience that wants to have an understanding of GTO play without needing a math or statistics degree to get it.

Let's see, about a year-and-a-half ago I wrote a review on Advanced Concepts in No-Limit Hold’em; A Modern Approach to Poker Analysis by Hunter Cichy which is a D&B Publishing book. On a scale of 1 to 10 with 10 being the best, I gave this book a 10.

But in many cases I don't think you do. Didn't I give an example in one of my posts above? Also, from the way the quote from the book was written, I can't help but wonder if the author understands statistical significance.

In the beginning of this book it states:

His background as a mathematician and expertise in process improvement helped him master the theoretical aspects of poker.

So, shouldn't his understanding of statistical theory, as it applies to poker, be very accurate?

And here's another example. Under the heading of "General Guidelines for Pre-flop Bet-sizing" this sentence appears:

Using solvers to test pre-flop bet sizing, I found that for similar bet-sizes, the differences in EV are quite small and even indifferent in many situations. For example, going from 2.25bb to a 2.3bb open size won't have a significant EV impact in your bottom line.

Do you really think he spent hours running solver programs to get this answer. Any competent math/stat with a good understanding of poker could have given you this answer instantly.

And by the way, on page 27 of No-Limit Hold 'em for Advanced Players by Matt Janda under the heading "A Note About the Bet Sizes" it says:

As a partial answer, when you look at statistical distributions their minimums and maximums tend to be broad. For example, instead of the maximum of a statistical distribution coming to a sharp peak where the ascent to the maximum is steep and the descent from the maximum is also steep, the (graphed) distribution will usually look more like a bell curve with a rounded top and moderate slopes on each side near the maximum point.

And this brings us to the point of this thread. Since this book came out, and actually a number of weeks before this book came out as you can see here:

https://forumserver.twoplustwo.com/s...59&postcount=7

I was reading comments as to how terrific Modern Poker Theory is.

As far as I can tell, and this could be wrong since I've only read about one-third of this book, the author runs solver programs to get answers to poker situations, and that's fine. But he often goes on to give explanations as to why the solver produced the answer it did, and I question some of these.

For instance under the heading of "Playing vs 3-bets: General Heuristics" the author writes:

You will get 3-bet a lot more often when opening UTG than when opening from the BN (despite your strong range) due to the number of players yet to act, and your opponents will have stronger ranges.

Is this true? And doesn't the last part of this sentence perhaps contradict the first part. Also, isn't this a straightforward math problem?

Mason

I only have equilibrium ranges for 6max but that should be enough to illustrate the point. Can someone run this math? I am unsure of how to calculate independent events.

LJ opens--->HJ 3bets 8.1% of the time
LJ opens--->CO 3bets 8.5% of the time
LJ opens--->BTN 3bets 7.5% of the time (it is less because BTN calls more)
LJ opens--->SB 3bets 7.6% of the time
LJ opens--->BB 3bets 6.0% of the time

LJ doesn't get 3bet 91.9% of the time
LJ doesn't get 3bet 91.5% of the time
LJ doesn't get 3bet 92.5% of the time
LJ doesn't get 3bet 92.4% of the time
LJ doesn't get 3bet 94.0% of the time

BTN opens--->SB 3bets 14.8% of the time
BTN opens--->BB 3bets 13.9% of the time

14.35% on average we get 3bet

Odds we do not get 3bet by SB-->85.2%
Odds we do not get 3bet by BB-->86.1%

85.65% on average we do not get 3bet

Okay I think I did this correctly someone please confirm:

Add up all the percentages and get an average.

So 8.1 +8.5 + 7.5 + 7.6 +6.0 divided by 5 = 7.54% chance on average

100-7.54 = 92.46% chance of not getting 3bet on average.

times 5 independent events so (92.46) to the 5th power = 67.57% chance of not getting 3bet

so 32.43% chance of getting 3bet when UTG in 6max game.

Button is 14.8% + 13.9%/2 = 14.35% of getting 3bet OR 85.65% of NOT getting 3bet.

so (85.65%) to the 2nd power = 73.35% chance of not getting 3bet

26.65% chance of getting 3bet when raising OTB
32.43% chance of getting 3bet when raising in the Lojack (UTG in 6max)

Someone please confirm i want to make sure I did this correctly

Your math is wrong. This is how you should be calculating it:

For BTN:

Ignoring blockers, BTN gets 3bet by SB 14.8% of the time.
The other 85.2% of the time that he doesn't get 3bet by SB, he still gets 3bet 13.9% of the time by BB (assuming SB never calls or that BB has the same squeezing % as 3b %).
13.9% of 85.2% is 11.84%. So when BTN opens, he gets 3bet by BB 11.84% of the time.
Therefore, he will overall get 3bet 11.84% + 14.8% = 26.64% of the time.

Use this same logic for other situations (like UTG) but don't forget about blockers and the fact that cold calls change the 3bet frequencies of players.

Also, the frequencies that you posted aren't equilibrium.

Okay thanks - but we came to the same conclusion.

I'm pretty sure there are multiple ways to figure this problem out - you just took a different approach.

My ranges are from Monker. Feel free to post more accurate ones.

Our approaches are very different. The results just happened to be similar.

Imagine if SB 3bets 0% and BB 3bets 28.7%. How often does the Button get 3bet?
28.7% obviously.

However, if you used your approach, you'd get the 26.65% result, which wouldn't be correct.

Okay thanks. Can you do the UTG one

I'm a bit confused by DooDoo's methods, but I get the same answers, albeit with a rounding error. *shrugs*.
I'm terrible at maths and probability, so maybe this method is wrong, but I just take this list:
and multiply all those percentages, (0.919 *0.915 * 0.925* 0.924 * 0.94) to get 0.6756
and then subtract it from 1 (or 100%).
1-0.6756 = 0.3244, or a 32.4% chance of being 3-bet when opening UTG.

For the BTN, I just multiply 85.2% and 86.1% (the probabilities of not getting 3-bet) to get 0.7336.
I subtract that from 1, and the result is 26.64%.

Hi Everyone:

Arty, using DooDoo's numbers, which I assume are based on solver results, has this done right, and we can see that at equilibrium (aka the saddle point) against a GTO opponent you'll be 3-bet more when opening from the early position than from the late position.

But it's my experience/belief, based on playing small stakes live no-limit, that the opposite actually happens. That is when you open from late position you'll be three-bet more than when you open early. I suspect that this has more to do with players at these live small stakes not three-betting the early position opener enough as opposed to them three-betting the late position player too much. But this is just my opinion.

Best wishes,
Mason

My experience with sslnl involves many tables where 3bets often have a range of AA-KK or a small stack shove that often prefers to squeeze a few callers.

I do not live in Las Vegas. I suspect that the player pool there has a higher percentage of players who are tight and willing to isolate. You probably see more players who take a fold-or-raise line in the small blind against a late-position open than I do. In one casino I play in, almost no player 3bets AK against a button raise.

This quote is a gem:

The set of functions that qualify as probability distributions/densities is really, really, really big and it's just silly to say most of them (and we're talking about continuous distributions right? Pretty sloppy to leave off that detail) when graphed look the way he's describing. He's ignoring skew and kurtosis.

I know he says "tend" and he's not intending to make a strong mathematical statement, but just for a really simple counterexample, what about the Uniform[0,1] distribution? How about Poisson(1)?


This is such BS because for one, yeah of course a difference of a mere 2% is not gonna have an appreciable difference on EV. What about 2x vs 3x? That's a much more valuable test.

And how the hell did he "solve" this problem in multiway spots? I wish he'd share this multiway GTO solver dream machine with me because I'm stuck with a HU only one (of course he doesn't actually have that software).

No mention of ranges? Antes or not? Stack sizes? Cash or MTT?


Not equilibrium, if you purchased these ranges under the impression they are true equilibrium ranges then you were misled.

The multiway solvers take all sorts of liberties to simplify the game tree (because even a 3-player preflop tree, with a reasonable postflop strategy profile, and a reasonable number of flop subsets is gonna be MASSIVE), and I do believe the creators of the multiway solvers even go as far as explicity saying they make no guarantee that their solutions are converging properly.

Not at all saying these ranges you have are bad, but caveat emptor.


This is definitely just pulled out of thin air.


Seems pretty silly to think anyone can ever know their true winrate.

In fact I'm not sure it's even statistically valid to think of winrate in any other terms except "greater than some hypothesis" or "less than some hypothesis" (i.e. you'd need a hypothesis test to determine that) but that's pretty tricky too because there is no way the underlying distribution of wins/losses is normal, it's probably not even close to normal (I believe the Kolmogorov-Smirnov Test does that), and I'm pretty sure you'd need some exotic hypothesis test. Can someone correct me if I'm wrong?

Do you know how to derive required sample size and if so would you be willing to quickly do that for me? I totally forgot how to do that!

Hi Everyone:

Here's another quote from this book, that came from under the heading of "Poker Applications," which from a theory perspective should be worth some discussion:

So, players who play a lot of volume will experience less variance than the occasional players.

There are also similar statements later in the book in the tournament discussion such as

Also, there aren't nearly as many high stakes online tournameents as there are low-mid stakes tournaments, so the volume a high stakes online MTT player can put in on a yearly basis is limited and that increases variance.

Mason

I think we know what the author is trying to say - something like the law of large numbers will lead us towards our actual win rate as our volume increases - he’s just not saying it in a nuanced way.

He has probably never studied statistics or game theory, which is consistent with most poker players who go around parroting terms like “variance” and “GTO” without having any real understanding of what the words actually mean.

In the “Michael Acevedo” page at the beginning of the book it says that he has a background as a mathematician.

I’ll come back and comment more specifically later, but I think this confusion is probably common among many poker players, so I think it’s worth addressing.

Best wishes,
Mason