This thread idea was inspired by bumpking's recent post, who raised interesting problems about all-in EV luck in his very large SNG sample, and was wondering if he was running abnormally bad. I also saw in the discussion a lot of misconceptions about luck and variance. I will try to settle things a bit, and show how you can analyze your luck statistics yourself.
This post is also an answer to some of bumpking's questions. I hope it will also contain enough information to answer all the questions asked about variance / all-in luck we keep going through here.
We'll also assume that 321 is a number special enough to make a milestone post
A related thead has been done by k4b4l in the past about how brutal variance is in SNGs :
http://forumserver.twoplustwo.com/36...riance-295301/
WARNING : Very long math / theory post. I'll try to keep this interesting till the end but there is no guarantee
There are three parts :
1 - What is Luck, and how to measure it
2 - The Flaws in All-in EV luck
3 - Some statistics applied to SNGs
The real math only lies in part 3.
Part One : What is Luck, and how to measure it
My dictionnary has it that way : Luck is the sum of events that happens beyond a person's control. In poker there are an incredible amount of things you don't control, and that take part in the outcome of a hand. Often you will hear :
- "that's lucky to get AA"
- "that's lucky to get KK"
- "that's unlucky to get KK and to have another player get AA"
- "that's lucky to crack his AA with your KK"
The AA vs KK where KK cracks AA postflop is an overly funny situation. Who was unlucky here ? The guy with AA who got pwned, or the guy with KK that picked it up just when AA was out ? And what if KK lost ? There could be endless arguments about it, and it shows how unaccurate our conception of luck is.
But actually who cares ? We don't play poker to get dealt AA the maximum number of times, or catch the maximum number of draws, or whatever.
We have to define luck in poker in relation to our objective : winning money. In the end what matters is how many dollars you have won, no matter how many times your AA were cracked by trash in a tournament, you can still end up running hot and taking it down. Can you still tell you were unlucky ? Hey you won the thing, no one has been more lucky overall this day probably. More funnily, not getting your AA cracked one time could have changed the course of events and make you lose earlier. But that's another story
So having said that, what would be the best definition, a mathematical one, of luck in poker ?
Luck over a sample of games, is the difference between the money you won over these games, minus the money you deserved to win.
I'm sure anyone will agree with that. There is a big catch though, it's calculating the money you deserved to win. There is only one way : playing a HUGE sample of SNGs, or cash hands if you are into cash and see how you perform on average. That way, you will be put in all kinds of poker situations, "lucky" and "unlucky" ones, and on a really large sample it will eventually even out, and we'll see what your skill is responsible of, what you deserve to win.
The catch is : the long run is way too long for almost eveyone. Later in this post, you will be able to calculate how many SNGs you have to play to have an accurate idea (within 1%) of the ROI you deserve : in the order of 100,000.
So that is our fate : we have to evaluate our luck a limited sample, but we have only that limited sample to estimate what we deserve on the long run !
It's still ok for small samples (say 200 SNGs), where you can use your sample of one entire year (say 5,000 SNGs) to check how the small sample went, but there is no way to know if you were lucky or not on your yearly sample, or at least, not accurately. There will be an uncertainty on the evaluation that depends on how many games you played. So that's it. To get things more complicated, you style of play is not constant over time, as is the toughness of games, so your true ROI varies as a function of time. This is simply unextricable.
Bottom Line : Work your game and play your best. Don't spend too much time wondering how good/bad you run. You can get a rough idea, but you'll never know the ultimate answer. So stop wondering and focus on what you can control in this game.
Part Two : The Flaws in All-in EV luck
We know that luck involved in a single poker hand is not mathematically measurable because of all it involves : the starting hand, you position, the people's betting, the flop, and so much more. But we can evaluate it in a very simple and specific case : all involved players push all-in preflop. Using ICM, we can even calculate by how many dollars you were lucky/unlucky on such a allin hand. Let's see how All-in EV programs work on a simple example.
9-man SNG, $100 buyin, 1500 starting stacks,
First hand. You are in the BB, blinds 15/30, and you have AA.
7 folds. SB raises to 120. You re-raise to 350. He pushes, you snapcall. He shows you KK, a K flops and you're out.
How unlucky was that eh ?
When the hand started everyone had the same ICM equity : $100 of course.
The one who loses this hand will end up with a $0 equity very obviously.
The one who wins it, according to an ICM calculator : $182.50
You are supposed to win 80% of the time. So your expected value in dollars, on this hand, i.e. what you "deserve" to win :
$EV = 80% * $182.50 + 20% * $0
$EV = $146
Luck is the difference between what you won and what you "deserved" to win :
$Luck_lose = $0 - $EV = -$146
So you lost $146 in bad luck according to ICM here. One funny thing before continuing, what if you won the hand ?
$Luck_win = $182.5 - $146 = +$36.5
Yes, you are considered lucky mathematically. Simply because you won while not being supposed to win 100% of the time. So remember to make a fistpump when your AA holds, because that won't be always that way
Now the thing is, suppose you won the hand. 99% of the people in this situation will consider they deserved it, AA held g00t, gg nh, onto the next hand. All-in EV calculator will credit you with a +$36.50 luck on this hand.
My opinion is you don't deserve it, because you made absolutely no use of your skill here. Anyone, even braindead, would have done the same, because hey, being dealt AA in the BB while the SB has KK is nothing less than a dream situation. The thing is, no program can consider accurately the preflop situation, among many other things of course.
If a program was able to consider all parameters involved in this hand, like the fact you just have to press the "call all-in" button and win because this is a superb setup for you, it would credit you with SIGNIFICANTLY MORE that +$36.50 luck. In this hand, your skill is not responsible of anything, only elements not in your control are.
Another extreme example would be that from the beginning of the pushbotting phase in the end of a SNG, you get AA every hand. All-in EV wise, you won't be that lucky, because you ARE almost supposed to win hand after hand. The program won't see you are picking up aces all the time. It also doesn't factor the luck involved when people are folding to your pushes because they got dealt crap hands.
And I'm not even talking about postflop play. Luck calculators won't see the fact your AA got five-outered on a K32 flop by KJ, or that you flopped a set on two overpairs and tripled up.
Is it therefore useless ? No. A significant part of your SNG profit comes from shown down push/fold hands so it's not as far off as one could think. It can give you a rough idea when you are running bad, and sometimes it can be reconforting. But it is really not able to evaluate accurately what you deserve to win on short samples. Actually, it is just not mathematically possible to evaluate anything accurately on a short sample.
Part Three : Some statistics applied to SNGs
This part is an answer to some of the discussion in that thread :
http://forumserver.twoplustwo.com/36...report-316556/
I think it has its place here.
Here we are going to see the standard method how to estimate how good/bad you run over a sample of SNGs.
First remember we can't know exactly what your long run results are. Usually we always start by making an hypothesis on what your finish distribution on the long run should look like, based on what you already played, what the very best players are able to do, and other more or less honest guesstimates about how good or bad you ran so far.
NOTE: all the following assumes implicitely your true ROI does not vary over time, which means your skill does not vary nor the toughness of games. All the calculations are easily adaptable using your own data if you wish.
We are going to use bumpking's large sample of $114 9-man SNGs. He played 17388 SNGs, and all-in EV program says he had a -$19,148 luck. We want to know how unlucky it is, that means answering the mathematical question : what is the probability to have an EV luck this bad, or worse, over a 17k sample ? Side question : zomg is it rigged ?
Let's assume he is a very good $114 player, with this being the finish distribution he deserves on the long run (about 6% ROI) :
1st - 13%
2nd - 13%
3rd - 12%
OOTM - 62%
So the profit (prize - (buyin + rake)) made in a SNG can be represented by a random variable X, that can take the follwing values depending on the finishing position :
x1 = 358.5 | x2 = 169.5 | x3 = 75 | x4 = -114
Associated with probabilities :
p1 = 0.13 | p2 = 0.13 | p3 = 0.12 | p4 = 0.62
Average profit in a SNG :
m = p1.x1 + p2.x2 + p3.x3 + p4.x4 = 6.96
Standard deviation of profit in a SNG :
s = SQRT( p1.(x1-m)² + ... + p4.(x4-m)² ) = 170.67
Now let's consider a large number N of SNGs. The profit made over N SNGs follows a gaussian distribution with the following parameters :
Mean: M = N * m = 121 020
Standard dev. : S = SQRT(N) * s = 22 504
This is a consequence of this if anyone cares :
http://en.wikipedia.org/wiki/Central_limit_theorem
So what do these numbers mean ?
- First that he is supposed to win an average M over these SNGs.
M contains only the information relative to his skill, or what he deserves to win. When luck is perfectly neutral on a sample, you win exactly the average profit. It is also called expected value for a reason.
- Second, S gives you an idea of the power of luck on such a sample.
S contains only the information relative to luck, or what he does not control in play. That includes (but not only) All-in EV luck.
Some orders of magnitude about the meaning of standard deviation S in a gaussian law :
- On a given sample of N SNGs, you have a 68% chance to profit within the interval [M - S ; M + S], or you can note this M +/-S.
- On a given sample of N SNGs, you have a 95% chance to profit within the interval [M - 2S ; M + 2S], or you can note this M +/-2S.
Give it a try on your data, and you'll see how high standard dev. still is after 2,000 SNGs.
Now let's answer the initial question :
EV_luck = -19 148
S = 22 504
So absolute value of luck is less than ONE standard deviation, this is perfectly normal. If you use a gaussian table (see wikipedia for more details, too long to explain here), or the right excel function, you will find that there is a 20% chance to run worse on this (large) sample. Symmetrically, there would be a 20% chance to have a luck higher than +$19k. Usually in most sciences, one considers an event to be really abnormal if it is out of THREE standard deviations from the mean. Here that would mean to have an EV_luck below -$67,000 to start worrying about games being rigged. And that would still NOT be a proof in itself.
NOTE : yes, all-in luck is just a part of the whole luck in poker, so one could say hey, he's down 19k in all-in EV luck, but he could be down much more overall, taking in account other aspects ? Yes, but he could be also up overall. We don't know. And we can't know. Here the only rigorous thing we can do is to compare the only measurable aspect of luck, with an order of magnitude of OVERALL luck, which is S. There is no reason to think that other aspects of luck are also negative, or positive for whatever reason.
NOTE 2 : This calculation about how normal all-in EV luck is, is not sensitive to the finish distribution used. Changing the finish distribution changes M for sure, but almost not S. Variance is your friend no matter how good or bad you play.
Thanks and congratulations to those who got this far (if there are any). By the way, thanks to the STTF community for making me suck much less than before at poker in the past few months