This thread idea was inspired by bumpking's recent post, who raised interesting problems about all-in EV luck in his very large SNG sample, and was wondering if he was running abnormally bad. I also saw in the discussion a lot of misconceptions about luck and variance. I will try to settle things a bit, and show how you can analyze your luck statistics yourself.

This post is also an answer to some of bumpking's questions. I hope it will also contain enough information to answer all the questions asked about variance / all-in luck we keep going through here.

We'll also assume that 321 is a number special enough to make a milestone post

A related thead has been done by k4b4l in the past about how brutal variance is in SNGs :
http://forumserver.twoplustwo.com/36...riance-295301/


WARNING : Very long math / theory post. I'll try to keep this interesting till the end but there is no guarantee

There are three parts :
1 - What is Luck, and how to measure it
2 - The Flaws in All-in EV luck
3 - Some statistics applied to SNGs

The real math only lies in part 3.


Part One : What is Luck, and how to measure it


My dictionnary has it that way : Luck is the sum of events that happens beyond a person's control. In poker there are an incredible amount of things you don't control, and that take part in the outcome of a hand. Often you will hear :
- "that's lucky to get AA"
- "that's lucky to get KK"
- "that's unlucky to get KK and to have another player get AA"
- "that's lucky to crack his AA with your KK"

The AA vs KK where KK cracks AA postflop is an overly funny situation. Who was unlucky here ? The guy with AA who got pwned, or the guy with KK that picked it up just when AA was out ? And what if KK lost ? There could be endless arguments about it, and it shows how unaccurate our conception of luck is.

But actually who cares ? We don't play poker to get dealt AA the maximum number of times, or catch the maximum number of draws, or whatever. We have to define luck in poker in relation to our objective : winning money. In the end what matters is how many dollars you have won, no matter how many times your AA were cracked by trash in a tournament, you can still end up running hot and taking it down. Can you still tell you were unlucky ? Hey you won the thing, no one has been more lucky overall this day probably. More funnily, not getting your AA cracked one time could have changed the course of events and make you lose earlier. But that's another story

So having said that, what would be the best definition, a mathematical one, of luck in poker ?

Luck over a sample of games, is the difference between the money you won over these games, minus the money you deserved to win.

I'm sure anyone will agree with that. There is a big catch though, it's calculating the money you deserved to win. There is only one way : playing a HUGE sample of SNGs, or cash hands if you are into cash and see how you perform on average. That way, you will be put in all kinds of poker situations, "lucky" and "unlucky" ones, and on a really large sample it will eventually even out, and we'll see what your skill is responsible of, what you deserve to win.
The catch is : the long run is way too long for almost eveyone. Later in this post, you will be able to calculate how many SNGs you have to play to have an accurate idea (within 1%) of the ROI you deserve : in the order of 100,000.

So that is our fate : we have to evaluate our luck a limited sample, but we have only that limited sample to estimate what we deserve on the long run !
It's still ok for small samples (say 200 SNGs), where you can use your sample of one entire year (say 5,000 SNGs) to check how the small sample went, but there is no way to know if you were lucky or not on your yearly sample, or at least, not accurately. There will be an uncertainty on the evaluation that depends on how many games you played. So that's it. To get things more complicated, you style of play is not constant over time, as is the toughness of games, so your true ROI varies as a function of time. This is simply unextricable.

Bottom Line : Work your game and play your best. Don't spend too much time wondering how good/bad you run. You can get a rough idea, but you'll never know the ultimate answer. So stop wondering and focus on what you can control in this game.


Part Two : The Flaws in All-in EV luck


We know that luck involved in a single poker hand is not mathematically measurable because of all it involves : the starting hand, you position, the people's betting, the flop, and so much more. But we can evaluate it in a very simple and specific case : all involved players push all-in preflop. Using ICM, we can even calculate by how many dollars you were lucky/unlucky on such a allin hand. Let's see how All-in EV programs work on a simple example.

9-man SNG, $100 buyin, 1500 starting stacks,
First hand. You are in the BB, blinds 15/30, and you have AA.

7 folds. SB raises to 120. You re-raise to 350. He pushes, you snapcall. He shows you KK, a K flops and you're out.

How unlucky was that eh ?

When the hand started everyone had the same ICM equity : $100 of course.
The one who loses this hand will end up with a $0 equity very obviously.
The one who wins it, according to an ICM calculator : $182.50

You are supposed to win 80% of the time. So your expected value in dollars, on this hand, i.e. what you "deserve" to win :

$EV = 80% * $182.50 + 20% * $0
$EV = $146

Luck is the difference between what you won and what you "deserved" to win :
$Luck_lose = $0 - $EV = -$146

So you lost $146 in bad luck according to ICM here. One funny thing before continuing, what if you won the hand ?
$Luck_win = $182.5 - $146 = +$36.5

Yes, you are considered lucky mathematically. Simply because you won while not being supposed to win 100% of the time. So remember to make a fistpump when your AA holds, because that won't be always that way

Now the thing is, suppose you won the hand. 99% of the people in this situation will consider they deserved it, AA held g00t, gg nh, onto the next hand. All-in EV calculator will credit you with a +$36.50 luck on this hand. My opinion is you don't deserve it, because you made absolutely no use of your skill here. Anyone, even braindead, would have done the same, because hey, being dealt AA in the BB while the SB has KK is nothing less than a dream situation. The thing is, no program can consider accurately the preflop situation, among many other things of course.
If a program was able to consider all parameters involved in this hand, like the fact you just have to press the "call all-in" button and win because this is a superb setup for you, it would credit you with SIGNIFICANTLY MORE that +$36.50 luck. In this hand, your skill is not responsible of anything, only elements not in your control are.

Another extreme example would be that from the beginning of the pushbotting phase in the end of a SNG, you get AA every hand. All-in EV wise, you won't be that lucky, because you ARE almost supposed to win hand after hand. The program won't see you are picking up aces all the time. It also doesn't factor the luck involved when people are folding to your pushes because they got dealt crap hands.

And I'm not even talking about postflop play. Luck calculators won't see the fact your AA got five-outered on a K32 flop by KJ, or that you flopped a set on two overpairs and tripled up.

Is it therefore useless ? No. A significant part of your SNG profit comes from shown down push/fold hands so it's not as far off as one could think. It can give you a rough idea when you are running bad, and sometimes it can be reconforting. But it is really not able to evaluate accurately what you deserve to win on short samples. Actually, it is just not mathematically possible to evaluate anything accurately on a short sample.



Part Three : Some statistics applied to SNGs



This part is an answer to some of the discussion in that thread :
http://forumserver.twoplustwo.com/36...report-316556/

I think it has its place here.

Here we are going to see the standard method how to estimate how good/bad you run over a sample of SNGs.
First remember we can't know exactly what your long run results are. Usually we always start by making an hypothesis on what your finish distribution on the long run should look like, based on what you already played, what the very best players are able to do, and other more or less honest guesstimates about how good or bad you ran so far.
NOTE: all the following assumes implicitely your true ROI does not vary over time, which means your skill does not vary nor the toughness of games. All the calculations are easily adaptable using your own data if you wish.

We are going to use bumpking's large sample of $114 9-man SNGs. He played 17388 SNGs, and all-in EV program says he had a -$19,148 luck. We want to know how unlucky it is, that means answering the mathematical question : what is the probability to have an EV luck this bad, or worse, over a 17k sample ? Side question : zomg is it rigged ?

Let's assume he is a very good $114 player, with this being the finish distribution he deserves on the long run (about 6% ROI) :
1st - 13%
2nd - 13%
3rd - 12%
OOTM - 62%

So the profit (prize - (buyin + rake)) made in a SNG can be represented by a random variable X, that can take the follwing values depending on the finishing position :
x1 = 358.5 | x2 = 169.5 | x3 = 75 | x4 = -114

Associated with probabilities :
p1 = 0.13 | p2 = 0.13 | p3 = 0.12 | p4 = 0.62

Average profit in a SNG :
m = p1.x1 + p2.x2 + p3.x3 + p4.x4 = 6.96

Standard deviation of profit in a SNG :
s = SQRT( p1.(x1-m)² + ... + p4.(x4-m)² ) = 170.67


Now let's consider a large number N of SNGs. The profit made over N SNGs follows a gaussian distribution with the following parameters :

Mean: M = N * m = 121 020
Standard dev. : S = SQRT(N) * s = 22 504
This is a consequence of this if anyone cares : http://en.wikipedia.org/wiki/Central_limit_theorem

So what do these numbers mean ?

- First that he is supposed to win an average M over these SNGs. M contains only the information relative to his skill, or what he deserves to win. When luck is perfectly neutral on a sample, you win exactly the average profit. It is also called expected value for a reason.
- Second, S gives you an idea of the power of luck on such a sample. S contains only the information relative to luck, or what he does not control in play. That includes (but not only) All-in EV luck.

Some orders of magnitude about the meaning of standard deviation S in a gaussian law :
- On a given sample of N SNGs, you have a 68% chance to profit within the interval [M - S ; M + S], or you can note this M +/-S.
- On a given sample of N SNGs, you have a 95% chance to profit within the interval [M - 2S ; M + 2S], or you can note this M +/-2S.

Give it a try on your data, and you'll see how high standard dev. still is after 2,000 SNGs.

Now let's answer the initial question :
EV_luck = -19 148
S = 22 504

So absolute value of luck is less than ONE standard deviation, this is perfectly normal. If you use a gaussian table (see wikipedia for more details, too long to explain here), or the right excel function, you will find that there is a 20% chance to run worse on this (large) sample. Symmetrically, there would be a 20% chance to have a luck higher than +$19k. Usually in most sciences, one considers an event to be really abnormal if it is out of THREE standard deviations from the mean. Here that would mean to have an EV_luck below -$67,000 to start worrying about games being rigged. And that would still NOT be a proof in itself.

NOTE : yes, all-in luck is just a part of the whole luck in poker, so one could say hey, he's down 19k in all-in EV luck, but he could be down much more overall, taking in account other aspects ? Yes, but he could be also up overall. We don't know. And we can't know. Here the only rigorous thing we can do is to compare the only measurable aspect of luck, with an order of magnitude of OVERALL luck, which is S. There is no reason to think that other aspects of luck are also negative, or positive for whatever reason.

NOTE 2 : This calculation about how normal all-in EV luck is, is not sensitive to the finish distribution used. Changing the finish distribution changes M for sure, but almost not S. Variance is your friend no matter how good or bad you play.


Thanks and congratulations to those who got this far (if there are any). By the way, thanks to the STTF community for making me suck much less than before at poker in the past few months

Who nice post, post it please

Doesn't LC standard for low content? Great post, will tackle the maths bit after I've had morning coffee.

OMG, seems like there has been so many great posts lately, I'm unable to read and understand them all, please slow down

Probably a great 321st post, I will confirm this WE when I have time to read it all.

My head now hurts , but hey .. very nice post !

Hats off to you sir !

OK, read a large part of it.

Don't you think All-in EV luck ROI should still give us a better approximation of our theorical (?) ROI than our current ROI on a little to not huge sample? It reduces the part of randomness in the calculations. When I run it on tiny samples, my All-in luck ROI is almost always much closer of my overall ROI than my ROI on the tiny sample.

Real question is how many SNG's would it take to have the same approximation of your theorical SNG with the All-in EV luck calculator as your current ROI with 1K games for example...

My 1K post could talk about that if I manage to work it out on a big sample.

Amazing post, well-written, pleasant to read and easy to understand (I mean, explanations are well justified).

Very true, I agree 100%!

As you can see, there are

Keep up the good work! (thanks to retam for the translation of this sentence!)

Cant wait to see post #4321!

**** you stole my 1/2k post >

I don't agree.
I think the ROI that All-in EV luck programs give us isn't a good approximation of our theorical ROI. Because All-in EV luck make his calculations based on the cards that you and the villians showed. But to get our theorical ROI about our push/fold skills, calculations must be based on HRs you push/call/get called by villians etc...
Don't you agree?

I'm going to take an exemple:
You are on the bubble and pushing ATC from SB into the BB which is ridicously tight on the bubble. Your move is really $EV+, and this situation occurs a lot in low stakes SnG for exemple. But All-in EV luck will only take into account the hand when you are called, and BB will call you w/ a HR that crushes yours. So All-in EV luck program will say that you should lose a lot of money in that spot, and this is going to lower your All-in EV ROI, but not your theorical ROI.

I hope what I've written is comprehensible.

Thanks for the positive replies guys.

Yes that would be called a Skill $EV calculator and it would be much more useful. Though it will never exist because no program can estimate hand ranges like a players' mind.

No, simply because when you lose a hand you are always considered unlucky, even if you pushed 32o into AA. Simply because you aren't supposed to lose 100% of the time. Let's rather say you will be considered especially lucky if you win. I should take time to dig into what you said (it might take another looong post), but I don't think that in the end it does interfere negatively as you say. And also don't forget the fact that when he does fold, even if it is most of his HR, you are slightly lucky if your hand is crap like 74o (and unlucky if it is AA), and the program does not see that.

Well it might take much more than that to prove my point, but I'm least I'm sharing it quickly.

So on average, the all in EV correction gets you closer to your true ROI. It's just that many other aspects of luck aren't accounted for, so don't take the results for super accurate. I mean if it shows :

- ROI on the sample : -37%
- Adjusted ROI : -23%

You are not that much of a crappy player, just getting coolered in other aspects than all-in luck.

Or :

- ROI on the sample : 31%
- Adjusted ROI : 26%

Don't think "oh well, this is normal". You are not god gift's to poker.
As for less obvious and more quantitative statements, well, I don't think there are any.

Thanks again CJ,

I think we should keep this all in the same thread since your post mirrors the on in the original thread. I think your calculations are correct, but your assumptins are wrong. Here's the original thread and my question:

http://forumserver.twoplustwo.com/36...ml#post6545956

Thank you once again for all your work,
-BK

Great post. Head asplode.

Seemed clearly-written to me. I'm not much of a statistician though. Are you Irieguy's new gimmick account?

http://archives2.twoplustwo.com/show...b=5&o=0&fpart=


It would be awesome if someone could explain this, but it seems like an immensely-difficult problem to solve because you have to quantify the fraction of your total "luck" in the game your hot/cold all-in luck is.



It will make the expectation negative, but it won't change how the +-luck works. Back when the graphical luck analyzer worked, you could do stuff like this...

pushes


calls


Note the slopes of the purple lines.

you guys sure talk about variance a lot

i don't need a weatherman to tell me which way the wind blows