This thread idea was inspired by bumpking's recent post, who raised interesting problems about all-in EV luck in his very large SNG sample, and was wondering if he was running abnormally bad. I also saw in the discussion a lot of misconceptions about luck and variance. I will try to settle things a bit, and show how you can analyze your luck statistics yourself.

This post is also an answer to some of bumpking's questions. I hope it will also contain enough information to answer all the questions asked about variance / all-in luck we keep going through here.

We'll also assume that 321 is a number special enough to make a milestone post

A related thead has been done by k4b4l in the past about how brutal variance is in SNGs :
http://forumserver.twoplustwo.com/36...riance-295301/


WARNING : Very long math / theory post. I'll try to keep this interesting till the end but there is no guarantee

There are three parts :
1 - What is Luck, and how to measure it
2 - The Flaws in All-in EV luck
3 - Some statistics applied to SNGs

The real math only lies in part 3.


Part One : What is Luck, and how to measure it


My dictionnary has it that way : Luck is the sum of events that happens beyond a person's control. In poker there are an incredible amount of things you don't control, and that take part in the outcome of a hand. Often you will hear :
- "that's lucky to get AA"
- "that's lucky to get KK"
- "that's unlucky to get KK and to have another player get AA"
- "that's lucky to crack his AA with your KK"

The AA vs KK where KK cracks AA postflop is an overly funny situation. Who was unlucky here ? The guy with AA who got pwned, or the guy with KK that picked it up just when AA was out ? And what if KK lost ? There could be endless arguments about it, and it shows how unaccurate our conception of luck is.

But actually who cares ? We don't play poker to get dealt AA the maximum number of times, or catch the maximum number of draws, or whatever. We have to define luck in poker in relation to our objective : winning money. In the end what matters is how many dollars you have won, no matter how many times your AA were cracked by trash in a tournament, you can still end up running hot and taking it down. Can you still tell you were unlucky ? Hey you won the thing, no one has been more lucky overall this day probably. More funnily, not getting your AA cracked one time could have changed the course of events and make you lose earlier. But that's another story

So having said that, what would be the best definition, a mathematical one, of luck in poker ?

Luck over a sample of games, is the difference between the money you won over these games, minus the money you deserved to win.

I'm sure anyone will agree with that. There is a big catch though, it's calculating the money you deserved to win. There is only one way : playing a HUGE sample of SNGs, or cash hands if you are into cash and see how you perform on average. That way, you will be put in all kinds of poker situations, "lucky" and "unlucky" ones, and on a really large sample it will eventually even out, and we'll see what your skill is responsible of, what you deserve to win.
The catch is : the long run is way too long for almost eveyone. Later in this post, you will be able to calculate how many SNGs you have to play to have an accurate idea (within 1%) of the ROI you deserve : in the order of 100,000.

So that is our fate : we have to evaluate our luck a limited sample, but we have only that limited sample to estimate what we deserve on the long run !
It's still ok for small samples (say 200 SNGs), where you can use your sample of one entire year (say 5,000 SNGs) to check how the small sample went, but there is no way to know if you were lucky or not on your yearly sample, or at least, not accurately. There will be an uncertainty on the evaluation that depends on how many games you played. So that's it. To get things more complicated, you style of play is not constant over time, as is the toughness of games, so your true ROI varies as a function of time. This is simply unextricable.

Bottom Line : Work your game and play your best. Don't spend too much time wondering how good/bad you run. You can get a rough idea, but you'll never know the ultimate answer. So stop wondering and focus on what you can control in this game.


Part Two : The Flaws in All-in EV luck


We know that luck involved in a single poker hand is not mathematically measurable because of all it involves : the starting hand, you position, the people's betting, the flop, and so much more. But we can evaluate it in a very simple and specific case : all involved players push all-in preflop. Using ICM, we can even calculate by how many dollars you were lucky/unlucky on such a allin hand. Let's see how All-in EV programs work on a simple example.

9-man SNG, $100 buyin, 1500 starting stacks,
First hand. You are in the BB, blinds 15/30, and you have AA.

7 folds. SB raises to 120. You re-raise to 350. He pushes, you snapcall. He shows you KK, a K flops and you're out.

How unlucky was that eh ?

When the hand started everyone had the same ICM equity : $100 of course.
The one who loses this hand will end up with a $0 equity very obviously.
The one who wins it, according to an ICM calculator : $182.50

You are supposed to win 80% of the time. So your expected value in dollars, on this hand, i.e. what you "deserve" to win :

$EV = 80% * $182.50 + 20% * $0
$EV = $146

Luck is the difference between what you won and what you "deserved" to win :
$Luck_lose = $0 - $EV = -$146

So you lost $146 in bad luck according to ICM here. One funny thing before continuing, what if you won the hand ?
$Luck_win = $182.5 - $146 = +$36.5

Yes, you are considered lucky mathematically. Simply because you won while not being supposed to win 100% of the time. So remember to make a fistpump when your AA holds, because that won't be always that way

Now the thing is, suppose you won the hand. 99% of the people in this situation will consider they deserved it, AA held g00t, gg nh, onto the next hand. All-in EV calculator will credit you with a +$36.50 luck on this hand. My opinion is you don't deserve it, because you made absolutely no use of your skill here. Anyone, even braindead, would have done the same, because hey, being dealt AA in the BB while the SB has KK is nothing less than a dream situation. The thing is, no program can consider accurately the preflop situation, among many other things of course.
If a program was able to consider all parameters involved in this hand, like the fact you just have to press the "call all-in" button and win because this is a superb setup for you, it would credit you with SIGNIFICANTLY MORE that +$36.50 luck. In this hand, your skill is not responsible of anything, only elements not in your control are.

Another extreme example would be that from the beginning of the pushbotting phase in the end of a SNG, you get AA every hand. All-in EV wise, you won't be that lucky, because you ARE almost supposed to win hand after hand. The program won't see you are picking up aces all the time. It also doesn't factor the luck involved when people are folding to your pushes because they got dealt crap hands.

And I'm not even talking about postflop play. Luck calculators won't see the fact your AA got five-outered on a K32 flop by KJ, or that you flopped a set on two overpairs and tripled up.

Is it therefore useless ? No. A significant part of your SNG profit comes from shown down push/fold hands so it's not as far off as one could think. It can give you a rough idea when you are running bad, and sometimes it can be reconforting. But it is really not able to evaluate accurately what you deserve to win on short samples. Actually, it is just not mathematically possible to evaluate anything accurately on a short sample.



Part Three : Some statistics applied to SNGs



This part is an answer to some of the discussion in that thread :
http://forumserver.twoplustwo.com/36...report-316556/

I think it has its place here.

Here we are going to see the standard method how to estimate how good/bad you run over a sample of SNGs.
First remember we can't know exactly what your long run results are. Usually we always start by making an hypothesis on what your finish distribution on the long run should look like, based on what you already played, what the very best players are able to do, and other more or less honest guesstimates about how good or bad you ran so far.
NOTE: all the following assumes implicitely your true ROI does not vary over time, which means your skill does not vary nor the toughness of games. All the calculations are easily adaptable using your own data if you wish.

We are going to use bumpking's large sample of $114 9-man SNGs. He played 17388 SNGs, and all-in EV program says he had a -$19,148 luck. We want to know how unlucky it is, that means answering the mathematical question : what is the probability to have an EV luck this bad, or worse, over a 17k sample ? Side question : zomg is it rigged ?

Let's assume he is a very good $114 player, with this being the finish distribution he deserves on the long run (about 6% ROI) :
1st - 13%
2nd - 13%
3rd - 12%
OOTM - 62%

So the profit (prize - (buyin + rake)) made in a SNG can be represented by a random variable X, that can take the follwing values depending on the finishing position :
x1 = 358.5 | x2 = 169.5 | x3 = 75 | x4 = -114

Associated with probabilities :
p1 = 0.13 | p2 = 0.13 | p3 = 0.12 | p4 = 0.62

Average profit in a SNG :
m = p1.x1 + p2.x2 + p3.x3 + p4.x4 = 6.96

Standard deviation of profit in a SNG :
s = SQRT( p1.(x1-m)² + ... + p4.(x4-m)² ) = 170.67


Now let's consider a large number N of SNGs. The profit made over N SNGs follows a gaussian distribution with the following parameters :

Mean: M = N * m = 121 020
Standard dev. : S = SQRT(N) * s = 22 504
This is a consequence of this if anyone cares : http://en.wikipedia.org/wiki/Central_limit_theorem

So what do these numbers mean ?

- First that he is supposed to win an average M over these SNGs. M contains only the information relative to his skill, or what he deserves to win. When luck is perfectly neutral on a sample, you win exactly the average profit. It is also called expected value for a reason.
- Second, S gives you an idea of the power of luck on such a sample. S contains only the information relative to luck, or what he does not control in play. That includes (but not only) All-in EV luck.

Some orders of magnitude about the meaning of standard deviation S in a gaussian law :
- On a given sample of N SNGs, you have a 68% chance to profit within the interval [M - S ; M + S], or you can note this M +/-S.
- On a given sample of N SNGs, you have a 95% chance to profit within the interval [M - 2S ; M + 2S], or you can note this M +/-2S.

Give it a try on your data, and you'll see how high standard dev. still is after 2,000 SNGs.

Now let's answer the initial question :
EV_luck = -19 148
S = 22 504

So absolute value of luck is less than ONE standard deviation, this is perfectly normal. If you use a gaussian table (see wikipedia for more details, too long to explain here), or the right excel function, you will find that there is a 20% chance to run worse on this (large) sample. Symmetrically, there would be a 20% chance to have a luck higher than +$19k. Usually in most sciences, one considers an event to be really abnormal if it is out of THREE standard deviations from the mean. Here that would mean to have an EV_luck below -$67,000 to start worrying about games being rigged. And that would still NOT be a proof in itself.

NOTE : yes, all-in luck is just a part of the whole luck in poker, so one could say hey, he's down 19k in all-in EV luck, but he could be down much more overall, taking in account other aspects ? Yes, but he could be also up overall. We don't know. And we can't know. Here the only rigorous thing we can do is to compare the only measurable aspect of luck, with an order of magnitude of OVERALL luck, which is S. There is no reason to think that other aspects of luck are also negative, or positive for whatever reason.

NOTE 2 : This calculation about how normal all-in EV luck is, is not sensitive to the finish distribution used. Changing the finish distribution changes M for sure, but almost not S. Variance is your friend no matter how good or bad you play.


Thanks and congratulations to those who got this far (if there are any). By the way, thanks to the STTF community for making me suck much less than before at poker in the past few months

Who nice post, post it please

Doesn't LC standard for low content? Great post, will tackle the maths bit after I've had morning coffee.

OMG, seems like there has been so many great posts lately, I'm unable to read and understand them all, please slow down

Probably a great 321st post, I will confirm this WE when I have time to read it all.

My head now hurts , but hey .. very nice post !

Hats off to you sir !

OK, read a large part of it.

Don't you think All-in EV luck ROI should still give us a better approximation of our theorical (?) ROI than our current ROI on a little to not huge sample? It reduces the part of randomness in the calculations. When I run it on tiny samples, my All-in luck ROI is almost always much closer of my overall ROI than my ROI on the tiny sample.

Real question is how many SNG's would it take to have the same approximation of your theorical SNG with the All-in EV luck calculator as your current ROI with 1K games for example...

My 1K post could talk about that if I manage to work it out on a big sample.

Amazing post, well-written, pleasant to read and easy to understand (I mean, explanations are well justified).

Very true, I agree 100%!

As you can see, there are

Keep up the good work! (thanks to retam for the translation of this sentence!)

Cant wait to see post #4321!

**** you stole my 1/2k post >

I don't agree.
I think the ROI that All-in EV luck programs give us isn't a good approximation of our theorical ROI. Because All-in EV luck make his calculations based on the cards that you and the villians showed. But to get our theorical ROI about our push/fold skills, calculations must be based on HRs you push/call/get called by villians etc...
Don't you agree?

I'm going to take an exemple:
You are on the bubble and pushing ATC from SB into the BB which is ridicously tight on the bubble. Your move is really $EV+, and this situation occurs a lot in low stakes SnG for exemple. But All-in EV luck will only take into account the hand when you are called, and BB will call you w/ a HR that crushes yours. So All-in EV luck program will say that you should lose a lot of money in that spot, and this is going to lower your All-in EV ROI, but not your theorical ROI.

I hope what I've written is comprehensible.

Thanks for the positive replies guys.

Yes that would be called a Skill $EV calculator and it would be much more useful. Though it will never exist because no program can estimate hand ranges like a players' mind.

No, simply because when you lose a hand you are always considered unlucky, even if you pushed 32o into AA. Simply because you aren't supposed to lose 100% of the time. Let's rather say you will be considered especially lucky if you win. I should take time to dig into what you said (it might take another looong post), but I don't think that in the end it does interfere negatively as you say. And also don't forget the fact that when he does fold, even if it is most of his HR, you are slightly lucky if your hand is crap like 74o (and unlucky if it is AA), and the program does not see that.

Well it might take much more than that to prove my point, but I'm least I'm sharing it quickly.

So on average, the all in EV correction gets you closer to your true ROI. It's just that many other aspects of luck aren't accounted for, so don't take the results for super accurate. I mean if it shows :

- ROI on the sample : -37%
- Adjusted ROI : -23%

You are not that much of a crappy player, just getting coolered in other aspects than all-in luck.

Or :

- ROI on the sample : 31%
- Adjusted ROI : 26%

Don't think "oh well, this is normal". You are not god gift's to poker.
As for less obvious and more quantitative statements, well, I don't think there are any.

Thanks again CJ,

I think we should keep this all in the same thread since your post mirrors the on in the original thread. I think your calculations are correct, but your assumptins are wrong. Here's the original thread and my question:

http://forumserver.twoplustwo.com/36...ml#post6545956

Thank you once again for all your work,
-BK

Great post. Head asplode.

Seemed clearly-written to me. I'm not much of a statistician though. Are you Irieguy's new gimmick account?

http://archives2.twoplustwo.com/show...b=5&o=0&fpart=


It would be awesome if someone could explain this, but it seems like an immensely-difficult problem to solve because you have to quantify the fraction of your total "luck" in the game your hot/cold all-in luck is.



It will make the expectation negative, but it won't change how the +-luck works. Back when the graphical luck analyzer worked, you could do stuff like this...

pushes


calls


Note the slopes of the purple lines.

you guys sure talk about variance a lot

i don't need a weatherman to tell me which way the wind blows

Thanks for the further explanations Slim Pickens.

I have my idea about how it could work, I don't think we should necessarily quantify this fraction of luck represented by all-in luck. I'm afraid my own sample wouldn't be large enough to be really useful though. Maybe I will wait until my 2K post ^^

Hey CJ,

I've been toying with some of this myself lately (as it seems to be a hot topic). I noticed however that you used a Gaussian distribution (Normal) to compute SDs and confidence intervals. Can you please explain to me why this is appropriate for this type of data? Because to the best of my understanding tournament winnings are seriously non-normally distributed. Thanks.

Sherman

Edit: FWIW, I no almost nothing about handling data from non-normal distributions. That is, in non-normal distributions such as tournament winnings (with heavy skews to the left) do your typical formulas for a mean (sum of the Xs / N) and standard deviation (average of the squared mean deviations) still apply? Further, do inferential statistics like 95% confidence intervals apply?

This is clearly one of the top 10st 321th posts ever!

IIRC there is a good post somewhere deep in the archives where some smart people decided that as long as the samples were at least 20-35 SNGs, the results for those samples should be normally distributed.

Meh, it's written in OP :

http://en.wikipedia.org/wiki/Central_limit_theorem

"The central limit theorem (CLT) states that the sum of a sufficiently large number of identically distributed independent random variables, each with finite mean and variance, will be approximately normally distributed"

In other words :
Winnings on one tournament is a crappy random variable, but the sum of winnings on a sufficient number of identical tournaments is a gaussian random variable.

As slim said, winnings on ~30 or more 9-man SNGs are already close to gaussian distributed. If you consider, say 45-man SNGs, you might need significantly more.

Hi CJ,

Thanks for replying. But I still wonder how this could ever be. For 9 person SNGs I can understand how the winnings might approximate normality. A winning player is getting in the $ more than 33% of the time. But in large tournaments, typically only 10% of the field gets paid. That means a very good player might get in the money only 15% of the time. Additionally, the payouts are more severely weighted toward the top. This means that no matter how many tounaments are played (N) the left hand side of the ROI distribution will account for .85*N data points.

Maybe it is easier to think of it this way...if I play the lottery 100 Billion times will my results be normally distributed? If so, I don't understand how.

Sherman

I'm glad CJ pointed out that "luck" is not restricted to "all-in luck", as hood mentioned in the thread that sparked this post. I'd love for someone to write a HH analyser that told you how you are running against expectation for hands dealt. I mean, you're just as "lucky" if you get dealt top 10% hands 15% of the time as you are if you get on the right side of allins. And of course you are "lucky" if you pick up AA and someone else picks up something they'll give you action with, and "unlucky" those times you raise and all fold, because your expectation with AA would be higher than just the blinds.

Part of the reason I have been thinking about this is that Jukofyork's analyser has me +luck over a stretch of games that I felt had been very bad for me. But a lot of my "luck" in those games had been things like I have QQ, the other guy has A4, the flop is 532. When I get it all in there, I expect to lose a lot, so I haven't suffered bad allin luck. Or I am card dead for a whole tourney, pick up something like TT, first hand I play, shove it and get snapcalled by QQ. So I don't suffer bad allin luck, but going 30 hands without picking up something playable is "unlucky" too.

Yep, I agree this would be great in theory, but the problem is just too hard to really make anything work. It would basically require making models of the other player's behaviour for all but the "play themselves" AAvsKK type hands.


I don't really agree with Hood that luck-adjusted results can't be useful though. The main way that it's helped me is to figure out much more quickly what limits I should be playing to have the highest possible $s/hour.

Yes, I agree that if I spent even more time working on my game then I could ultimately make more money at higher limits, but I feel that I'm now at the stage where it would require an exponential amount of (sustained) effort to get there (ie: Scotty's intense level of note taking, etc), and after seeing my beloved LHE die a horrible death, I'm much more interested in milking the current games for as much as I can without worrying too much about moving up (I'm probably in the minority here though...). This for me is where the SNG luck thing really comes into it's own, as I can now get a fairly good idea of my true ROI from far fewer games than I could without it.

Also, after some sustained experiments over the last few months, I think I've finally found a better "table rejection" strategy than I used to have based on the number of regulars sitting (ie: balance wasting too much time rejecting tables and not playing vs playing in reg-infested games with a lower ROI). I could never had done this without it as I'd still be sitting here wondering if the perceived ROI change was real or or just down to variance.

Juk

Nice post CJ.

There are so many different ways one could attempt to measure luck on a per hand basis. I don't know how you would weight this for it to measure things accurately. For example, is it luckier to win a hand where your tourney life is on the stake? Or is is just as lucky win from BB at 15/30 and your 36o 'get in there' for t90? Are you lucky if you get dealt a top heavy distribution of hands? Are you unlucky if you lose an uneven distribution with those same hands?

I think the most significant factor of 'luck' from the poker players perspective as OP pointed out is tourney ROI, and what is an acceptable minimum sample to know how well you are running.

Well now that I have spoken well outside of my pay grade I'm going to attempt to be a lucky ******* in the micro-stakes.