Warning: This is a thinking outside the box post. Feel free to leave this thread if you are close minded

I know that not even 10,000 hands played is a good sample size to figure out your real BB/100. This is unfortunate because a live player would have to play something close to 400 hours to know how well they actually do. I've played 50 hours and have played 1250 hands of live cash game so far. I'm not interested in my specific BB/100, I am just trying to figure out how to calculate if I am a winning player or not. So I have done some calculations, and wanted to know your views on this (aka, if I'm missing something in my calculations then let me know. Well here it goes.

200 NL ($1/$2 blinds)
Total Hands: 1250
Avg BB/100: 16.88

Individual Session BB/100 (Sessions ranging from 3.5 to 6 hours)
24.67
(5.25)
38.25
(33.33)
10.89
14.75
13.25
33.00
3.80
64.57
(50.00)
81.50

Standard Deviation
7.786666667
-22.13
21.37
-50.21333333
-5.991111111
-2.13
-3.63
16.12
-13.08
47.69142857
-66.88
64.62

Deviations ^2
60.63217778
489.7369
456.6769
2521.378844
35.89341235
4.5369
13.1769
259.8544
171.0864
2274.472359
4472.9344
4175.7444
14936.12399 <-Sum of Deviations ^2

N = 12 Sessions
Sum of Deviations / N = 35.27998015 <- Standard Deviation

Standard Deviation/SQRT(N) = 10.18445302 <- Standard Error

Given a 95% confidence
Upper Limit = Mean + SE*(1.96)
Lower Limit = Mean - SE*(1.96)

I am 95% confident I make from -3.08 to 36.84 BB/100

Conclusion, there is a 95% chance that at the very least, I am a SMALL LOSING player (and I am more likely to be a winning player than a losing player).

Thoughts are very much appreciated. I realize there is a big swing in what my actual BB/100 is...It's saying that it can narrow down my BB/100 to 1 of 40 different whole numbers..not saying much for specifics, but do you think the generalization is useful to know?

This is really interesting, would you be able to elaborate on how you decided (or where you found) the formulas you used to make your calculations?

The range you get is so big it's basically useless though.

Well thanks! I used Statistical Analysis formulas for my calculations by setting up a Confidence Interval given a data set.

Is knowing whether or not you are a winning player or not without having to sit for 400 hours at a poker table useless? Surely it's useful to be able to derive something from playing rather than thinking to yourself, "Well, I have played for 50 hours, but that means absolutely nothing because I still have another 350 before I even have a clue. I could be a complete fish or the next Phil Ivey, who knows?"

The purpose of my post is to let players know that maybe they can have a clue..because my statistics say that there's a 95% chance that for the game I play at, my worst case scenario is that on average, I'm (roughly) losing $3/hr and my best case scenario is that I'm winning $36/hr. That's a lot better than saying, "I have no idea where I am because I haven't played 50k hands yet."

Looks okay to me. I’m sure you meant Deviation From Overall Mean where you labeled Standard Deviation for the 12 deviations. Since the sessions were of varying length, I think the following approach may be more accurate using time rather than 100 unit hands

Var = (1/N)*Sum[i = 1 to N](Xi – u*Ti)^2/Ti

Where
Var = statistical variance
N is the number of sessions
Xi is the amount you won on the ith session
u is your overall hourly win rate
Ti is the number of hours for session i.

This was in Mason Malmouth’s Gambling Theory book and summrized by Bruce Z in the Probability Forum.

NO, NO, NO, NO.

Given the sample you have it is 95%. However your sample is just a sample it is not population. You are misintepreting what as CI actually is (in fairness the name is stupid). What I mean is it likely that you have been running good and your 'real' winrate is lower.

You can revise your winrate by knowing what the priori distribution in the population you are playing in is (which will be mainly losing player) and use bayes theorem to use the underlying distribution to revise your 'real' winrate. Your winrate will regress to the mean and the mean being a losing winrate.

However in a live game getting this information can be tricky, but estimated using the amount of rake taken away from the game.


Seem a bit confusing? Here is an example to help you along.

Player 1 - 20bb/100 over 2000 hands

Player 2 - 10bb/100 oveer 20000000 hands

Who is the better player?

It's impossible to know.

I know your point is that the confidence is much higher that 2 is a winning player (it's virtually certain), and that's true, but it doesn't help us compare the two players. All we know is that 2 has a proven track record and 1 does not. But we have no way to say that 1 is not the best player here. What we can say is that we have more confidence that 2 will continue to win at near his past rate. But the lack of confidence for 1 goes both ways, he could be much better than a 20bb/100 player or much worse. For 2 we know for sure he is only a 10bb/100 player, and he is definitely not a 20bb/100 player.

But we approximately know the distribution of players' "skill" apriori, a plyer who win 10bb/100 is easily in top .5% of nlhe players (online 6 max) and the probability that this 20bb/100 clown is better than him is pretty slim.

That's not knowable from the information given. If we had their SD we could make a better prediction but we don't. And even with that information it is very likely that the range of possibility for #1's true win rate will include much higher rates than #2's potential range. What if #1 has an SD of less than 1 bb/100 and #2 has an SD of 200bb/100?

So what you're saying is, the math is correct, but for the wrong thing. I have a 95% chance to be a winning player when I get dealt the same "luck" (good and bad included) that I have had in these 12 sessions, but if my luck changes in my next 12 sessions then I may have a totally different range for those 12 sessions that I have chance to be within? (example: When I'm running bad, there is a 95% chance that I am between W and X BB/100...but this 12 session sample above has a 95% chance to be between Y and Z BB/100). Is this what you're saying?

I'm not sure I learned the Bayes theorem in Stats I and II, but I'll try to refresh myself on that and priori distribution I guess. Did you mention these because you're saying CI's don't work for this type of problem? Maybe you're saying that because Poker has the "skill" and "luck" aspects it means that a CI is impossible to use? If you were taking a test and had averages of what students made on a test, you wouldn't include the averages of students that took a test on material from the same chapter that were given different questions. But if that is the case, then every hand from dealt cards to river should be in its own sample (when I have KK and an Ace flops, a 2 comes on the turn, and a 4 comes on the river, there a 95% chance I make/lose between A and B in this hand)..unless you can include all forms of luck in Poker to be "Poker" as a whole, and know that sometimes the best hand does not always win (so players can create their own luck too and not just run good). If you don't make each hand a different "test" that belongs in its own category then you actually have some data to work with. I've lost some hands I was supposed to win, won some hands I was supposed to lose, folded the best hand, and called with the worst hand. So it's not like I've just never lost a big pot. I think my 2nd or 3rd session was losing a BI right off the bat.

Does it not make sense though that after 12 sessions of me playing $1/$2 blinds, with me being up $844 after 50 hours, that there is probably a 90% chance that I am a break even player, and a 95% chance that at worst I'm a 3% loser? In these 12 sessions, I have been buying in for $100 instead of the full $200. This makes it easier for opponents to call the $30 on the end or whatever because "it's just $30 more." But if I'm buying in for $100 every time, I'd have to lose 8.44 BI's to break even..so does it not seem like there's probably a 10% chance that I could lose 8.44 BI's in the next how ever many sessions I play and a 5% chance that I come out being negative? Surely that seems pretty reasonable? A big part of statistics is seeing whether or not your final result looks like it makes sense or not too.

Thanks for the input so far everybody! Glad people can actually think outside the box/show me the light/whatever and not just think "too small of a sample size, I know nothing about statistics, but that's what I read somewhere sometime and don't really think for myself."

Also, if my results were against the same players every time, would you say that i could then do these calculations I have done to say that there is a 90% chance I'm at least break even, 95% chance I'm a 3BB dog against these players but not necessarily against all players?

I can't remember how, but you CAN create a distribution for the difference in winrates between player A and B- I think you use the difference in the two winrates i.e. 20-10= 10bb/100 hands for the 'mean' of the distribution, and a 'pooled estimator of variance' for the variance for the difference between the two distributions, although it might be a different one. But this should provide a distribution in which you can work out the probability that player 2 is in fact better than player 1. But I think the pooled estimator of variance assumes player 1 and player 2's variance is the same, which may not be true...not quite sure how you would actually work this out...any ideas?

In fact, if you plot "(winrate of player 2) - (winrate of player 1)" vs "probability that player 2 and player 1 have these particular winrates" (or something similar to this: this does not quite work because for some probability p, there will be lots of winrates which give different values of p. e.g. if actual winrate of player 1= 18, actual winrate of player 2 = 5, then this give the same difference in winrate to actual winrate of player 1= 17, actual winrate of player 2 = 4, but the two values of p in either case is almost certainly different. So how can we correct this to get to what I want it to be?), I think you get a skewed (discrete but approximately) normal distribution with mean 10BB/100 hands, and the (correct) distribution qualitatively shows the likelihood of the difference in winrates being XBB/100 hands. You can also read off this graph the probability of player 2 and player 1 having the same winrate (this will be at the point "difference in winrates = 0"). I actually if you can construct a graph like this with constants, then depending on villain's winrate and your winrate, you can see qaulitatively "how much better you are then them". But this requires you finding out villain's winrate, so this is not realistic, unless you do play withregs and find out their winrates

And yes, this post is directed at spadebidder obv. Props to anyone else who attempts an answer

lol

What OP did is perfectly acceptable IMO. A confidence interval is obviously based on a sample. It provides one measure of the a parameter value, such as win rate, through the location and width of the interval. I'm not sure why you think OP is misinterpreting it.

A Bayesian approach is another way to use sample data to describe a population parameter. But, if you have an ill-defined prior, as I think the case here, I don't think it very useful.

To suggest that a person's win rate will regress to the mean of win rates of all players and therefore be negative, makes no sense to me. What happens is that as the player plays more, the observed win rate regresses to HIS real win rate, assuming some measure of stability, and that win rate may very well be positive.

The OP is using a frequentist approach. Which basically says that there is no uncertainity in the sample. This is false assumption because we know in poker we can be running good or running bad and being near the mean over a small sample is relatively unlikely.

This means that anything concluded from the frequentist approach is likely to not be the 'real' value.

The bayesian approach estimates parameters of the game. In this case you can use data for other players in the game and their winrates should obey the central limit theorem, creating a normal distribtion. The arthimetic mean (EV,peak) of this normal distribution will be negative because most players lose.

Now try to imagine this, infact draw it.

You have your frequentist normal distribution and the normal distribution of all the players winrates. What you should see is that the winrate you have is rather unlikely given the normal distribution of all the other players winrates. You can intutatively see that it is likelier that your winrate is lower in 'reality'


Does this help?

Sorry I forgot to tell you how to do this.

The really clever way to do it is to use something called a Kalman Filter. This bayesian filter intergrates the two normal distributions together and comes out with a more accurate distribution.


What I would reccomend is to use values along the normal distributioin and put it in a spreadsheet. Then apply bayes theorem to the values. Plot the output of bayes theorem and a normal distribution function should arrise. This distribution will be closer to reality than before.

This type of maths is at the heart things like artificial intelligence. And the art of hands reading in poker which most people would argue a computer could not do. But in reality it can if set up correctly.


One thing that is worth note is that priori distribution is uncertain itself. This means that the mean of the new distribution is still open to uncertaintity. However it is likely to be better than the frequentist distribution.

(Last OT on this tangent, sorry) - Whatever prediction model you use from the information given in that example, it will show that the likely range of true winrate for #1 will extend both higher and lower than the likely range for #2 (using the same confidence interval). That's why I said it's impossible to tell which is the best player.

You could make a judgement though. Which ever player has a confidence interval of 2 sd that aggregated is greater is likelier to be a better player.

BTW I should have said but in the question the SD for 1 and 2 are the same (sorry)

The comparison of two means from normal distributions with different standard deviations is known as the Behrens-Fisher problem. A number of approximate methods of performing a statistical test are available, the easiest being to assume the variances are equal.

Isn't a sample uncertain in itself..it's a sample? You have the same odds of running good as you do of running bad do you not? So if I'm "picking my sessions out of a hat" who cares which sessions I picked? That's the whole point of it being a sample, is it not? I could have picked my best sessions, I could have picked my worst sessions. I'm not trying to figure out if I'm a better player than anybody. The goal is not to get anywhere near my actual win rate, it is simply to find out if I am a winner or a loser in general. I could care less who I'm better than.

Also, currently my average win rate per 100 hands is 16.88 BB/100. I understand that this is a high win rate, but given that I am not playing deep stacked (I'm playing with $100 in 200 NL), it is harder for me to have big swings in the first place. I'd have to lose 8.44 BI's in a row to break even (so imagine how many more than that I'd have to lose to be a -3 BB/100 player).

I did the math for you:

Assume that I played 5 more sessions and ran bad. This is what happened in every session:

1) Each session lasted for 4 hours with 25 hands/hour being dealt.
2) On each session I lost 2 BI's ($100 each) for -$200 in each session.

At the end of those 5 sessions, I would have played for 1750 hands with a new average BB/100 of -2.23 (this is not even the -3 BB/100 that my previous calculations said was my worst case scenario).

I would say there there's probably a low chance that I will run WORSE than the situation above, so maybe my calculations are correct in saying there is a 95% chance that my actual win rate is somewhere between -3 BB/100 and 36 BB/100.

I didn't read the whole thread, so if it's been mentioned I apologize, but if you don't use a weighted average of your BB/100 your numbers are completely useless. Even if you do use a weighted average, your numbers are completely useless over this sample size.

^

Yes, I have to agree, you COULD create a confidence interval for your BB/100, but the interval would be huge because the sample is so small

@ FourSpades

Yes if all you had was your own sample then yes the only thing that you could conclude is that mean of your distribution is the the most likely winrate.

However that is not the only information available. You also know the winrate of other players. You can use this information to refine your distribution.

Take this example.

You are winning at 10000bb/100 hands over 1000 hands. The % of the players in thegame that have that winrate is 0.00011%

If you think about it, it is quite likely that you have been running good and your 'real' winrate is likelier to be lower because your sample is very uncertain and the winrate itself is very unlikely over the sample.


This maths applys to everyperson that is not on the mean of the distribution. All of their winrates are likelier to be closer to the mean of the distribution of the population. The bigger the players sample the less is regresses to the mean.

For example Dusty Schmidt real winrate is likelier to be lower than the winrate is data shows. However his sample is so big that the regression to the mean will be minute.

In contrast your sample is smallish. Therefore your regression to the mean will be greater.

It is still likely that you are a winning player even using this approach. It just means that you are a little less likely to be a winner.

I am going to write a paper on this topic for a group of mathematicians I am working with and hopefully they can show me how to use a Kalman Filter to solve this problem.

The approach is pretty standard in my opinion. There's nothing wrong with it. The only proviso I would make is that it indicates that the OP is probably a winning player under the conditions applying to those 12 sessions, and if conditions change his results may also.

Correct the approach is standard. And there is nothing wrong with it.

However there is a better method which is to apply bayes filters.


I might note that using bayes filters on normal distributions becomes very complicated and something that only graduates of mathematics can realisticly do. (BTW I am not a math grad so I do not know how to but I have an understanding)


There is a more simple approach that changes the guassian distriution from a continous data set to a discrete one.