Two Plus Two Poker Forums Winrate required sample size
 Register FAQ Search Today's Posts Mark Forums Read Video Directory TwoPlusTwo.com

 Notices

 Micro Stakes Limit Discussions of micro stakes limit Texas Hold'em

 05-07-2011, 09:36 AM #1 stranger   Join Date: Mar 2011 Posts: 7 Winrate required sample size I was looking around trying to find out how many hands I'd have to play before being confident of my win rate... I found this formula, which I think is based on a normal distribution (is poker?) and a 95% confidence interval: 1.96*(standard deviation)=(winrate)*(square root of number of hands) For me, my SD is 17BB/100, i'm winning at 3BB/100 (and want to be winning at that in n hands anyway, somewhere there abouts), so the formula works out at n=([1.96*17]/[3/100])^2 =1.09 million hands Please tell me that isn't right! The other option is that the SD is 17/100 which makes sense, but then: n=([1.96*17]/3)^2=123 hands Is there something I'm doing wrong or is this just the way it is? I'm playing .25/.5 6-max limit holdem btw if that makes any difference at all.
 05-07-2011, 10:37 AM #2 Pooh-Bah     Join Date: Jan 2009 Location: Ohio Posts: 3,828 Re: Winrate required sample size Try it this way: winrate +/- (SD/(square root of (n/100)))*1.96 That is your 95% confidence interval. For 10,000 hands, 3 +/- (17/(square root of (10,000/100)))*1.96 = 3 +/-3.33 we are 90% sure your winrate is between -0.33 and 6.33 if you play a million hands, it's somewhere between 2.67 and 3.33 Standard error of the mean Last edited by Cranberry Tea; 05-07-2011 at 10:48 AM.
 05-07-2011, 10:50 AM #3 stranger   Join Date: Mar 2011 Posts: 7 Re: Winrate required sample size Hmmm, I'm not sure how to solve that for n...
 05-07-2011, 12:38 PM #4 What SN is this?     Join Date: Aug 2005 Location: ~DANZA KUDURO~ Posts: 22,434 Re: Winrate required sample size i have avoided saying this for so long because i never wanted to get in to this argument, but i dont believe the bell curve can be used to measure some mythical fixed winrate. First, why there is no such thing as a 'winrate': -You have dynamic games. Different tables, different opponents, changing opponents (ie the opponents skill level will change). Your winrate is based entirely on the tables you choose to sit on. It's useless to measure some sort of hypothetical winrate on today's pokerstars .25/.50 tables and use hands from different games with different players. -You have a dynamic skill level. Ideally you will get better yourself over time and as such by the time you have collected enough data to measure your winrate your skill level will be completely different and you will have an entirely different winrate. -Over a sample you will have hands at which you arent even playing at your A/B game. What winrate are you trying to find? Whether you are capable of beating the games when fully concentrated? If so there is no point including hands you played tired or tilted. Second, why the bell curve is fundamentally flawed for measuring poker 'winrates': -In analysing a given player's poker results on 2+2 we miss all the hidden data. Lets say we take 100 random players who are new to online poker and each play an initial 10k hands while learning as they go. Assuming similar personality traits, the 10% most unlucky will walk away and never look back. The middle 80% are likely to take it as just a hobby. However, the top 10% are likely to think about it as a possible income source and want to know if they are beating the games. They are the ones most likely to seek out forums like 2+2 and programs like HEM. So the fact that we are having this conversation means that its already likely that op and others like him are in that top 10%. -A normal distribution does not take in to account the cumulative effects of wins and losses. A player that loses 100BB in 5k hands is more likely to lose more money in his next 5k hands than if he had won 20BB in those 5k hands due to tilt and self-doubt. As a result significant downswings (and for some players upswings) often skew data. As an example, lets say we take two players, each is equal in skill, and each has two downswings over a 100k hand sample. However, one has their downswings at the 10k mark and the 30k mark, and the other has their downswings at the 60k mark and the 95k mark. Even if luck is exactly equal with the exception of timing, i would bet good money the second player has a far better winrate than the first. -A normal distribution ignores the fact that we already know a LOT about how poker player's winrates are. If we take a player with 100/85 stats and see that they are winning at 3BB/100 over 50k hands, our response should not be 'oh wow they must have found some new strategy', the response is should be 'this player is most likely in that .01% of players because such a strategy has been proven repeatedly in the past not to work'. op - if you are winning at a decent rate after 100k hands, and more importantly can recognise spots where your opponents are making mistakes and are exploiting them then you are probably a winning player. and imo noone can say more than that. /rant
 05-07-2011, 12:43 PM #5 What SN is this?     Join Date: Aug 2005 Location: ~DANZA KUDURO~ Posts: 22,434 Re: Winrate required sample size (btw yes im aware a lot of that is practically copy pasted from black swan heh)
05-07-2011, 01:20 PM   #6
Pooh-Bah

Join Date: Jan 2009
Location: Ohio
Posts: 3,828
Re: Winrate required sample size

Quote:
 Originally Posted by ESKiMO-SiCKNE5S -You have dynamic games. Different tables, different opponents, changing opponents (ie the opponents skill level will change). Your winrate is based entirely on the tables you choose to sit on. It's useless to measure some sort of hypothetical winrate on today's pokerstars .25/.50 tables and use hands from different games with different players. -You have a dynamic skill level. Ideally you will get better yourself over time and as such by the time you have collected enough data to measure your winrate your skill level will be completely different and you will have an entirely different winrate.
I agree with all of that, although I think winrate confidence levels can still be useful approximations if we use them sensibly.

Quote:
 Originally Posted by ESKiMO-SiCKNE5S -Over a sample you will have hands at which you arent even playing at your A/B game. What winrate are you trying to find? Whether you are capable of beating the games when fully concentrated? If so there is no point including hands you played tired or tilted.
Here, I disagree, because imo tilt control and staying focused is part of what we want to measure when we look at winrates.

I did not read the rest of your post just yet, but I'm sure it's pretty solid.

Update:

Quote:
 Originally Posted by ESKiMO-SiCKNE5S -A normal distribution ignores the fact that we already know a LOT about how poker player's winrates are.
Agreed. This is essentially a Bayesian inference, we are adjusting our interpretation of the data based on outside information.

Last edited by Cranberry Tea; 05-07-2011 at 01:28 PM.

 05-07-2011, 02:37 PM #7 stranger   Join Date: Mar 2011 Posts: 7 Re: Winrate required sample size If the sample I'm looking at includes my ups and downs, A-games and Z-games, etc. etc. then any results based on that would surely also account for the same swings/moods/focus levels in future sessions (granted not the improvement, but I'm happy to look at a worst case scenario). The question I want the answer to is: if I continue doing what I'm doing for a long time, will I earn money from it? And to be honest, I agree with your answer. I can tell when people are making mistakes, and when I've lost value and when I've gained value from a hand, so I know I'm probably going to win. When I saw a million hands come out of that formula, I thought, if I'm still winning at 3BB/100 after a million hands I don't think I'll care. That'd be about \$35,000 a year playing two tables for four hours a day (and that's assuming I don't move up limits). On the other hand, if I was instead winning at 1BB/100, that'd be crud. So I would want a way to quantify it, and I can't think of a better way than winrates, rather than calculating the expected value of each play I make in every conceivable situation, and I don't see myself doing that.
05-07-2011, 05:45 PM   #8

Join Date: Sep 2002
Location: Boulder, CO
Posts: 14,682
Re: Winrate required sample size

Quote:
 Originally Posted by handorfff If the sample I'm looking at includes my ups and downs, A-games and Z-games, etc. etc. then any results based on that would surely also account for the same swings/moods/focus levels in future sessions (granted not the improvement, but I'm happy to look at a worst case scenario).
This doesn't address eski's contention that you have self selected and that you've been lucky. Also, you didn't mention how many hands (and hours) you've played, so I don't know how I feel about your sample covering all potential conditions.

Quote:
 When I saw a million hands come out of that formula, I thought, if I'm still winning at 3BB/100 after a million hands I don't think I'll care. That'd be about \$35,000 a year playing two tables for four hours a day (and that's assuming I don't move up limits). On the other hand, if I was instead winning at 1BB/100, that'd be crud. So I would want a way to quantify it, and I can't think of a better way than winrates, rather than calculating the expected value of each play I make in every conceivable situation, and I don't see myself doing that.
What limit are you playing? 3BB/100 at 3/6 6m is impossible. 3BB/100 at .25/.50 is \$1.20/hour two tabling.

Start here, limits of 2/4 and above, 1BB/100 is about the best you can hope for playing FR or 6m. You'll have trouble doing a lot more than 80 hands an hour per table. \$35/hour (which isn't enough to go pro, imo) requires that you 4 table at 3/6 and have good rakeback/bonus. Poker is a tough game mentally, and a good number of people who could hit these win rates playing short sessions as a hobby had no hope doing it full time as a pro. No one knows this better than Eski, who is a high volume grinder.

05-07-2011, 10:55 PM   #9
What SN is this?

Join Date: Aug 2005
Location: ~DANZA KUDURO~
Posts: 22,434
Re: Winrate required sample size

Quote:
 a good number of people who could hit these win rates playing short sessions as a hobby had no hope doing it full time as a pro. No one knows this better than Eski
aha this could be taken oh so many different ways =P

05-08-2011, 12:16 AM   #10
veteran

Join Date: Mar 2010
Posts: 2,934
Re: Winrate required sample size

Quote:
 Originally Posted by ESKiMO-SiCKNE5S -Over a sample you will have hands at which you arent even playing at your A/B game. What winrate are you trying to find? Whether you are capable of beating the games when fully concentrated? If so there is no point including hands you played tired or tilted.
Quote:
 Originally Posted by Cranberry Tea Here, I disagree, because imo tilt control and staying focused is part of what we want to measure when we look at winrates.

1+ to CBT's comment. We all have our fundamental tilt frequency. Some players lacking self-control may be playing tilted 25+% of the time. Other players in very tight control may play tilted less than 1% of the time. For many people (disclaimer: though not for all), I think that tendency toward tilt will remain more or less constant over time (it being a function of how hot-headed, thinking/emotional, anxiety-prone, etc that they are). Thus it's an integral consideration in their long-term winrate.

I agree (to varying degrees) with most of the rest of Eski's implications. Though I still think winrate calculators are of some value if you need to get a little idea of where you're at, but when evaluating relatively small sample sizes, I like to combine the info they provide with a more subjective judgment of how I've been running. Incidentally, this is part of my tilt-control strategy: to recognize when I got lucky, when I was running good. Helps with not steaming when you run bad or get sucked out on.

At any rate, OP I'm too lazy to calculate winrates myself, so I use some sites to do it for me:

http://www.castrovalva.com/~la/winlose.htm

http://www.evplusplus.com/poker_tool...te_confidence/

Quote:
 Originally Posted by DougL What limit are you playing? 3BB/100 at 3/6 6m is impossible. 3BB/100 at .25/.50 is \$1.20/hour two tabling.
Picking nits, but I think 3 BB/100 2-tabling at 0.25/0.50 comes to more like \$2.00/hour (figuring will get in 130 or so hands/hour). But yeah, that comes well short of \$35k/year at 4 hours/day.

 05-08-2011, 05:25 PM #11 stranger   Join Date: Mar 2011 Posts: 7 Re: Winrate required sample size Hmm, I may have been out by an order of ten
05-27-2011, 09:54 AM   #12
enthusiast

Join Date: Mar 2011
Posts: 71
Re: Winrate required sample size

Quote:
 Originally Posted by handorfff required sample size
So is there any conclusion:::
Is 10K hand enough to have an idea::
Is 100K enough:::

05-27-2011, 11:17 AM   #13
What SN is this?

Join Date: Aug 2005
Location: ~DANZA KUDURO~
Posts: 22,434
Re: Winrate required sample size

Quote:
 Originally Posted by dayancaaf So is there any conclusion::: Is 10K hand enough to have an idea:: Is 100K enough:::
no
no

05-27-2011, 11:30 AM   #14
Pooh-Bah

Join Date: Jan 2009
Location: Ohio
Posts: 3,828
Re: Winrate required sample size

Quote:
 Originally Posted by dayancaaf So is there any conclusion::: Is 10K hand enough to have an idea:: Is 100K enough:::
Depends on what level of certainty you want.

05-27-2011, 12:15 PM   #15

Join Date: Sep 2002
Location: Boulder, CO
Posts: 14,682
Re: Winrate required sample size

Quote:
 Originally Posted by dayancaaf So is there any conclusion::: Is 10K hand enough to have an idea:: Is 100K enough:::
You're asking for a precise answer to an imprecise question. For very good players, the answer in the FAQ is probably better than a 10K sample. Any serious question would look like "how many hands would I have to play starting today, to know if my WR is 2BB/100 +/- 1BB/100 with a 95% confidence". Otherwise, results bias kills most people. Re-read monk's post until it makes perfect sense.

Honestly, you should make enough money to move up from .25/.50 before you have a large enough sample to have a high confidence in WR. Long run is a long way away.

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is OffTrackbacks are Off Pingbacks are Off Refbacks are Off Forum Rules

All times are GMT -4. The time now is 05:53 PM.

 Contact Us - Two Plus Two Publishing LLC - Privacy Statement - Top