Hey All,

II think the results I posted above about first quarter vs fourth quarter are less statistically rigorous than people ITT think: it's not only a sample-size issue; also notice that I'm comparing the top chunk of high-volume players in first quarter to the ones in the fourth quarter, but it's not necessarily the same group of players. Stuff like that.

Anyway, I see there's a bunch of interest, so I just wrote new code for doing this analysis in a more rigorous way. This time I'm going over all the players who played at least 40k hands in the whole year, and comparing the results of this group of players across all quarters. Here are the results.

(Just the players with at least 40000 hands annually (210 players))

quarter	#hands	winrate (bb/100)
1	4.9M	1.25
2	5.4M	-1.10
3	4.9M	-1.13
4	4.6M	-0.35

Notice these results still have the same sample-size issues (std dev is around 0.75bb/100 for each of these figures), and also probably some other subtle statistical issues (for example maybe many new players became bad regs in mid-2012 and that made the game easier for good regs, but it made the above numbers worse because these numbers include the bad regs; or maybe market segregation took a lot of fish away in mid-2012).

I'll keep looking for an even more rigorous way to do this analysis. If anyone has any ideas, or wants me to run the same analysis with different numbers, just say so.

Here is the same analysis for 10k annual hands, 20k, and 80k, if anyone's interested



Incidentally, we see that judging by these tables, the games actually got harder on the second half of 2012, and then got easier again in the beginning of 2013. I don't know if we're seeing here mostly a real effect or mostly variance and statistical artifacts. (I suspect it's mostly the latter, but I'm open to being convinced it's the former, esp if I find a better way to analyze the data).

Edit: Now that I look at the data, I see that it looks like something horrible (and statistically significant) happened somewhere around June 2012. Does anyone know what happened around then? Was some player pool segregated? I'll do some analysis on the data to figure out what's the key date when winrates started going down.

summer came in june
and winter again in the 4th quarter

Spain segregated its player pool on June 1st and asked the EU for financial help shortly after (that started a new wave of the overall euro crisis which calmed down by 2013)

Probably the most interesting part of the analysis is the lineup of an average table. I didn't realise that there are so many regfishes at PLO100, it looks like there's 1 recreational, 2 semi-serious players (who have breakevenish rakefree winnings, so regs don't make money off them, but their presence reduces the number of regs at the table among whom fishes' money is divided) and 3 good winning regs on average.

I wonder if people putting volume to be SN or SNE might have lowered the winrates?

Zoom poker was launched May 15 2012. That would probably have attracted some of the biggest whales, and possibly some of the sharper regs.

It would be interesting to see a similar analysis for 50plo and 200plo, is you have the ability. Clearly, no-one should play 100plo, but should they move up, or move down?

I can't imagine moving down having a positive impact on the situation.

I agree, the typical table composition that the data paints is *very* interesting, but I think you have the numbers off. A typical table will have one 80k+ player (i.e. more than 80k hands per year), one 20k-80k player, one 10k-20k player, and two recs with less than 10k hands per year. The reason we know this is because the database has a total of 10M hands, and 53M (hand,player) pairs (where a hand where 5 players were dealt in is counted as 5 such pairs). Thus, the 12.4M total hands played by the 40k+ players only account for 23% of the volume (or one-and-a-bit seats in a 5-handed hand) and so on.

I got the 100PLO hand histories from gui166, who bought them from HH sites. To do the same analysis for other games or stakes, I'd need hand histories. After I finish the analysis on PLO100, and if it is interesting and goes well, I plan to try to acquire such HHs, either donated by the sites or using donations from the community. If anyone has something like all the hands in some stakes in a significant portion of time, please PM me. I am gradually writing program code and developing techniques, and this code and techniques can be re-used for any game and stakes (I cal also share my code with anyone interested).

I don't necessarily agree with this. We really can't tell that with certainty yet. (I personally guess that there are some winners in PLO100, but the data doesn't show that yet, either.) By the time I'm finished with the analysis (hopefully in two weeks or so) I hope we'll have a good idea of how many winners there are in PLO100 and what their winrates look like. But for now, we really just don't know IMO.

How good is the coverage of the hands that gui66 obtained? Like 85% of play? More? How much of a problem do you think this would be in analysis? My inclination is that it shouldn't matter as long as there is no bias in the sample selection, i.e. that it is a random sample.

What is our hypothesis regarding percentage of pre-rakeback winning players. 5% of the population? If it were 5% would people perceive this to be a good or bad thing?

I'm also interested in seeing the percentage of winning players quarter by quarter. Make it everyone that has positive post rakeback winnings, no matter the number of hands played.

This needs a twist i guess. Someone with 4 hands played and profit will otherwise count as a winning player.

Yeah, but those people will be different in the different periods and there will always be such people. I want to see the percentage of people that actually made any money in these games by quarters and to see if there's any decline in that number.

I'm also interested in the total number of players by quarters and to see how these numbers relate to the percentage drop of the whole traffic. Steve claims that PLO traffic is seeing a constant growth and is more healthy than NLHE one but I'm not buying this.

I'm thinking from my perspective, as a recreational player who doesn't play freerolls. I should restate: it appears to me that no-one without a plan to make SN should be playing 100plo on Stars.

This morning, though, I wonder about the data presentation and selection bias. Great players will "move through" 100plo, so there might be an element of the Peter Principle (http://en.wikipedia.org/wiki/Peter_Principle) happening. Someone with 100k hands at 100plo is likely to be a rakeback grinder. It's reasonable that a rakeback grinder will have a negligible winrate, almost by definition.

I'd love to see the following table: split each of your "hand sample buckets" into deciles based on winrate in the first quarter, then calculate winrates for those deciles for subsequent quarters. It would also be interesting to show average number of hands played. If we are correct that the rake is killing the games, we should find losing players have rapid decay in the number of hands played. This table should visibly show player loss.

After that, we can think about new player gain. Has anyone asked Stars about whether they might provide anonymised player data?

You're actually correct on both accounts. Those data have been run. It was referenced in the most recent Stars player meeting discussion thread in the zoo. Even though you'd have to be a bit masochistic to wade through that thread looking for the 1:10000 signal:noise bit, it's there. It's also not pretty, even Star's PR guy Steve openly admitted that earn rates for the top bracket of players uniformly decreased over later samples - in other words their win rates were largely a result of variance.

You're also correct in regards to losing players quitting quickly. This is discussed at length in the public reports of companies like Party. It's also the reason for the recent trend amongst most of all sites of implementing various sort of player segregation changes. Even Star's 'Full Tilt brand is hopping on the bandwagon with their 'beginner tables'.

Hi All,

Just a quick progress update: I have some preliminary results on the distribution of "true winrates" among players who played at least 40k hands. The statistics behind them are still somewhat shady: I'm trying to make the analysis more rigorous, and at the same time I'm trying to use other approaches to sanity-check my primary approach. I hope to have a report out in a few days, a week at most.

When I'm done with that, I'll come back to this thread and answer all questions and requests that are still unaddressed.

For those in the know, here's a description of my approach: I'm using a Maximum Likelihood (=ML) algorithm to estimate the "true winrates" by thinking of the empirical winrates as true winrates plus noise. I know how big the noise are since I have the standard variations of the players. Thus, I'm essentially using ML to compute a deconvolution. Presumably, the ML algorithm should recover the true winrates, assuming I have enough data. The problem is that, while it seems my data should be enough, ML is over-fitting to the noise in the sample (as it tends to do). I still get some ballpark estimates, but the over-fitting makes them non-rigorous. So I'm trying to find approaches to reduce over-fitting in ML, or to find an alternative to ML altogether. And at the same time I'm trying other forms of analysis to maybe get corroboration for the non-rigorous results obtained from ML. If anyone has some experience with ML, over-fitting, and statistical machine learning, now is a good time to PM me.

Hi all,

Sorry for taking a long time with posting the next part. I still didn't gather the analytical know-how needed to get rigorous results. I guess I'll proceed with my original plan and do cross-validation analysis, and answer some questions that were raised in this thread. I'll post these in the next couple of days. After that I'll go back to the drawing board and try to get a good rigorous answer about how the distribution of true winrates looks.

As a small spoiler: My best guess to date is that out of the players who played at least 40k hands in the sample period (one year), 22% of them are winners, with average winrate of around 4bb/100. The rest are losers (mostly small losers of around -2bb/100). This is all pre-rakeback. I'll try to post the details of how I got this guess in a few days. As I said, I don't yet have the technical know-how to make this analysis in a rigorous way, so this should be viewed strictly as a guess. Hopefully in a week or two I'll be able to give more rigorous analysis.

Thank you Eldodo for this thread

Good thread. Thanks for the work. I wish there would be a way to do something for live PLO at 1/2 vs 2/5 and $6 rake vs $4 rake.

Hi All,

Since the stars PLO rake meeting is coming up, I have less time to work on the data than I wanted. So let me describe here my most advanced analysis to date: the one I used to derive my guess above that 25% or so of the players are winning at about 4bb/100. This analysis has multiple flaws, but I feel it's still better than nothing.

Obviously, when I get back from the meeting I'll keep working on the data and trying to develop a methodology to rigorously analyze the distribution of true winrates. To the best of my knowledge, stars don't know the distribution of true winrates of PLO players, either. They have more data, but still not enough to know the true winrate of any particular player, and I don't think they have a statistical method of the sort I'm trying to develop here, so the results of this analysis might be news to stars as well.

Anyway, here is what I did:

The Data

I took all the players who played at least 40k hands in the sample period. I calculated all their winrates and standard deviations. (Call these the "empirical winrates", to distinguish from the "true winrates"; all winrates in this post are pre-rake.)

I assume that there is some underlying distribution of true winrates, call it D. I assume that each player's true winrate is independently sampled from D. Then the empirical winrate is equal to the true winrate plus normally-distributed noise (according to the player's standard deviation). Our goal is to deduce the distribution D from the empirical winrates.

Note that the assumption I made here, that all players' true winrates are draws from the same distribution D, is wrong: the population of players who played 100k hands probably has a different true winrate distribution that the population of players who played 40k hands. But I think it's a reasonable assumption anyway which should give approximately corrects results.

Now, recall that we have the empirical winrates, and we need to deduce the distribution D. This is a deconvolution problem: we get a bunch of noisy samples, and we wish to estimate the distribution that these samples are drawn from.

There is a software package called "extreme deconvolution" that does this, but I'm not sure their methodology is appropriate for our case (I intend to try using it in the future). Instead, I wrote my own algorithm.

The Algorithm

I wrote a maximum likelihood algorithm. I'll describe it in a follow-up post, but the jist of it is that it performs a local search over possible D's, and tries to find the distribution D that maximizes the chance of obtaining the empirical winrates. Running it over gui166's hand histories, the algorithm spit out the following hypothesis:

Results - first method



The red normalized histogram is the empirical winrates. The green curve is the output: the distribution D. It says roughly that 23% of regs (with >=40k hands) have true winrate around 5bb/100 and the rest have winrate around -2bb/100. The blue histogram is a sanity check of sorts: it shows what happens when the noise is applied back on D: if D is a good solution, then the blue histogram should be very similar to the red histogram.

As we see the blue histogram is indeed similar to the red histogram, so the algorithm worked well. However, the solution it gave cannot possibly be the correct distribution of true winrates: there's no way the true winrates have only two peaks and nothing in between. So, what's going on?

Well, what we're seeing here is severe over-fitting, which is a very common problem of maximum likelihood algorithms. The green hypothesis does explain the red empirical data, but many other similarly-good solutions exist, so the algorithm took one of them that fit the irregularities in the red data really well, and that created a weird looking hypothesis. I'll discuss this issue of over-fitting in a future post.

Now, can we still get some useful information from the green solution, even though it's obviously not the correct solution? To find this out, I created some synthetic data from various winrate distributions and checked to see what my algorithm spits out for that synthetic data. The results were encouraging (I'll detail them in a future post): it seems that if we mash the spikes towards the center, that seems to give a decent estimation of the true winrate distribution on the synthetic trials that I ran: So a reasonable guess for the true winrate distribution for gui166's empirical data is that around 22% of players have positive winrates of around 4bb/100 on average, and the rest have negative winrates of around 2bb/100 on average.

This method is highly non rigorous, but I think it gives a decent guess (for now).

Second method

How does one solve overfitting? There is a lot of literature on this subject. One way is to make the search space smaller. For example, here is what my maximum likelihood algorithm spits out when I allow it only to use hypothesis which are first increasing and then decreasing:



As you see, this graph makes more sense than the first graph, but it still doesn't look right. And also we already know that these methods are prone to overfitting. But we at least feel better seeing that the properties of this solution are similar to the first solution: 27% of players are winners with average winrate of 4bb/100 and the rest are losers with winrate of around -1.5bb/100.

Summary

We get some sort of guess for the distribution of true winrates. It is still a guess, but trials with synthetic data show that in some aspects it's at least somewhat credible.

In future work I intend to try to solve the over-fitting problem, either by modifying the maximum likelihood algorithm or by using a different algorithm.

Looks good so far. Wish I could contribute more in terms of methods of analysis, but it would not be productive as my knowledge is only superficial. I can try asking some friends whom I studied with who are now doing Master/Phd level econometrics and if they come up with anything that I think is worth sharing, I will post it.

does this mean anything below 400plo is unbeatable due to high rake and level of competition catching up?


I am trying to build a bankroll at nano plo but after reading this I don't know anymore. PLO o8 any better for trying to build? Depositing enough money to play ssnl is not an option either because I am still a losing player and I can't afford to lose that much money.

PLO and NL/PLO8 are raked the same, and if you don't like the rake in PLO don't think about playing O8.

This is a bit of topic maybe but the PL08 and NLO8 games should have less rake then PLO. If it's bad for PLO what kind of rake**** is O8 inn.

The results are consistent with what i found years ago. Basically what you see is the pile of sne grinders who make a profit after rake back. Many, myself included, can tell you from personal experience that the sne grind can stunt skill growth in long run and cause suboptimal decisions in short run.

The spike on the right was closer to 6 though when i did my analysis.

I bought my hands off some guy on hem forums. I did the analysis for plo100 and higher.

grizy: what kind of analysis did you do? Was it maximum likelihood, or something else?

I just did cluster analysis. It was actually a byproduct of me trying to identify common stats of long term winners.

can you give some more details? As you might know, I'm a bit stuck. I'll resolve it eventually, but if you can give a bit of info on what you did, I'd really appreciate it.