Open Side Menu Go to the Top
Register
How Many SD's Before You Can Call Foul Play? How Many SD's Before You Can Call Foul Play?

02-22-2011 , 10:39 AM
I would be interested in others' view on this:

One site I play, where I was formerly very profitable, I just seem to be on an amazing bad run now.

Based on PFAI I am running 2 SD's left of the mean (3% prob) , close to being outside the 95% CI for randomness. I have used MC analyses to generate the actual pdf for the 520 AI's in my sample. This is shown below:


It appeared my luck suddenly turned bad at a particular point in time. If I do the analyses from that point onwards I get the following result:


This run has a probability of less than 1% (2.5 SD's).

Now I know that if you have 10,000 players at a site, and they are normally distributed, then some poor souls will end up 3 SD's left of the mean. But as an individual without access to the complete distribution at what point does it become rational for you to cry foul and decide to boycott the site?

By analogy if it was a coin toss game against an unknown opponent where you had no definitive means of determining whether the coin was actually fair other that what other people told you, there would be some point where you would say 'Hang on something is not right here!'. The fact that other players had said they seemed to be running normally would not in itself be proof that the game was not somehow rigged against you personally.

I am not a riggie by nature but open to the possibility of some isolated shenanigans occurring, either through cheating or conceivably rogue insiders.

I am really just looking for opinions as to the point at which a rational person could reasonably cry foul.
How Many SD's Before You Can Call Foul Play? Quote
02-22-2011 , 01:29 PM
You can't just pick out where you started running bad and say this is outside the range of normality. You can start counting right now and see what happens. To use your coin flipping example, if I watched flip after flip just waiting around for ten heads in a row (a very rare event) could I then say, "hey there is something wrong here!"

Beyond that, I doubt you would be crying foul if you ran 2 SDs above the mean (we seem to never get anyone to post that in here). That would be akin to only crying foul about the coin coming up with 10 heads in a row, but not worrying about it when 10 tails in a row happen.

In terms of rationally boycotting the site, I say only you can make that decision. If you personally think there is any chance at all you are being treated unfairly, all things else being equal, you should just switch sites. It will certainly help your psyche.
How Many SD's Before You Can Call Foul Play? Quote
02-22-2011 , 02:05 PM
I take your point about cherry-picking. If I take the entire data set then I am 2 SD's left of the the mean. Looking at the EV graph (which I posted in the 'EV all-in' Thread) There is an initial period where I was running more or less as per expectation. Then a point at which I started to diverge and it has been more or less been downhill ever since.

If I am testing the theory that something happened at the divergence point then it is not unreasonable to compare the stats before and after that point.

While I am still not having to put money in the account (I took most of my money out just after a big win which occurred coincidentally at the divergence point) I will continue playing and keep monitoring the stats. We will see what happens.

If I ever had any EV graphs where I was running way above expectation I would certainly post them. The best I have ever done at any site is to run close to the line, never much above. Never seen any from anyone else either. Funny that.
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 01:25 PM
Played a bunch more games at the site. Here is my latest EV graph for PFAI.

Now running at 2.14 SD's left of the mean with probability 0.019.

How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 04:42 PM
wykh -

The same thing has happened to me - may I assume you are talking about PS? I have taken samples from before I noticed something amiss, after, sliced and diced, and it comes out the same way. For me, this happened about 14 months ago.

Assumption - The only reason that PS (or roges, if you will) would adjust play would be to get hands into high volume players. $1000 in the account of a high volume regular will cycle through and into the PS account, while low volume players may only cycle it trough occationally.

favors high volume playersMy sample is 14,000 hands. Besides the usual sorting that is available on PD, I also sorted tourneys in 20% increments to correspond with the usual terminology or E, ME, M ML, L. Some of this required hand by hand counting, such as looking at performance of AK, when facing Ax, and how it performed differently in the 1,2,3 quintile, verses the 4,5 quints. Toward the end, I also began factoring in the amount of total tourneys the opposing player had played in. This was a substitute for the amount of rake they generated for PS.

Looking for some sort of consistency in the change, I have arrived at a fairly consistent deviation from expected outcomes. Here is how it goes:

PS assignes each player a number -25 to + 25. It is a normal bell curve distribution. Players that generate high rakes, on line pro types will get a higher number. Most people are about 0. But, some players rate a negative number, based on variables that PS is able to gather from hand palying history, perhaps age, gender, etc.

How it works using an example: Late in tourney, QQ vs AKs flip. If it is a -25 vs +25, this now becomes a 25% vs 75% outcome, depending who has each of the hands.

In fact, this is exactly what I have found. Across the board, I am running outcomes that are approximately 25% below what would be expected.

Well? Do you think I nuts? This would also account for beginners luck. Anyone on their first deposit may be given a +5 or + 10 for their first 200 hands or so, and there may be a temporay deduction after a cash. Also, forgot to mention, it comes into full effect only by the 4-5 quintile.

Sorry for the long email,

Max
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 06:00 PM
Thats a very interesting story Max, Thanks for posting it.

Actually my data do not come from PS but another network. But I have observed similar phenomena at other networks without having sufficent HH's to make such a good case.

For my network, I had been playing a $1 and $2 MTT every night, each with a field of around 650. I had developed a very effective strategy whereby I would make the money 2/3 times and occasionally make FT (I won each tourney one time too). So I had an ROI of something like 150% and was basically just milking money out of the site, while generating hardly any rake.

Not sure if its relevant but I know I also annoyed a few other players by using the clock in the last 20 mins or so approaching the bubble to guarantee making the money.

I had just taken out about $60 from the site at the point where my graph went AWOL.

While it would seem to make a lot of sense for the site to want to 'doomswitch' me I have always been reluctant to postulate this because if you even mention this word in the 'poker is rigged' thread you get subjected to a torrent of abuse.

However I do think that once you start lying outside the 95% confidence limit and you see a behaviour that just doesn't look right from a random walk point of view you are certainly not nuts to start having a few suspicions that something funny is going on.

The funny thing is that why I could quite easily just stop playing at this site I find it hard to close my account because I am curious to see what will happen to the graph in the future. How many SD's to the left is it possible to go or will it suddenly start moving up again until I eventually reach the EV line again? If I ended up 3SD's left of the mean is there anything I could do about it? I guess not.
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 06:07 PM
Quote:
Originally Posted by wykh
Played a bunch more games at the site. Here is my latest EV graph for PFAI.

Now running at 2.14 SD's left of the mean with probability 0.019.
I assume your 2.14 is rounded as I get .0162.

So with a million players, a poker site will have about 19,000 of them running worse than you. You aren't special.
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 06:31 PM
I don't use a normal or binomial distribution, I compute a bespoke probability density function by MonteCarloing over 10,000 sets of board cards over the array of my/villain's hole cards. It does always come up very close to working out the binimial probability using the expected/actual win percentages and number of all ins.

I take your point that over a million players 19/1000 will run at -2.14 sds. I could accept that I was just one of the unlucky 19 provided:

-I was able to see the total distribution
-I could see that there was nothing that separated the +sd players from the -sd ones based on the sort of games they played, their ROI, where they were from etc etc

Without this information we are just taking it on trust. Rationally, why should we do this?
The fact that an outcome is possible but extremely unlikely does in itself mean that everything must be as it should be.

I am also struggling a bit in my head as to whether 1 million is the right number to use. Should you use the total number of players who have played at the site? If I am playing just one particular MTT every night and it has 650 runners is there not a case for using 650?
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 07:08 PM
Quote:
Originally Posted by wykh
IThe fact that an outcome is possible but extremely unlikely does in itself mean that everything must be as it should be.
It isn't just "possible" it is guaranteed to happen to the expected portion of the population, by definition. There isn't some Lake Wobeegon effect where everyone can be above average.

And some portion of those will post here about it.

As to your point about what players are in the tails of the distribution, that's completely irrelevant if the distribution has the expected proportions. Think about that before you argue against it.

Last edited by spadebidder; 02-23-2011 at 07:16 PM.
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 07:44 PM
Since we are talking about the distribution of PFAI's won then the distribution should be completely random in terms of player's location and types of games played. Say you found that the top 5% all came from, say, China would you not be a bit suspicious?

There should indeed be a correlation between PFAI's won and ROI because one affects the other.

When I said that an outcome is 'possible' I meant that it is possible to happen to me, not the population in general. I will repeat: Without information about the population in general I cannot conclude from the data I have that everything is above board.
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 08:47 PM
Quote:
Originally Posted by wykh
Since we are talking about the distribution of PFAI's won then the distribution should be completely random in terms of player's location and types of games played. Say you found that the top 5% all came from, say, China would you not be a bit suspicious?
I think that would depend on how many people played from China relative to the rest of the population of players on that site.
How Many SD's Before You Can Call Foul Play? Quote
02-23-2011 , 09:11 PM
Quote:
Originally Posted by wykh
I will repeat: Without information about the population in general I cannot conclude from the data I have that everything is above board.
The fallacy in that statement is the premise of proving a negative (that the deal is not random). The only thing you can conclude is that you have no evidence of a non-random deal.

Also, in case you were alluding to some kind of cheating and not a non-random deal, cheating can't alter the AIPF outcome distribution if the deal is still random. I think you get that, so you were in fact speculating that the deal isn't random.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 05:31 AM
Quote:
Originally Posted by spadebidder
The fallacy in that statement is the premise of proving a negative (that the deal is not random). The only thing you can conclude is that you have no evidence of a non-random deal.
Did you mean here that I have no evidence of a random deal? (as opposed to not random). I make a null hypothesis that the deal is random and test it at the 95% CI and it fails. However this failure would not necessarily be meaningful. As you say, I could just be an outlier in a much larger distribution.

Once the test for randomness fails (from my personal perspective) then there are many possible reasons for this. It could either be due to rigging or cheating. I have no way of knowing unless I have information about the sytem as a whole.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 07:59 AM
Quote:
Originally Posted by wykh
Did you mean here that I have no evidence of a random deal? (as opposed to not random). I make a null hypothesis that the deal is random and test it at the 95% CI and it fails. However this failure would not necessarily be meaningful. As you say, I could just be an outlier in a much larger distribution.
Yes, my statement was mistyped and I realized it after the editing period.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 12:13 PM
Quote:
Originally Posted by spadebidder
You aren't special.
Quote:
Originally Posted by spadebidder
As to your point about what players are in the tails of the distribution, that's completely irrelevant if the distribution has the expected proportions. Think about that before you argue against it.
I do appreciate the stats chat you guys are having, but ironically this is less of a stats issue and more of a human issue.

What many riggies (whether they are dumb or smart) require for all of their theories is that they are the main character in the world they analyze. Even the title of this thread is self-centered.

You make the point over and over that part of math is that people will fall at the various points of the graphs because that is exactly how math works and the math does not care if it is you, me, them, everybody, everybody (cue Blues Brothers song).

Problems get created when paranoia creeps in and a person tries to justify why he or she falls onto a part of the graph that he or she does not like and starts looking for reasons to explain it when basically it is just math being math (with perhaps a bit of rationalization of losses in the mix to avoid accepting one is not that good a player).

Taken to the next level this is why we see youtube filled with videos with bad beats in freerolls.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 12:57 PM
Monteroy this is a thread to do with probability and Statistics. Please take your philosophising back to the Poker Is Rigged thread where you and Wiki can merrily carry on abusing anyone wanting an adult debate. Boy am I glad I stopped posting there.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 02:06 PM
Well, you have had a couple real stats guys basically tell you nothing is unusual at all and a riggie post a new player boomswitch theory. You kind of keep ignoring the stats guys who say nothing is wrong by extending your "something might be wrong look after all look what is happening to ME" concepts then you gratefully thank the riggie for posting his new player boomswitch/ wealth distribution theory which if true would be easily detected or revealed by an insider and could of course be exploited for millions by anybody once discovered if true.

Since you seem to want to keep the discussion going even after the stats guys shrugged their shoulders multiple times you should thank me since I am actually appealing to what really matters to you, namely the mild delusions of grandeur that require all of your theories to be you-centric.

While more polite and dry all this is is the exact same conversation as found in the rigged thread only with fewer direct uses of the word ******, since after all you have not changed your mind or feelings on the topic despite Sherman and spade both saying nothing is wrong from what you present. You need to believe something is wrong and you should thank guys like me who understand that is what really drives you, even if we do make a bit of fun of you in the process since if you ever really listened to what I say you would change your whole approach to the situation and you would start using math more to your advantage (by playing solid long term poker) instead of allowing yourself to be a victim of it (by looking for reasons to play less/switch sites/worry).

You problems with poker have nothing to do with the math of the game, and I do agree with Sherman that if changing sites does make you FEEL better than it is the correct thing for you to do though again that has nothing to do with the actual math of the situation, and realistically you will eventually feel the same way on any site you play when you hit a patch on the wrong side of the normal curve.

Last edited by Monteroy; 02-24-2011 at 02:20 PM. Reason: typos within 2.1 SD that needed to be corrected
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 07:10 PM
Repeatedly tested, suing any number of variables, my PS data fails the randomness test. The stats checked are not about winning a tourney...that certainly is skill based. i used simple outcomes of various matchups with known statistical outcomes. Over enough hands, QQ v AKs will tend towards 54:46 ratio. I only looked at a 14,000 game sample, but found that:
When I had QQ - playing against a PS regular who played, according to on line services, more than 15,000 torneys, my win rate was less than 25%. Conversly, when holding AKs, my win rate was also about this range.

Did the same for AKs vs JJ, 10 10, not enough of the others, but all about the same.

PS failed randomness because the outcomes - at least for me - did not tend toward the random outcomes over many situations and many trials.
Said another way, randomness and PS are not the same.

A corellary - the outcomes for the above, and any other combination varied from the random outcome by some sort of consistent amount. The only factoing I could figure out was to subtract about 25% from my own outcomes. This seemed about right overall, but outcomes were worse, depending the volume of games played by opponenet.

Conclusion - There exists an internal system that is manipulating outcomes by some set system - at least enough to be significant for some small group of players.

Max
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 09:14 PM
Quote:
Originally Posted by Max2
Repeatedly tested, suing any number of variables, my PS data fails the randomness test. The stats checked are not about winning a tourney...that certainly is skill based. i used simple outcomes of various matchups with known statistical outcomes. Over enough hands, QQ v AKs will tend towards 54:46 ratio. I only looked at a 14,000 game sample, but found that:
When I had QQ - playing against a PS regular who played, according to on line services, more than 15,000 torneys, my win rate was less than 25%. Conversly, when holding AKs, my win rate was also about this range.

Did the same for AKs vs JJ, 10 10, not enough of the others, but all about the same.

PS failed randomness because the outcomes - at least for me - did not tend toward the random outcomes over many situations and many trials.
Said another way, randomness and PS are not the same.

A corellary - the outcomes for the above, and any other combination varied from the random outcome by some sort of consistent amount. The only factoing I could figure out was to subtract about 25% from my own outcomes. This seemed about right overall, but outcomes were worse, depending the volume of games played by opponenet.

Conclusion - There exists an internal system that is manipulating outcomes by some set system - at least enough to be significant for some small group of players.

Max
Funny, I read your post and came to a totally different conclusion.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 09:30 PM
Max you might like this EV graph, isolating every time I went all in with KK. The data come from another site where it seemed every time I shoved preflop with KK, typically vs Ax, I would very often find an A coming down, or get rivered by a straight.

The stats on this run, based on 109 clashes, are as follows:
Expected win rate: 0.747
Actual win rate: 0.633
Position relative to mean: -2.769 SD's
Probability: 0.005

I really get the feeling that KK is the hand of death often, particularly when shoving against a bigger stack towards the end of an MTT. In fact I bubbled in a tourney last night showing with KK and getting called by KT. In spite of massive equity in my favour 2 Ten's came down on the board and I was eliminated with nothing.

How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 09:43 PM
Quote:
Originally Posted by Max2
PS failed randomness because the outcomes - at least for me - did not tend toward the random outcomes over many situations and many trials.
Said another way, randomness and PS are not the same.

A corellary - the outcomes for the above, and any other combination varied from the random outcome by some sort of consistent amount. The only factoing I could figure out was to subtract about 25% from my own outcomes. This seemed about right overall, but outcomes were worse, depending the volume of games played by opponenet.

Conclusion - There exists an internal system that is manipulating outcomes by some set system - at least enough to be significant for some small group of players.

Max

Quote:
Originally Posted by wykh
I really get the feeling that KK is the hand of death often, particularly when shoving against a bigger stack towards the end of an MTT. In fact I bubbled in a tourney last night showing with KK and getting called by KT. In spite of massive equity in my favour 2 Ten's came down on the board and I was eliminated with nothing.

This is the key to nearly every riggie belief - the need to be the main character in the story while re-shaping the entire world to help explain why they fail at something, thus avoiding any personal responsibility for the failure. Max is probably talking about $1 MTTs or freerolls that the entire Pokerstars company is rigging against him in his world, and wykh is certainly talking about games of that level of buy in because that is the buy in level he plays.

This is an extremely powerful belief system and no number of math guys using logic and reason will ever have much of an impact on these beliefs.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 09:54 PM
p.s. I showed the data from another site because I have a very large number of HH's from there. Out of interest I ran the stats for the site where my earlier data came from:

Incredibly I found I had gone All-in 10 times with KK and lost every single time!

Here are the match-ups:

KS KH AD QC 0 0.74 -100 49
KS KH AD 3C 0 0.75 -200 99
KS KH QD TC 0 0.85 -300 169
KS KH AD QD 0 0.7 -400 209
KS KH AD AC 0 0.19 -500 147
KS KH AD QC 0 0.74 -600 196
KS KH AD AC 0 0.19 -700 134
KS KH AD 7C 0 0.75 -800 183
KS KH AD KD 0 0.69 -900 220
KS KH JD JC 0 0.81 -1000 282

[the way I compute my probs I cannot take into account all 4 suits, so I convert the hands to being either Spades or Hearts depending on whether suited or off-suit]

This is -4.5 SD's left of the mean, probability ~0.0005!
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 09:55 PM
Quote:
Originally Posted by Max2
Over enough hands, QQ v AKs will tend towards 54:46 ratio. I only looked at a 14,000 game sample, but found that:
When I had QQ - playing against a PS regular who played, according to on line services, more than 15,000 torneys, my win rate was less than 25%. Conversly, when holding AKs, my win rate was also about this range.

\
If I did the math correctly, the chance of you having QQ versus AKs in a 9 player table is on the order of 1 in 8000. Therefore in 14,000 games, you could not have faced this situation but for a few times. I would not want to allude to some form of fraudulent activity on the part of a poker site with such a small sample.
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 10:04 PM
Quote:
Originally Posted by Max2
Repeatedly tested, suing any number of variables, my PS data fails the randomness test. The stats checked are not about winning a tourney...that certainly is skill based. i used simple outcomes of various matchups with known statistical outcomes. Over enough hands, QQ v AKs will tend towards 54:46 ratio. I only looked at a 14,000 game sample, but found that:
When I had QQ - playing against a PS regular who played, according to on line services, more than 15,000 torneys, my win rate was less than 25%. Conversly, when holding AKs, my win rate was also about this range.

Did the same for AKs vs JJ, 10 10, not enough of the others, but all about the same.

PS failed randomness because the outcomes - at least for me - did not tend toward the random outcomes over many situations and many trials.
Said another way, randomness and PS are not the same.

A corellary - the outcomes for the above, and any other combination varied from the random outcome by some sort of consistent amount. The only factoing I could figure out was to subtract about 25% from my own outcomes. This seemed about right overall, but outcomes were worse, depending the volume of games played by opponenet.

Conclusion - There exists an internal system that is manipulating outcomes by some set system - at least enough to be significant for some small group of players.

Max

Where does one begin to point out the errors of math, logic, and probability here? You might ask for help before you make yourself look so foolish (but not now in this thread, too late).
How Many SD's Before You Can Call Foul Play? Quote
02-24-2011 , 10:06 PM
If you do the stats correctly you take sample size into account when you calculate the probability, this is what I have done. Spadebidder will confirm this.

For the umpteenth time, if you are analysing PFAI's your skill as a player, or the level of buy-in to a tourney, is totally irrelevant.

Monteroy is on a mission to belittle us as poker players because his business is poker coaching. Not one stat which I have posted in this thread relates to my skill as a player or the financial aspect of the game. For the record my stats for the site in question are as follows:

Games, av buy-in, av win, ROI, total winnings, skill rating
557 $1 $3 71% $465 70

Frankly I am only interested in contributions relating to probability in this thread, not the garbage that gets put up every day on the 'Poker Is Rigged' thread. If you cannot produce any data or analysis then please don't bother to post.
How Many SD's Before You Can Call Foul Play? Quote

      
m