Hi guys.

So Im back and thought I bring you a little wb present.

I just read this thread from mckrogh. There was a discussion Ive seen quite a lot in Sttf over the time. Its about small samplesizes and how reliable they are.

Here is a little gem from the archives about the small sample size problem. Enjoy.

Great Link Ronny. Its and very interessting topic. Thanks for sharing!

What are you thoughts on small sample size and how reliable they are?

Cool topic and cool link indeed.

The linked post argues that they can be very important if you have a good grasp for the prior probabilities. A good grasp on that comes with experience... so that's what it depends on imo.

Also, totally adapting your play because of some tendency observed over a small sample size that has a very low prior probability seems like a bad idea, the observed sample increases the likelyhood that villain actually has the tendency a lot relatively, but due to the low prior probability, it is still very low.

In practice that means that the given 5:6 VPIP example is a good one to adapt your play to since there are loads of loosetards. But for instance in wouldnt be a good idea to assume that someone pushes ATC in some spot where it is insane to do so, just since he pushed the last 6 times. Thats because prior probability will be very low.

So it also depends for what read you are using the small sample for.

EDIT: Oh and also, since small sample sizes always provide reads in percentages (in the VPIP example there is a 82% chance that villain is very loose) and never in almostfacts ("villain is 50/12 over 1000 hands so there is a 99.9% chance his true VPIP is over 40") whether you should adapt your play to it obviously also heavily depends on how profitable your play is gonna be if you are right and how bad your play is gonna be if you are wrong.

Thanks Ronny. That was a gem.

I think Hirle nails it here. We don't want to be applying Bayes Theorem necessarily but instead Bayesian updating.

Say we want to know how often a player 3 b shoves our button raise with 15-20 BBs. This is a spot that doesn't converge quickly. So suppose villain has VPIP/PFR of 15/13 with around 100 hands played together. This is a spot where I start to consider someone regish with a brain (noted that is for 6m I have less of a grasp of what that should look like at FR or 18m or w/e but you guys do if you play them). So now I have a set of prior probabilities about this guy. I am going to believe thinking players are more likely to 3b shove button raises on me light. This means that if he 3b shoves me once it is going to influence me towards thinking more than I am right and if he doesn't it won't change my view very much.

Yeaaaaaah I doubt that is very clear or informative but I need to get back to some work.

Some flawy math... still a nice post, though

You mean that he forgets the permutations of the 5:6 VPIP?

(I caught that, wondering if there is something wrong that I missed?)

Cliffs - small sample sizes can mean something if that's all you have to go by (??)

Poker Sleuth HUD might be of interest regarding this topic (see post #8 and below).

Juk

Brilliant stuff Hirle and one more fine link Juk

Exactly
Same in the 3bet-calc

cliffs?

Ah ok, thanks, I think his point still stands though. Edit: yeah it does, one of the replies point it out...

Small small samples can be useful but you must understand and apply Bayes theorem to use them correctly.

Basically it says if someone is 50/30 after 10 hands he is more likely to be 50/30 who plays his usual game then a 10/6 guy running hot.

Oh right, I saw that too now...
Lucky OP, the forgotten factor cancels out in the division afterwards (I missed that)

It really isn't a correct application of Bayes' Theorem since that requires us to know all these conditional probabilities. With small samples we can't estimate these well.

It's really more this which is an application of Bayesian updating. I really don't see what's so profound about this post

I dont get this. We guesstimate the prior probability of anyone VPIPing 5:6 and the prior probability of any villain being a loosetard, we calculate the probability of villain VPIPing 5:6 given that he is a loosetard (we need to quantify loosetardism) and then we use this information to calculate the chance of villain being loosetard given that he has VPIPed 5:6 by applying Bayes' Theorem.

P(loosetard|VPIP 5:6) = P(VPIP 5:6|loosetard)*P(loosetard)/P(VPIP 5:6)

Sure, we have to guesstimate, but how is this not an application of Bayes' Theorem?

(disclaimer: I'm no expert on the topic, I'm actually asking)

Btw. While I'm at it, what is Bayesian updating? Cant find an easily accessible source quickly... I'd guess it is updating probability estimates based on new information using Bayes theorem (which is indeed what is being done here), but then every use Bayesian updating would be use of Bayes Theorem...) EDIT: and when I use google in a less moronic way I get this http://www.statisticalengineering.com/bayesian.htm which seems to support the above.

As with most things, if you already know this, it is not profound at all.

EDIT: And to say what "this" is once again: some people think "100/0 over 2 hands, he's limping ATC!!!". Some people think "small sample, no use at all". Both are wrong. Small samples can be very useful, but you must have an idea of what the prior probabilities are, they matter. Most posters already know this on an intuitive level, but its nice to see it formalized. Some (you) already know it on a formal level, but even then it is relevant for the forum cause I see still people replying "why do you give stats? its 10 hands FFS!" all the time, this sheds some light on that.

While I'm at it, I might as well make this a bit less sarcastic and add

When you put it this way I suppose it is. I think when I read the link posted in the OP I only saw the part where he was talking about shoving sets or sumfin. Anywho I always thought of it more like a maximum likelyhood estimation. We have P(loosetard|fold limp fold fold fold) <--VPIP 20% but w/e I didn't want to type fold a mirron times.

I think using Bayes Thm is a bit difficult in this spot. It requires us to totally partition the space of players and assign probabilities to all things. This isn't something we can likely do on the fly nor would it be all that useful to sit down in front of HEM for hours and decide the P(VPIP in some range over some small number of hands). And because Bayes Thm can't account for the high variance of our estimations (given we have very little information) we need to be aware of that as well.

Btw. While I'm at it, what is Bayesian updating? Cant find an easily accessible source quickly... I'd guess it is updating probability estimates based on new information using Bayes theorem (which is indeed what is being done here), but then every use Bayesian updating would be use of Bayes Theorem...) EDIT: and when I use google in a less moronic way I get this http://www.statisticalengineering.com/bayesian.htm which seems to support the above.
[/QUOTE]

That is about the best source I can quickly find that gives an intuitive idea of Bayesian Updating.

I have to get to class here in a minute but let me get to my real point. The formal mathematics here are very confusing. You noted well that good players intuitively understand how to handle small samples. I think people get too excited when someone posts "math" to apply to their game. You can't really apply Bayes on more than intuitive level and while Bayes Thm underlies Updating I doubt anyone calculates those probabilities on the fly. They also aren't worth calculating by hand due to the high variance nature of seeing the same vector of actions repeated in similar situations.

Ah check, I see the point and think I agree. This is not gonna be directly useful in game. I cant do these calcs on the fly and I doubt anyone can. Also I indeed simplified the problem by having a "probability that villain is a loosetard" while there are a million ways/degrees in which one can be a loosetard, things like that would matter if you are actually going to apply this. Still, I think that (as I posted earlier) the math does give some insights that are/might be useful in game.

It's a valid point though. If someone raises the first three hands in a row, they're somewhat more likely to be a maniac than have been dealt three hands in a nit's raising range. You don't really need to know the exact probability of either in most cases.

It's obviously helpful when you're BvB and trying to decide whether the SB, who has "reglike" stats, is a reg type or some other type of nit. If SB shoves three hands running < 10BB, he's probably not just a nit.