Suppose you know in the population that the average success rate of a binary trial (eg. probability of hitting a 3 pointer in basketball) is 30% with some known variance. Then, you observe an individual drawn from this population who has a 40% success rate after N trials. What is your expectation of this individual's true success rate?

In the extremes, it should be clear if N is really small that the answer is 30% and if N is really large that the answer is 40%. I'm trying to derive a formula for the in between.

Appreciate even some brainstorming or thinking out loud.

Thanks

Using the z test, a normal approximation to the binomial, a sample size of 80 or more is required to state that the player hitting at 40% does not come from the 30% hit population; significance level = 5%

Formula: z = (Pobs - Ppop)/ sqrt(Ppop*(1-Ppop)/n)

= (0.40 - 0.30)/sqrt(.21/n) = 1.95

z is standardized normal deviate
Pobs = observed success prob.
Ppop = population success prob.
n = sample size
For 5%, significance, z = 1.96

For more exact methods, Google "Test for a proportion"

That's a test for whether that player was actually drawn from that population. I want to know what you expect the player's true probability of a success given their success rate over N trials and taking as given they are drawn from the population.

This has been discussed before. You can search for "beta prior" here on 2+2.

Formulas exist for a few "simple" cases. If you assume the prior population of a percentage is Beta with mean M and variance V, and you then observe T new observations of which K are "successes", there is a formula that gives you the mean of the posterior distribution.

IIRC a fair amount of algebra is required, but it is doable. I suppose the formula may be given on the internet somewhere as well. If I have some time this afternoon I'll try to re-derive this myself and post what I come up with.

Thanks, I found this thread where you contributed: https://forumserver.twoplustwo.com/2...ght=beta+prior

This thread is reasonably close to what I'm going for. I guess I was looking for a more rigorous way to determine my posterior distribution, but I think I'll take a shortcut there.

I've done the easy algebra. Others can carry it as far as they'd like. The Beta distribution relies upon two-parameters called A and B. They are related to the mean M and variance V as follows:

M = A/(A+B)

V = (AB)/[((A+B)^2)*(A+B+1)]

Let R=(1-M)/M. If I did the algebra correctly, A and B in terms of R and V:

A = [R-V*((1+R)^2)]/[V*((1+R)^3)]

B= A*R

Then suppose you observe T new observations of which K are successes. Then the posterior distribution is Beta with parameters A' = A+K and B' = B+T-K.

Of course, the mean of the posterior is given by A'/(A'+B') which is easily found. A complete "standalone" formula for the posterior mean can be derived via more algebra if so desired.

No you can't state that. You can only state that fewer than 5% of of 30% shooters will hit 32 or more out of 80.

Say the population rate was based on each of one million persons each taking 1,000 trial shots for an overall success rate of 0.30. Out of those you pick a guy that shot 0.40 in his 1,000 trials. What is the best estimate of his true success rate? 0.40, right? Why would the other people's rates matter?

Since 30% shooters never hit 400 out of a thousand you are basically right. But your logic becomes faulty had you used a percentage like 34%. Some of those 34 percenters were lucky 30 percenters.

Yes I agree.

This. Classic hypothesis tests are so misinterpreted even by experts that I think one might be better ignoring them entirely. In this specific case, using them is very wrong from many points of view.