Let's say someone claims the coin is fair - 50% heads, 50% tails?

You toss it:
a) 3 times - 3 heads
b) 9 times - 9 heads
c) 100 times - 100 heads

I guess getting 3 heads out of 3 tosses wouldn't surprise anyone, getting 9 heads out of 9 tosses would raise an eyebrow, getting 100 heads out of 100 tosses from presumably fair coin would certainly be tagged as cheating.

At what number/point would you determine the coin not to be fair?

Or to paraphrase a question (superturbos run mostly on all-in EV luck) at what relation of your -EV running vs. stakes could you/would you conclude that such amount of bad luck is "very, very, very.... improbable" thus cheating?

There is a statistical field called Inference and one of the main parts is Testing Hypothesis. For the fair coin question, the hypothesis is that the coin is fair, or equivalently, the probability the coin will fall heads is 0.50. An experiment is run, namely tossing the coin a number of times, and the hypothesis is tested by comparing the results to that which would be expected if the hypothesis were true. Since the experiment involves a sample that produces random results, one can rarely be certain that the results absolutely confirm or deny the validity of the hypothesis, so statisticians use such concepts as confidence and significance levels to validate or refute the hypothesis.

A simple test for the fair coin, is to toss it n times. Assume r successes (heads) were observed. Then the estimated probability of a head is r/n. The following t statistic is then computed:

t = (r/n -0.5) / [sqrt(0.5^2/n]

The numerator is the difference between the observed and hypothesized probability of a head. The denominator is the standard deviation of the proportion of heads if the hypothesis was true. This t statistic is compared to a critical t value based on the desired significance level and sample size to determine if the observed proportion of heads is significantly different from the hypothesized value of 0.50.

Note: The above uses the normal approximation to the binomial distribution. There are other tests that are more sophisticated, but the above is usually satisfactory if the sample size, n, is not too small, say over 30.

Another approach is to use the Excel binomial distribution function. If r heads are observed in n trials, enter the following in a cell:

Binomdist(r, n, 0.5, 1)

This is the probability that r or fewer heads would be observed in n tosses if the true probability of a head was 0.5. If this probability is less than your significance level, e.g., 5%, then you reject the hypothesis that the coin is fair.

I think just knowing the probability of the streak happening isn't enough. We also need to have some idea of the trial universe we are working in. For example, 1 in a billion events have certainly happened multiple times on Poker Stars, but probably not on NakedPoker. If one happened twice on NakedPoker it would be suspicious (but once really means nothing). Likewise, since 9 heads in a row happens every 512 flips on average, if we see this happen once it means nothing, but if we see it happen multiple times in a small number of trials, or the streak keeps getting longer, it would be noteworthy.

And 100 in a row probably hasn't happened in the history of the universe in any kind of 50/50 wager, coins or whatever. Simply because there haven't been enough trials. If something has a one in a googillion chance, then on average that is how many trials it takes for the first occurrence.

To fully specify the problem you need to know two more things: the prior probability that the coin is fair (this is was NewOldGuy is referring to) and the gains and losses from decisions.

For example, suppose you have a gamble where you pay $10 if you lose a coin flip and win $11 if you win. You think there's a 98% chance the game is honest, but a 2% chance that the coin is weighted so you always lose. Your EV per flip is +$0.50 in an honest game, -$10 in a dishonest one. Since there's a 98% chance the game is honest, your EV for the first flip is 0.98*$0.50 - 0.02*$10 = +$0.29.

If you lose the first flips, the chance that the game is rigged increases to 3.92% (out of 100 games, 2 will be rigged and 98 fair, in 49 of the 98 fair games you win the first flip, so if you lose the remaining chance is 2 / 51 = 3.92% that the game is rigged). So your EV of a second flip given that you lose the first one is 0.9608*$0.50 - 0.0392*$10 = $0.09.

If you lose the first two flips, the game becomes negative EV. You're not sure the coin is rigged, 25% of the time you lose the first two flips in an honest game. But your suspicion level has increased enough that it no longer makes sense to play.

This is probably a good thread to ask this question.

Let's say we know that in the population of coins any given coin can have a "true" heads value of 0% to 100%, but that the expected value (mean) is 50% and the coins are normally distributed.

Question 1: What is the variance (or SD) of this population? How do you compute it?

Question 2: What is the expected variance (or SD) for a coin drawn from this population after three flips?

Now let's say we have a coin from this population and we witness it land on heads three times.

Question 3: Given the answers to questions 1 and 2, can we now combine these two pieces of information to make a prediction about the next coin flip (as follows):

(popMean / popVar + obsMean / expVar) / (1/popVar + 1/expVar) = p(Next flip heads)

And:

1 / (1/popVar + 1/expVar) = Variance of p(Next flip heads)

Where popMean is the known population mean (in this case .50), popVar is the known population variance (Question 1), obsMean is the observed mean (in this case 1.0), and expVar is the expected variance for the observed mean (Question 2).

If this is not right, can someone please explain what is wrong and/or how it can be fixed.

That makes the coin population as a whole equivalent to a random number generator.

Yes. With a mean of .50.

This should make it clear why you can't "calculate" the variance, because it's not a fixed emergent property of the population of coins you describe. You could pick any variance you wanted and construct a population of coins that fits your description. It's a little like saying "I have a population of people with heights ranging from 3 to 8 feet, with an average of 5'10". What's the variance?"

Well it can't be any number. There is a limit of 0% to 100% heads.

But I understand. This was the problem I ran into as well.

So let's change the problem a bit and make another assumption. Let's say .5% of coins land on tails 100% of the time and .5% land on heads 100% of the time. The other 99% of coins are exactly fair. What's the variance here? And if we can solve for it (and assume our coin comes from this population) does the rest of my example hold?

I knew you were going to say that, but somehow I thought you wouldn't at the same time

I don't know that I think computing the variance of your sample makes much sense, because it's clearly not normally distributed. But fwiw, the variance would be
(5*.5^2 + 5*.5^2 * 995*0^2)/1000
because the all-tails coins (5/1000) deviate from the mean by .5 and so do the all heads, the rest don't deviate from the mean at all.
This is variance=.0025 (or .25%)

What's the expected variance of a coin drawn from the population? I *think* this variance will be the same for any coin, whether it's all heads, all tails, or any combo in between. Why? Because if you say that heads is 0 and tails is 1, then the mean is M where 0 <= M <= 1
If we did N flips, where N is large, we expect to get MN tails and (1-M)N heads
V = MN * (1-M)^2 + (1-M)N * M^2
uh, I think that's right. That is, the tails will deviate from the mean by 1-M and the heads will deviate from the mean by M. Normally M=0.5 and 1-M = M but in your case it isn't necessarily so
V = [ MN*(1 - 2M + M^2) + (1-M)N*M^2 ] / N
V = [ MN - 2NM^2 + NM^3 + NM^2 - NM^3 ] / N
V = [ MN - NM^2 ] / N
V = [ MN(1-M) ] / N
V = M(1-M)
And this looks right to me, because, duh, it's the variance of a binomial. Which if I was thinking about it, would be sort of the obvious way to figure the variance.

So I guess the expected variance of any distribution of rigged coins is the variance of a binomial variable the same as the average of your population?

I've had a few beers so I'm not sure if the above is totally obvious, dead wrong, or somewhere in between to be frank, and I am not sure how to approach the rest of your questions.