i give you 1.5 inconsistency score for not having an incoherence score.
was the inconsistency score just a rating of how much our answered changed when we had to deal with balls in a jar instead of cards?

i thought at least one question was badly worded. didn't understand what it was asking. cards relation to other card (but i think it missed key further info... i had to assume what you were asking)

the percentile score surprises me. 12th percentile i got.. but i was top student at big state university. took 2nd year probability and other math courses. and i use this type of stuff all the time. i did the math in my head. didn't get out pen and paper.......... that percentile just doesn't make sense to me. i am thinking of course that your universe of participants may be high mathematical people. so maybe. it was easy to make mistakes too. but surely that's better than just doing a pure non-quant estimate.

anyway, good survey. glad to participate.

Thanks a lot for your responses everyone! We will reveal details about how the scores are calculated, along with the distribution of 2+2 scores, when we close the survey in 10 days time.

I can also answer questions about the survey's content at that point.

2.268
.895

Pretty sure I'm the worst score.
Had I been asked about my confidence in my answers, however, I would have guessed I would score in like the bottom 10 percentile, but not the .00.

You guys did math? I saw the 60 second clock and began to give it my best intuitive guess from the get go.

0.347
0.085

"You did better than 82% of non-expert poker players from a previous experiment in terms of accuracy, and 83% of non-expert poker players in terms of consistency."

I question the scores and/or conditions of the previous probability experiment, as I would expect to score way higher than top 82/83% of non-expert poker players (i.e. the general population.) The probability of that being correct is extremely low!

EDIT: I notice someone else above made a similar comment. I also worked out the answers in my head, and vs non-expert poker players, any competent "number sense" guy would *easily* be in the top 5%. (Speaking as an ex-teacher of mathematics and ex-poker pro.) 90% - 95% of the population have a very poor grasp of probability.

I'm better than 31%!

Two of the follow-up questions are interesting. I'd like to know the "correct" answers.

Yeah, something interesting appears to be going on with the sample. I have no illusions that us 2p2ers are exceptionally good at probabity, but compared with the general population, I feel like most would be at least in the top half. My best guess would be that everyone who's completing the survey are in OP's very mathy circles? Or maybe just the only people reporting on here are the people who do the worst? I guess the type drawn to do a statistics survey are very likely to be pretty statish. For .08 to be 83rd percentile, there must be a fair portion of people who hit perfect consistency.

1.47 and .686

Guess I'm a ****** ******. Or there's too many smart ppl here.

"Your inaccuracy score is 1.152


Your inconsistency score is 0.368


You did better than 4% of non-expert poker players from a previous experiment in terms of accuracy, and 48% of non-expert poker players in terms of consistency."

I think it's possible that a good number of people will opt out when seeing the questions. At least one former poker player I shared the survey with did so.

Got 0.784 and 0.18.

im better than 8% of non expert players in accuracy even though i'm almost as good as a super computer at poker, ok. Is the results part of this test designed to decimate someones ego or what?

Your inaccuracy score is 0.989
(the smaller the number, the more accurate you are with respect to true probabilities; 0 means perfect accuracy).


Your inconsistency score is 0.075
(the smaller the number, the more consistent you are with respect to probability theory; 0 means perfect consistency).


You did better than 10% of non-expert poker players from a previous experiment in terms of accuracy, and 84% of non-expert poker players in terms of consistency.

Dang it, I thought I was better at probability than that!!!

Yikes. I did awful, and I took three years of stats in college.

Great survey, but obviously I'm an idiot doing math on the fly. Best of luck!

Lot of good poker players who've gotten pretty **** scores itt. hmm

inaccuracy 0.512
inconsistency 0.324

Interesting survey, several times in the middle of it when I knew my answers didn't add up coherently but I felt the time pressure and just had to try to try not to mess up too hard.

Thanks to everyone who has taken part! The response really has been incredible.

The survey will close on Thursday (March 5th) -- probably in the morning US time.

I'll do a full debrief then.

Your inaccuracy score is 1.105
(the smaller the number, the more accurate you are with respect to true probabilities; 0 means perfect accuracy).


Your inconsistency score is 0.405
(the smaller the number, the more consistent you are with respect to probability theory; 0 means perfect consistency).


You did better than 5% of non-expert poker players from a previous experiment in terms of accuracy, and 43% of non-expert poker players in terms of consistency.



Never took stats, tried to figure some on the fly, realized I had some better answers later. Oh well.

Your inaccuracy score is 0.784
(the smaller the number, the more accurate you are with respect to true probabilities; 0 means perfect accuracy).


Your inconsistency score is 0.84
(the smaller the number, the more consistent you are with respect to probability theory; 0 means perfect consistency).


You did better than 26% of non-expert poker players from a previous experiment in terms of accuracy, and 6% of non-expert poker players in terms of consistency.


So i'm one of the most inconsistent one here .
Very curious about how this is calculated.

Your inaccuracy score is 0.663
Your inconsistency score is 0.325

Yes, you tried to make a living playing against competent poker players while being better than 11% of non expert poker players. Are you still trying?

I wanted to test myself but its closed now. Can you just open the survey without collecting new results?

I must be finding the 10% dumber than me to play against cause I keep winning! Probably guys like you!

Thanks so much to the hundreds of 2+2ers who have taken part in this study. The response has really been way better than expected and you’ve given us lots of great data for this research. The original link will no longer work, but we’ve created a copy here that you can use to go back and look at it here.

I know a few of you were a bit disappointed with how your scores compared against the previous sample. There was an approximation used in these comparisons so the comparison you got may not have reflected your true rank.



As you can see, the 2+2 sample was overall much better at this task than the previous recreational poker player sample. The vertical lines are median inaccuracy scores and incoherence scores. The 2+2 sample achieved a median inaccuracy score of 0.314, whereas the other group has a median inaccuracy of 0.589. The incoherence scores also suggest the 2+2 sample were better with a median of 0.15 whereas the other group has a median of 0.35.

The 10 questions at the end came from some previous research (“The Relationship Between Gambling Fallacies and Problem Gambling”). The 2+2 sample scored an average of 8 items correct, compared to 6 items in the recreational player sample. There’s a lot of previous research in gambling suggesting that disordered gamblers tend to endorse many incorrect beliefs about probability. For example, disordered gamblers (as do people in general, but disordered gamblers are more likely than a normal person to score badly on this test) tend to often believe that a lottery is more likely to be drawn with the sequence:

8, 18, 3, 55, 32, 28

Than with the sequence:

1, 2, 3, 4, 5, 6

Question 8 in that survey is as follows:

Which game can you consistently win money at if you use the right strategy?
a. Slot machines
b. Roulette
c. Bingo (Baccarat for Asian populations)
d. None of the above

Someone did point out to be that roulette can be beaten if you have a wearable computer that can predict the ball’s final location. While this is true, it’s also highly illegal everywhere, and so d) is scored by the original researchers as the correct answer to this question.

We’re can also answer any other questions you might have at this point.