Previous psychology research has shown that people’s intuitions often violate the laws of probability. We are university researchers interested in whether expert poker players show these same biases in their intuitions.

This survey involves judging the relative frequency of card combinations, as well as making other judgments and answering demographic questions. Please answer based only on your memory and intuitions, and please do not attempt to look up the answers or use calculation software.

We will give you two performance scores at the end of the survey. One score, your “inaccuracy score”, tells you how far you were from the correct answers. The other score, your “incoherence score”, tells you how strongly your judgments violated the laws of probability. At the end of the experiment we will also tell you your relative rank against a group of recreational poker players who already took the survey.

Feel free to post your scores in this thread to see how well you did against other members of the 2+2 community.

Finally, so that you can see how you performed relative to the entire 2+2 community, we will also post the anonymous distribution of performance scores in this thread after we close the experiment.

Please do not discuss the questions in this thread as that will interfere with the experiment and give others an unfair advantage.

The survey can be found at:

http://warwick.co1.qualtrics.com/jfe...2WCMla8AP4BRT7 ***

It should take less than 15 minutes to complete.

+++

Edit/MH:

*** Please note that the original survey link has now been closed, but below is a copy to use if you still wish to see what the survey contents were:

https://warwick.co1.qualtrics.com/jf...h5JfQwOWDPX8IB

++++

Edit/MH:

Here’s some updated information on me.

I’m currently a postdoctoral researcher in gambling studies at an Australian university. Most of my research investigates the tactics used by the gambling industry to exploit recreational gamblers’ biases. I think this is an interesting and productive use of a typical 2+2 skillset (but definitely not the best paid). A lot of gambling researchers by contrast have little personal experience of gambling.

The above study is a part of a more basic research program some colleagues are conducting, which is investigating people’s ability to make rational probabilistic judgments in general. My colleague Jianqiao Zhu has done previous studies both on members of the public and on recreational poker players, and is happy to answer in this thread any questions you have about this research in general.

Please do not discuss aspects of the study’s content in this thread while we are collecting data, as this would contaminate the results we get. You are more than welcome to post your performance scores, and whether you found the study interesting or challenging (or not).

I’m also very happy to answer in this thread any questions you might have about the world of gambling research in general.

What is a high or low inaccuracy score? Do you believe putting the 0-1000 constraint on it creates biases when people are "guessing"? I'm not sure why you didn't choose to ask in terms of %'s.

Well the lowest (best) score is zero. You'll be shown how you rank against a group of recreational players at the end.

There's some previous research suggesting that getting people to make probability forecasts out of whole amounts (e.g. 1000) leads to better responses than asking for a decimal between 0 and 1.

What do you mean by "intuitions often violate the laws of probability" ?

Can you elaborate what that statement means ?

Ok, what's the worst possible score though? 0% "better" means one of the absolute inaccurate scores out of your whole sample?

Very interested and will partake tomorrow but must go to bed. PM so I remember if you like.

Sure. There have been many examples.

Perhaps the best well-known is called the Linda problem.

People read the following description:

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

And then frequently report (via multiple methods), that it is more likely that:

"Linda is a bank teller and is active in the feminist movement."

than it is that:

"Linda is a bank teller."


This is even though the more detailed description is a conjunction of two events, which cannot -- via the laws of probability -- be more likely than the constituent event that "Linda is a bank teller."

That's one that has caused a lot of debate/controversy. But there are many others.

You did the whole survey? What was your feedback at the end?

This is interesting, but it seems that it may expose people’s lack of logical thinking more than their lack of understanding of probability, although I suppose they are intertwined.

Hi Everyone:

This is from our author Philip Newall. I’m sure any participation will be much appreciated.

Best wishes,
Mason

Here's another example from my own research.

Electronic gambling machines in various jurisdictions have to include information on the gambler's long-run expected return. The gambling industry prefers to present this information in a format called the "return-to-player," where:

return-to-player % = 100 - house-edge

so that a return-to-player of 90% means that you'll on average lose $1 for every $10 bet.

we gave participants one of two equivalent pieces of information, say either a return-to-player of 90% or a house-edge of 10%. Participants systematically rated their chances of winning as higher under the return-to-player than house-edge format, even though the underlying information is the same.

This is an example of a "framing effect." You should be able to make the same judgment no matter how the information is provided.

Sounds like one of those psychology studies that made it to popular culture but has nothing to do with reality and attempting to replicate it in fair conditions would fail.
I mean, many people have very cloudy concept of probability in the first place, if you read things fast to them it's easy to confuse them. I wouldn't attach any value to result of such studies unless they are replicated at least a few times and conditions the question is asked in are clear and fair (giving people time to consider it, giving them a text of a problem and not just reading it fast to them etc.).

Did the test. I was just below average in both categories, so now I am rustled. In my defense: the last time I played poker was 6 years ago and I'm over 50. I'm sure the young and active will do better than me.

EDIT: My scores were 0.6 and 0.4 IIRC.

What would the score be of someone who consistently guesses 125% of the correct answer?
(so he says 125 when the correct answer is 100 and in his similarity again, if he previously said 100, now he says 125).
Just to get an idea what scores are worth.

done! Needed to dig deep as I don't play poker regularly anymore..

scores
1.124

0.428

I adjusted correctly as the test got on..

cool survey, few years back I was working on phd in formal epistemology (sadly, never finished) and modelling cognitive biases connected to probability was one of my favourite research areas, it can be fascinating to observe how poorly our brains are adjusted to deal with concept of probability

unfortunately, I was taking the survey at work and I had stop right at the start of 2nd section, so close to half of my answers were timed out, maybe add a one-time stop to the survey to deal with cases like this? These days, it's hard to demand 15 minutes of people's undivided attention

I also wrote it in feedback already, but some questions were imprecisly worded, for example ...


< mod snipped discussion of questions >

1.219 and 0.227

Fun quiz. Would do again once I clear the probability cobwebs from my brain.

Your inaccuracy score is 0.717
Your inconsistency score is 0.363

I guess you missed the part quoted from the OP below since it was in bold.

Yes, I am going to edit out that part from Tutejszy's post and from Eponymous's reply.

Sorry about that.

yup, sorry about that, missed that part from OP

That's hilarious! Those 2% that I did better than must really be donks.

0.642 inaacuracy
0.66 inconsistency

Are the practice questions used to remove data from the survey where people get them wrong?

Phillip - Have you read "Thinking Fast and Slow" by Daniel Kahneman? I'm only a few chapters in at the moment but I see some similarities so you may find it interesting.