Open Side Menu Go to the Top
Register
Super satellite comparison with multiple choice exam Super satellite comparison with multiple choice exam

08-28-2012 , 03:30 AM
Multiple Choice Exam - You receive 2.5 marks for a correct answer, and are deducted 0.7 marks for an incorrect answer. There are 34 questions with 5 answers per question.

So, if you have no idea about a question and take a totally random guess, you're expected result from that question is (0.2*2.5)+(0.8*-0.7) = -0.06

As is the case in most MCEs, you will usually have some sort of inkling for questions - if you can just rule out one answer in a question, you're expected result goes from -0.06 to (0.25*2.5)+(0.75*-0.7) = +0.1. So obviously the more sure you are about a question, the higher this number goes. This definitely indicates that guessing even when not totally sure is a better strategy than only guessing when 100% sure. Only when you have absolutely no idea about the correct answer should you not answer the question.


So anyway, this particular exam I am taking, is a repeat exam. That means the grade is capped at a C3 (i.e. the minimum grade you need to pass). So if you go in, answer all questions correctly and get 100%, you are still only awarded a grade of C3 (a C3 is between 40-44% btw).


How does this impact how you should "guess"? Does it mean the less +EV options now become 'no answers' to 'reduce your variance'? Basically, there's nothing to be gained by achieving anything higher than 40%.

To draw a poker parallel, it's similar to playing a super-satellite, once you have enough chips to win a ticket, any additional chips won are basically worthless. Likewise, when approaching the bubble with an average stack (ie when approaching what you perceive to be ~40%), how much of a +EV spot do you need to gamble to win enough chips to guarentee yourself a ticket?

Last edited by Jam-Fly; 08-28-2012 at 03:39 AM.
Super satellite comparison with multiple choice exam Quote
08-28-2012 , 04:19 AM
Obviously if you know you have 40% and you need 44% or 42% but most definitely you are sure you do not have enough you need to gamble as much as possible. But the ethical thing is to not gamble and to answer to the best of your knowledge and if you fail study better and win the right way because at some future point in life you will pay for getting lucky if so here. There is a reason there are exams out there to test us. The degree is not the prize, knowledge is.


That said of course returning to the cruel objective of the original problem here and leaving ethics aside the proper problem to consider is this;

You need a score N and you have M<N and you have K problems left and you want to know how many of them to try to guess if not all. I mean at some point the fact EV is negative (if thats the case, otherwise if you can take it to 4 guesses or 3 its advantageous to answer as much as possible of them) makes it a bad idea to answer many of them.

For example the distribution if you answer k of them goes like a normal of (k*e,sd*k^(1/2)) . Your problem becomes where to stop basically because if you overdo it the chance to fail is increased actually. You need to select k optimally.

So the problem becomes a bit more interesting. I will try to formulate properly to try to answer a more general situation of that type of problem where failure is all the same and you no longer care about the absolute points EV of the strategy just to maximize the probability of being above the threshold. I will return to it or someone may pick it up from the formulation i am suggesting that will try to answer more general problems of this kind.

Last edited by masque de Z; 08-28-2012 at 04:36 AM.
Super satellite comparison with multiple choice exam Quote
08-28-2012 , 05:15 PM
Very roughly it seems unless i missed something (because the real calculation is a lot more complicated and of course necessary when it matters due to the nature of distribution of each answer scoring system) if your score is say x and you need to reach b>x and you have n questions left with EV -> E<0 and SD->s each then you basically try to minimize the expression;

(b-x-E*n)/(s*n^(1/2)) in order to see at what n you stop when E<0.

Last edited by masque de Z; 08-28-2012 at 05:35 PM.
Super satellite comparison with multiple choice exam Quote
08-28-2012 , 07:23 PM
thanks for the reply.

As it happened, I took the exam today. I actually got some of the marking wrong, so it changes some specifics a bit but still the same principle.

As it actually happened, it was 2.94 for a correct answer and -0.6 for a wrong answer (so it was actually always +EV to give an answer). Funnily enough though, just because it was +EV, I still didn't answer a lot of questions, because the risk-reward was not right.

The exam was 90% of the module. You need 30% in the module to receive a D2 or D1 which is a 'compensating fail' (this means you are allowed pass the module if your overall GPA is above a certain point - mine was). You need 40% in the module for an outright pass - of course an outright pass increases your GPA more than a 'compensating fail' would.

This actually adds to the problem a bit. Anything <30% is an absolute disaster and must be avoided at all costs, 30%-39.9% is an acceptable result but 40%+ is the aim, (however anything over 40% is not counted).

(so, just to be clear, since the exam is worth 90% of the module, I needed 34% in the EXAM for 30% in the module, or 45% in the exam for 40% in this module).


At the end of the exam, I was in a situation where I had 13 answers I was 95%+ sure were correct. 10 answers I was confident in (to assign a ball-park figure, I would have been between 40% and 80% sure of the various answers. However, I didn't work out exactly how confident I was of each of these answers in the test, this is just a rough estimate), and 11 answers I wasn't very confident in (still probably confident enough for them to be a very +EV guess, but given that anything over 40% is needless, I decided not to answer them).

So, when deciding what to actually answer, my thought process went like this...

13*2.94=38.22% (remember, 34% is the minimum needed, and 45% is the desired). So if I just answer the questions I am totally sure, I'm almost a cert to get greater than 34%.

But there is some reward for getting higher than 34% up to the point of 45%.

If all my guesses were incorrect, this is the result:

(13*2.94)+(10*-0.6)= 32.22%

32.22% is a total disaster meaning I fail the module. Although this is very unlikely (in all 10 questions, I'm a minimum of 40% sure, so to get all 10 questions wrong would be 0.6^10 which is less than 1% chance of happening), I still felt it was not worth the risk. It's difficult to quantify the difference between the reward of getting a D1/D2, the reward of getting a C3, or the risk of outright failing. Basically, outright failing would be a massive disaster. The difference between a D2 and a C3 was not negligible - (the higher the GPA the better), but it was not a massive deal either (obviously if I could quantify these precisely it would make things bit easier).


So, I decided that I could not risk dropping below 34%. This meant

(13*2.94)+(X*-0.6)=34%

X was the amount of questions of the 10 I could guess (effectively freerolling the D2 while attempting to reach the C3)

So, X works out to be 7 (I obviously eliminated the 3 questions I was least sure about, changing those questions to 'no answers') giving

(13*2.94)+(7*-0.6)=34.02%

So this means, that assuming all my 95%+ answers are correct (tbh, I didn't factor in that I was only 95%+ sure, I assumed I was 100% in these 13 answers**), then even if all my 7 'guesses' are wrong, I am still guaranteed to get a D2 in the exam.

So I need 3 of my 7 guesses to be correct to get the score I desire:

(16*2.94)+(4*-0.6)= 44.64% (I rounded up earlier, so 44.64% of the the 90% exam actually gives a total score of 40.176% in the module, meaning a C3)


If we say, on average, I was 50% confident in my 7 guesses, this gives an expected result of:

(13*2.94)+(7*((0.5*2.94)+(0.5*-0.6))) = 46.41% which equals 41.77% of the module, meaning the C3 is secured.


So in summary, my 'equity' in the exam was 46.41% (45% was the aim) whilst giving myself a nearly 0% chance of outright failing (which was less than 34%).



**At the time, I did not factor in the possibility of any of my 13 'bankers' being wrong. Let's say for 10 of them, I was close to 100% sure, for the other 3, I was 95% sure. This is not a negligible difference. Being 100% sure for 10, and 95% sure for 3 means there's a 1-(.95*.95*.95)=14.26% that at least one of the 13 answers is actually wrong - (a 13.54% chance one is wrong, a 0.71% chance two are wrong, and a 0.01% chance three are wrong).

This changes the figures a bit. 13*2.94 was assumed to be 38.22%, but a more correct expected value of these 13 answers would be:

(10*2.94)+[(.8574*3*2.94)+(.1354*2*2.94*1*-0.6)+(0.0071*1*2.94*2*0.6)+(0.0001*3*-0.6)] = 29.4% + 7.11% = 36.51%

36.51% is significantly different to the expected value of 38.22%

It means that if all my 7 guesses are incorrect, I receive a score of:

(36.51)+(7*-0.6) = 32.31%

and my expected result is

(36.51)+(7*((0.5*2.94)+(0.5*-0.6))) = 44.7%

these are noticeably different to my scores of 34.02% and 46.41%

So basically, I either need my 3 95% answers to be correct, or I need just one of my 7 (which I gave a ball park figure of 50% on average to) guesses to be correct.

I still like my chances of not failing, but I'm now aware of the fact the possibility of failing is still above 0%!!





Hopefully all of the above is correct. Very liable to make mistakes or work something out incorrectly, so please correct me if I did!
Super satellite comparison with multiple choice exam Quote
08-30-2012 , 10:24 AM
If you know the answer to 48% of the questions, gamble it up, close your eyes, and fill in the rest randomly to give yourself a sweat. Where's the fun in knowing that you passed? Gotta have a sweat.
Super satellite comparison with multiple choice exam Quote

      
m