Two Plus Two Poker Forums Assigning proper weights for linear weights formula
 Register FAQ Search Today's Posts Mark Forums Read Video Directory TwoPlusTwo.com

 Notices

 Probability Discussions of probability theory

 07-24-2012, 01:32 PM #1 veteran     Join Date: Feb 2005 Posts: 3,353 Assigning proper weights for linear weights formula Hi all, Let me preface by saying that I've never had any formal statistics training, am trying to teach myself, and I'm not sure if my question is even feasible. I'm working on a project where I'm creating surveys for users to take. I'm trying to figure out which questions are the best, so I'm going to develop a "Question Quality Score" (QQS). There are two factors that go into the QQS - the question's answer rate, and the standard deviation of the chosen answers (i.e, a question with stddev=0 is most valuable because it splits up the user population evenly. On the other hand, if I had a yes/no question where 100% people answered yes, that wouldn't help me split up the population at all). My first thought was to just normalize the answer rates and stddev, and then average the normalized values together to get the QQS. However, I suspect that having a low stddev is more valuable than having a high answer rate. How could I quantify my suspicion? My end goal would be to prove that, say, stddev is twice as important as answer rate, then my QQS formula could factor that in.
 07-24-2012, 02:11 PM #2 veteran   Join Date: Mar 2009 Location: In the wires Posts: 2,266 Re: Assigning proper weights for linear weights formula SD is directly related to how many answers you get, not independent.
07-24-2012, 02:20 PM   #3
Carpal \'Tunnel

Join Date: Jun 2005
Location: Psychology Department
Posts: 7,425
Re: Assigning proper weights for linear weights formula

Quote:
 Originally Posted by Colombo I'm trying to figure out which questions are the best, so I'm going to develop a "Question Quality Score" (QQS).
Here is the key to us helping you with your problem. What do you mean by "which questions are the best?" Do you mean the questions that are the most difficult? Do you mean the questions that are the most discriminative (i.e., the questions that best identify differences in the test takers' true skill levels)?

Last edited by Sherman; 07-24-2012 at 02:26 PM.

07-24-2012, 02:50 PM   #4
veteran

Join Date: Feb 2005
Posts: 3,353
Re: Assigning proper weights for linear weights formula

Quote:
 Originally Posted by Sherman Here is the key to us helping you with your problem. What do you mean by "which questions are the best?" Do you mean the questions that are the most difficult? Do you mean the questions that are the most discriminative (i.e., the questions that best identify differences in the test takers' true skill levels)? Depending on your answer, you might look into item response theory.
The questions that are the most discriminative is what I'm trying to accomplish.

Just Wiki'd IRT. Seems like pretty interesting stuff. Going to try to familiarize myself with it now and see if I can find it applicable.

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is OffTrackbacks are Off Pingbacks are Off Refbacks are Off Forum Rules

All times are GMT -4. The time now is 08:40 PM.

 Contact Us - Two Plus Two Publishing LLC - Privacy Statement - Top