Students often ask me why they get lower scores than expected when they make
mistakes on Quest. Allow me to explain the rationale.
Suppose we simplify Quest a bit so that every question is multiple-choice,
with exactly two possible answers. What would be an appropriate "monkey score"?
That is, what score would you think is the appropriate measurement that should
be assigned to a monkey who is picking answers randomly without any knowledge of
the material? You could set any kind of measurement scale, of course, but to me
(and to the designers of Quest) it makes sense to report that this monkey has
"zero" knowledge of the material by reporting a score of zero.
Of course, the monkey might get some answers correct simply by chance; indeed,
we expect statistically that monkeys will get about half the answers correct
if there are only two choices. So if we simply assign (say) 10 points for a
correct answer and none for an incorrect answer, the monkey will get a positive
score. If for example there are 30 questions, the monkey would get about 15
correct, garnering 150 points out of the 300 possible, with a score of 50%.
To prevent this from happening, Quest chooses to deduct 10 points for every
wrong answer. Thus the monkey will probably have about 15 correct answers
(earning 150 points) and 15 incorrect answers (losing 150 points), with a
net score of zero.
Unfortunately for some of the monkeys, there is no guarantee that they will
get 15 (or more) questions right by chance. If they have fewer than 15 correct,
they will end up with a negative score.
You, who have considerable knowledge of the material, are unlikely in general
to get a negative score in this case, but you will still lose points for your
wrong answers; if for example you had 20 correct answers and 10 incorrect ones,
you will end up with a net of 100 points, a score of 33%. This reflects
reality: a person who knows 1/3 of the questions would get 10 of the 30
correct on that basis; s/he would guess at the other 20 questions and can be
expected to get half of those right by pure chance, thus getting a total of
20 correct answers, just like you. Your score reflects the number of questions
that you actually KNOW the answer to, not the number of questions you
manage (sometimes by chance) to answer correctly.
The situation is more complicated when there are more than two candidate
answers. If we decide to award P points whenever the question is answered
correctly, then each incorrect answer should cost P/N points, where N
is the number of wrong answers (i.e. there are N+1 possible answers per
question). If there are Q questions, a monkey would expect to get Q/(N+1)
of them right on the first guess, gaining PQ/(N+1) points for the right
answers, but then the Q - Q/(N+1) = Q N/(N+1) wrong answers would cost
P/N points each, for a total loss of P Q / (N+1) points for the wrong
answers, and thus an expected gain of zero for the initial guesses. Further
rounds of guessing (with the decreasing pool of responses to choose from)
will each also produce an expected gain of zero points, and thus monkeys
will end with zero points (on average). The value of P can change
between rounds, that is, the "monkey score" will end up being zero as
long as the penalty at each stage is 1/N of the reward available at
that stage, where N is the number of distractors (wrong answers) available
at that stage.
It may be disorienting to get lower numbers than you expect, and most students
still end up with scores in the 80s or 90s by the end of the semester, but
this is the way we ensure you get an expected benefit of zero for just guessing
answers to questions about which you are in fact clueless. It's much better
not to be clueless in the first place! Ask for help if you don't know what's
going on!