Scoring types

Introduction

The distribution of points for multiple-correct questions requires more attention than the normal multiple-choice questions. With multiple-correct questions, candidates can choose multiple answers and decide how many answers they choose. This affects the point distribution of the multiple-correct question.

To complete the standardization of this question, RemindoTest offers four options for the distribution of points:

• Correct answer
• Score distribution
• Quotient rule
• Product line

These options determine when the candidate gets a point (or points) for this question and what this result should be. It therefore answers the question: in what way does the candidate become equal?

As an example, here is a multiple-correct question:

Some countries are listed below. countries Which are in Europe?

a. Canada
b. Netherlands
c. Morocco
d. Montenegro

Correct antwoord

The 'correct answer' option means the following:

"When the candidate chooses exactly the correct answer, he will receive a point for this. In case of a (partially) incorrect answer, no marks will be awarded."

In this case, if the candidate gives the correct answer to the sample question, he gets a point for choosing answer b in combination with choosing answer d. Any other combination earns no points.

There is a major drawback to this choice. If the candidate knows one answer option for sure and chooses a second one, no points are awarded, even though the answer to the question is partially correct. The validity of the question thus goes down.

Score mapping
The 'score mapping' option means:

"When the candidate chooses the correct answer, he gets a point for this. If the candidate chooses a wrong answer, a point will be deducted."

Compared to the 'correct answer' option, here something is also done with wrong answers. The candidate who ticks all the answers gets no points (2 points for correct answers - 2 points for wrong answers = 0 points).

A candidate who selects three answers in the sample question scores one point (i.e. 2 points for correct answers - 1 because of wrong answers = 1 point).

The disadvantage of this option is that it does not take into account the number of right or wrong answers in total. For example, if a candidate in the sample question chooses answer b and c, he gets 0 points (while one option is right), if he chooses answer a, b and c, he gets 0 points and if he chooses only wrong answers, he also gets 0 points. This ensures that the question has little distinction in this way and is therefore not very effective.

Quotient rule
To overcome the point distribution problem in multiple correct questions, there is the quotient rule. The quotient rule uses a formula that takes into account the proportion of correct and incorrect answers. The formula is as follows:

= proportie goed = aantal juist gekozen antwoorden gedeeld door aantal juiste antwoorden in de vraag
= proportie fout = aantal fout gekozen antwoorden gedeeld door het aantal foute antwoorden in de vraag

De formule lost veel problemen op met betrekking tot de scoreverdeling bij meervoudig-juistvragen. Het kiezen van deze optie houdt in: "I first weigh how much the candidate's correct answer is worth and I do the same for the wrong answer. The ratio between these two numbers gives the score."

A candidate who chooses answer a, b and c on the sample question thus gets a score of 0.25. If the candidate chooses answer a and b, he gets a score of 0.33. It provides a very nuanced picture of the value assigned to the candidate.

However, this rule has one drawback: when the candidate ticks all the answer options, it always results in a score of 0.5. This is therefore more than when the candidate specifically chose answer a and b. This could encourage candidates to tick all the answers anyway, if they are not sure. This can be prevented by choosing multiple answers (limited) in RemindoToets and specifying a limit for the number of answers the candidate can choose.

Product line
With the product rule it is not necessary to specify a limit for the number of answer options that can be chosen. In the product rule, the proportion of error is not counted in its entirety, but is compared to the number of correct answers. The formula is as follows:

= proportion correct = number of correct answers divided by number of correct answers in the question
= error proportion = number of incorrect answers divided by the number of incorrect answers in the question
= the total number of answer options
= the number of correct answer options

Choosing this option means the following:

How ​​severely an incorrect answer must be penalized depends on the number of correct answers and the number of answer options in total. The comparison between the values ​​of the correct and incorrect answer yields the score.

In the product rule, the extent to which the proportion of error is taken into account and leads to a deduction varies. This makes it less useful for the candidate to tick all the answer options. It leads to a better balanced score.

Different options, different scores
What do the 4 options yield in terms of score when we look at the sample question? The matrix below shows the results for different answer combinations. There are no bonus points in this example. The candidate can get a maximum of 1 point for this question.

As an example, we use the following multiple-true question:

Some countries are mentioned below. Which countries are in Europe?

a. Canada
b. Netherlands
c. Morocco
d. Montenegro

Outcomes
Based on a total score of 1 point.

• The product rule compares the number of wrong answers with the number of correct answers, therefore using the product rule results in a lower score for these answers.

** In this example, you can see that the effect of the product rule is better than that of the quotient rule. This is an example of a candidate who ticked all the answers.

Pre-indicate how many answers are correct

For multiple correct questions, it is possible to indicate in advance how many answers are correct or to leave this blank. It only affects the score distribution if a limit has been set. This is possible in RemindoToets by choosing multiple answers (limited). When a limit is active, all norming options can be chosen.

Communicating in advance to the candidate how many answers are correct also affects the difficulty of the question.

Disclaimer: This text was automatically translated from the Dutch version.