Designing and Managing MCQs:
Guidelines for the use
of
Question Mark
in the
Faculty of Health Sciences
NOTE: These guidelines were taken from a meeting of Health Science staff
members on 1999/11/18. They are not meant to be prescriptive, and no staff
member is bound by these. The chief reason for these guidelines is to have
some degree of consistency across the Faculty so that the students find the
environment of the test familiar and non-threatening, and are more concerned
with the subject matter at heart than the particular vagaries of the software.
1. For each question. Although the staff member may select any layout,
the three components of the question should be clearly visible to the student:
-
The question itself
-
The mark allocation for that question
-
The choices / selection area
2. Negative marking. Most lecturers do use negative marking. If
the weighting of the negative marking varies at all from question to question,
then the mark allocation should be clearly visible. See point 1 above. For
more information on negative marking, see chapter
4 of this Manual
3. Question types: Matching and Ranking. The following should be applied:
-
1 mark for each correct answer
-
No negative marking
-
No scroll bar necessary in the drop-down list (So 5 or fewer options)
-
If you feel the need for negative marking, then rather break them up into
MCQ questions
4. Question Types: Multiple Response. There are two possible
methods that could be applied, but whichever method is used, these are the
guidelines:
-
There should be negative marking, with the aim being that if a student selects
all the options, then the mark award should be 0. (C x CM + I x IM = 0).
If this cannot be achieved then a sufficiently low mark (e.g. 1/4 or at best
1/3) - should be attempted, thereby making it simply not worthwhile to guess
all).
-
Candidates should not be told how many options in the multiple selection
question are correct, although the mark allocation should be clear.
4.1 Method 1
-
1 mark for each correct answer
-
-1 mark for each incorrect answer
This table shows various examples and possible mark allocations for Method
1.
| Options |
Correct (C) |
Incor-
rect(I) |
n |
Mark for
each correct
(CM) |
Mark for
each incorrect
(IM) |
If correctly
answered |
If incor-
rectly answered |
If all selected
(CxCM + IxIM) |
| 5 |
2 |
3 |
5 |
1 |
-1 |
5 |
-5 |
1 |
| 6 |
2 |
4 |
6 |
1 |
-1 |
6 |
-6 |
2 |
| 6 |
3 |
3 |
6 |
1 |
-1 |
6 |
-6 |
0 |
| 7 |
3 |
4 |
7 |
1 |
-1 |
7 |
-7 |
-1 |
| 7 |
4 |
3 |
7 |
1 |
-1 |
7 |
-7 |
1 |
| 8 |
3 |
5 |
8 |
1 |
-1 |
8 |
-8 |
-2 |
| 8 |
4 |
4 |
8 |
1 |
-1 |
8 |
-8 |
0 |
| 8 |
5 |
3 |
8 |
1 |
-1 |
8 |
-8 |
2 |
4.2 Method 2
A similar method, but that it attempts to use the use the n-1 standard, but
n = Number of incorrect choices + 1 and the points awarded to the correct
answers should be split. Again, the perfect allocation would be that if a
student selects all the options, then the mark award should be 0. (C x CM
+ I x IM = 0). If this cannot be achieved then a sufficiently low mark (e.g.
1/4 or at best 1/3) - should be attempted, thereby making it simply not
worthwhile to guess all).
This table shows various examples and possible mark allocations for Method
2.
| Options |
Correct (C) |
Incor-
rect(I) |
n |
Mark for
each correct
(CM) |
Mark for
each incorrect
(IM) |
If correctly
answered |
If incor-
rectly answered |
If all selected
(CxCM + IxIM) |
| 5 |
2 |
3 |
4 |
2 |
-1 |
4 |
-4 |
1 |
| 6 |
2 |
4 |
5 |
2 |
-1 |
4 |
-4 |
0 |
| 6 |
3 |
3 |
4 |
1 |
-1 |
3 |
-3 |
0 |
| 7 |
3 |
4 |
5 |
2 |
-1 |
6 |
-6 |
2 |
| 7 |
4 |
3 |
4 |
1 |
-1 |
4 |
-4 |
1 |
| 8 |
3 |
5 |
6 |
2 |
-1 |
6 |
-6 |
1 |
| 8 |
4 |
4 |
5 |
1 |
-1 |
4 |
-4 |
0 |
| 8 |
5 |
3 |
4 |
1 |
-1 |
5 |
-5 |
2 |
Which method. The real advantage of method 1 over method 2 is that
it is easier to explain to the students without getting bogged down in the
theory of MCQs. Also it is consistent for each question, and also does not
give an indication of the correct number of answers in each question.
On the problem of a student's answering all and receiving an unfairly high
or low mark; this can be resolved by having a set of these, which overall,
give a score of 0 if all options are chosen.
5. Randomisation of questions from libraries. The following should
be applied:
-
Tests: There may be randomisation of questions from libraries (item banks)
of questions. If students receive different questions, that is not an issue,
although one might feel it preferable to ensure that the maximum number of
marks is the same for all students. If questions, therefore, are banked according
to question type, and questions of each type are of the same difficulty and
the same mark, then this will ensure that the maximum mark is equal for all
students.
-
If randomisation is used, and there is a risk that a student may receive
similar questions, then one might wish to place a group of these into a bank,
and select a small number from that bank.
-
Exams: In order to avoid complications, all students should receive identical
questions. Randomisation may be applied to item banks (i.e. the number of
qestions taken from each bank should be the total number of questions in
that bank), and to the placement of alternatives in the MCQ questions.
6. Feedback. The following should be applied:
-
Tutorials: detailed feedback may be given; in fact, is desirable if the test
is used for teaching
-
Tests and exams: no feedback should be given; in tests might be acceptable
to give marks, but definitely not in exams.
Links to other Chapters:
|Title
Page|Contents|Chapter
1|Chapter 2|Chapter
3|Chapter 4|Appendix
A|Appendix B|Appendix
C|Appendix E|
