Designing and Managing MCQs:
Chapter 2. What are Multiple Choice Questions?
Contents of This Chapter
2.1 Definition
A "multiple choice question" (MCQ) is a question in which students are asked
to select one alternative from a given list of alternatives in response to
a "question stem".
For example,
Question stem:
"Which one of the following is the currency unit used in Albania?"
Alternatives:
-
Dinar
-
Rouble
-
Kwacha
-
Lek
-
Rupee
The student answers the question by selecting a number 1-5, and entering
this either on a form provided for the purpose ("off-line", in computer jargon),
or at a computer keyboard ("on-line"). In this example, the correct alternative
(the "answer") is 4. The other incorrect alternatives are sometimes called
"distractors".
One feature of this type of question is that there is a finite possibility
of selecting the correct alternative by purely guessing. In fact, in this
example you have 1 chance in 5 of answering the question correctly if you
know absolutely nothing about the currencies in use in various countries.
This problem is not insurmountable, and will be discussed further in
Chapter 4 .
The so-called "TRUE/FALSE" question, in which the student chooses between
two alternatives "True" or "False", is a special case of an MCQ. "True/False"
questions are used extensively in the UCT Medical School.
Certain MCQs have more than one correct response in the set of alternatives.
Such questions should not be used, not only because they tend to confuse
students, but also because not all computer programmes available at UCT cater
for such a question type. For example, the programme "MCQ", which marks the
responses of students who have written the test "off- line", does not admit
this type of question. On the other hand, the programme
"Question Mark", which is designed
principally for "on-line" testing, does allow a much broader range of question
types. These questions, however, are not, strictly speaking, MCQ's. As a
result, they fall outside the scope of this document.
Return to Contents of this Chapter
2.2 Advantages of Multiple Choice Questions.
-
MCQs can be easily marked/scored, and this scoring can be both accurate and
objective. In particular, MCQs can be MACHINE SCORED (This means that a computer
will do all the marking for you!)
-
MCQs can be set at different cognitive levels. For example, a question may
simply challenge a student's ability to recall facts, while another may test
a student's ability to apply factual knowledge to given situations; or, at
a higher level, a question may test a student's ability to evaluate given
information. For a discussion of this, please turn to
Appendix C .
-
MCQs can be designed with a diagnostic end in mind, in order to find out
whether specific areas of a given subject are adequately known, or in order
to detect misconceptions. This can provide feedback on the effectiveness
of instruction.
-
Students with poor reading skills and second-language learners need not be
disadvantaged, provided the tests are designed appropriately Misreading of
a question stem or an alternative may result in the loss of a few marks only;
conversely, the misreading of an essay-type question, followed by the poor
writing skills normally associated with a second language, can cause the
student to lose an inordinate number of marks.
-
Statistical information on performance can be readily obtained. One can not
only find out how the class performed on a particular question, but determine
whether the question was suitable in the context in which it was presented.
The statistics that are gathered may also be used to rank questions with
respect to their difficulty and their ability to discriminate between students
of different competences.
-
Tests made up of MCQs can be administered frequently, and thus provide regular
information on student performance, not only to the instructor but also to
the students.
-
Tests made up of MCQs can provide a better coverage of content than essay-type
questions, thus broadening the scope of the test. Such tests can be structured
to include questions of defined grades of difficulty and discriminating power
(See Chapter 4 ).
-
In MCQ testing, the instructor "sets the agenda", and there are no opportunities
for the candidate to avoid complexities and concentrate on the superficial
aspects of the topic, as is often encountered in essay-type questions.
Return to Contents of this Chapter
2.3 Problems associated with Multiple Choice Questions.
-
The construction of good test items (questions) requires special care and
is therefore time-consuming.
-
Instructors tend to favour "recall" type questions, as they are the easiest
to design. This manual will provide you with examples which should convince
you that it is possible to go far beyond this type of question.
-
Experience has shown that in some environments, it may be necessary to win
the acceptance of students to this type of testing. Proper communication
with the students is therefore essential in order to ensure that they feel
comfortable with this type of testing. Feedback from students can be very
important and can only lead to an overall improvement of the existing question,
for possible inclusion into a question bank.
-
Creativity cannot easily be tested (the SYNTHESIS level of Bloom's taxonomy
is very difficult to test by using MCQs). Creativity is best tested by discursive
questions such as the "ESSAY-type" question. Bear in mind however that essay
questions tend to have a poor sampling of content, have a poor scoring
reliability (particularly if several persons are marking large numbers of
scripts), and are notoriously time consuming to mark. (Bloom's taxonomy is
discussed further in Appendix C ).
-
Candidates cannot justify choices that have been deemed "incorrect" by the
examiner. Two ways (apart from the necessary clarity of language in the stem
and alternatives) of removing this problem are:
-
[a] building the argument into the alternative. Examples of this are given
in later sections of this document.
-
[b] allowing students to discuss the questions with the lecturers and tutors
after the test. In addition to allowing issues to be clarified, the questions
might possibly need to be ignored in the calculation of test scores, and
corrected for or eliminated from future tests.
Return to Contents of this Chapter
2.4 The multiple uses of Multiple Choice Question tests
The most obvious use of MCQs is in formal class tests and examinations, and
this use is covered elesewhere in this handbook. In this section we want
to draw your attention to a number of other uses, such as:
-
Designing a tutorial around a few MCQs. They can serve to guide the tutor;
stimulate discussion (perhaps in groups) and test understanding. Students
who are normally reticent to speak may do so in defence of their choice of
answer. The tutor prepare explanations prepared for both the right answer
and distractors. Again, thereaction of students should be used to improve
the question data bank where necessary.
-
MCQs can be used in a variety of ways in lectures. For example, MCQs can
be put on overhead (OHP) transparencies and students individually or in pairs
can come up with answers. This can help to facilitate interactive lecturing
and usefully 'break' up lectures. If the distractors are carefully selected
to 'catch' misunderstandings, then the response of the class can be used
as a diagnostic indicator. MCQs could also be used to test prior understanding
and to break the ice when starting a new topic.
-
Short tests - e.g., of 5 MCQs - can be administered and marked by computer
to provide a quick diagnostic feedback on the progress of a class, or as
a revision exercise to prepare for exams, or as a bridge into "essay" tasks.
They need not carry marks or have the students' names on them: the lecturer
could follow up with a general explanation of why one answer wascorrect and
the others were not.
The above suggestions represent just a few of the possible uses of MCQs.
Their use in wider contexts than just formal testing is important because
it serves to familiarise the students with a range of answering techniques
and the MCQ process in general. The methods needed to answer different kinds
of MCQs should be made explicit to students and discussed with them. For
a variety of reasons there may be considerable resistance from students to
MCQs and it is essential that the benefits of MCQs are clearly demonstrated
to them.
Return to Contents of this Chapter
2.5 Managing the use of MCQs at departmental and course team level
Limiting the use of MCQs to one or two individuals in a department is
problematical for three basic reasons:
-
the use of MCQs should be taken into account when designing courses, which
implies the course team should be involved;
-
it is unlikely that an individual lecturer can effectively develop a bank
of properly evaluated questions which are used in conjunction with a range
of assessment methods;
-
winning acceptance of MCQs among the students is crucial and this entails
action at the level of thedepartment. Each department will have its own problems
and possibilities with regard to the setting up of MCQ banks and integrating
MCQs into the design of courses. The section "Where to get help" indicates
who on the TAT programme is available to work with departments.
Return to Contents of this Chapter
Links to other Chapters:
|Title
Page|Contents|Chapter
1|Chapter 3|Chapter
4|Appendix A|Appendix
B|Appendix C|Appendix
D| Appendix E|