Multiple-Choice Questions in Teaching and Assessment

What can multiple-choice questions assess?

As an assessment format, multiple-choice questions can cover a broad range of content and test students on large parts of the syllabus in a relatively short time. There must be a clear connection between the established learning outcomes and the multiple-choice questions.

One useful approach is to create questions based on the cognitive skills you want to assess. A widely used model is Bloom’s taxonomy, which divides knowledge and cognitive skills into six hierarchical levels.

Knowledge based on remembering and understanding is at the lowest level. This includes the ability to recall and summarise information. Applying and analysing are placed in the middle of the knowledge hierarchy. Synthesising and evaluating are higher-level skills, while creating original material is at the top of Bloom’s Taxonomy.

Multiple-choice questions can assess skills at several levels. You can test students’ ability to memorise and recall course material, as well as their ability to understand and apply knowledge. Problem-solving skills can also be assessed with multiple-choice questions. For example, students can be presented with a text extract containing a clear problem and then answer questions about it.

At a higher cognitive level, multiple-choice questions can assess the ability to analyse an issue, compare information, and apply knowledge in new situations. Interactive question types, images, and video can help create a multiple-choice test that is challenging, demanding, and educational. Constructive feedback can also provide a good basis for formative tests that support learning.

The key is to formulate questions and design multiple-choice tasks in a way that assesses the skills you want to test.

How do you create effective multiple-choice questions?

KU Leuven (2022) provides guidelines and advice for creating multiple-choice questions: Formulating multiple choice questions - Education (kuleuven.be). These recommendations can be used as a checklist when developing multiple-choice tests, but they are not exhaustive. Below is a selection of tips for creating effective questions and answer options.

The question

• Write a clear question. The question should present a clear problem, with little room for misunderstanding. Students should not need the answer options to understand the question.
• Answer the multiple-choice question yourself, or test it on a colleague. This helps you check whether the previous point has been met. Make adjustments if you cannot answer the question yourself or if you hesitate between answer options.
• Questions should be independent of each other. Each question should address one issue and should not depend on the answers to previous questions.

The answer options

• Formulate the correct answer before creating the alternatives. Incorrect answer options are called distractors.
• You can find the best distractors by looking at the mistakes students most often make in the subject.
• Answer options should not overlap. Overlapping options may suggest that more than one answer is correct and can confuse students.
• Use a logical order. This makes it less likely that correct answers are repeatedly placed in the same position. Avoid recurring patterns in answer options, as these may indicate which answer is correct.
• Distractors must be plausible. Avoid answer options that are obviously correct or incorrect. This reduces the chance that students can guess the correct answer.

Read the full guide here: Formulating multiple choice questions - Education (kuleuven.be)

Choosing a platform

Several platforms can be used to create multiple-choice questions. For teachers, Inspera, Canvas, and Mentimeter are the most relevant platforms. The best choice depends on the purpose of the multiple-choice test.

Multiple-choice questions as an exam format: Inspera

Inspera is used when multiple-choice questions are part of an assessment or exam. Creating questions in Inspera can take time because each question must be entered manually. However, the first exam usually takes the most time to create, as the same questions can be reused in later years. For example, you can add 10 new questions and use the “random selection” function.

If exam candidates have either Nynorsk or Bokmål as their main written language, the multiple-choice test must be prepared in both written standards. This is required under the Regulations on language forms in examination papers. As this must be done manually, an exception may apply if the exam is considered particularly extensive.

UiO has prepared useful guides for importing and exporting questions in Inspera and Canvas: Importing and exporting questions

Multiple-choice tests as mandatory or optional activities: Canvas

Canvas can be used to create mandatory coursework requirements or optional multiple-choice tasks. As a teacher, you can create tests that students can use for independent learning. Canvas is more user-friendly and less time-consuming than Inspera.

To create a multiple-choice test, follow these steps:

1. Go to the course room in Canvas.
2. Find “Quizzes” in the left-hand menu.
3. Select “New Quizzes” to create a new quiz.
4. To create a multiple-choice question, select “Multiple Choice”.
5. Select “Save and Publish” when the test is complete. This makes it available to students.

CELL (Centre for Experiential Legal Learning) has prepared a user guide: How to create quizzes in Canvas. The guide provides simple instructions for creating, structuring, and publishing quizzes.

Multiple-choice questions for active learning in class: Mentimeter

Mentimeter is a good option if you want to engage students through more interactive teaching. The platform can be used during teaching sessions. Mentimeter has created an informative tutorial video: Create an interactive Mentimeter quiz

Video: Mentimeter

Considerations

There are some choices you may need to make. The final decision is yours, but it can be useful to understand the arguments for and against each option before deciding.

Total number of questions

The appropriate number of questions depends on the type of questions and how extensive they are. A larger number of questions may provide greater differentiation between grades than a small number. The time a student needs for each question depends on its level of difficulty. SVEXFAC estimated that students needed 1–1.5 minutes per question. With one hour available, they therefore used a total of 50 questions.

Number of answer options

We recommend using three answer options. More than two distractors may be unnecessary and difficult to make plausible. However, some use more than three options to reduce the likelihood of guessing.

Scoring: with or without negative marking?

There are differing views on negative marking in multiple-choice tests. One argument is that negative marking prevents guessing from being rewarded. Others believe that it may lead to unanswered questions, as some students may prefer to leave a question blank rather than risk losing points. Choosing not to use negative marking may help avoid students performing below their actual level. A middle option is to use low penalties, for example $-0.25$ points for an incorrect answer when a correct answer gives 1 point.

Accommodations for students

Multiple-choice exams can be challenging for students with reading difficulties or dyslexia. These students often need more time to read and understand questions and answer options. Norwegian higher education institutions are legally required to ensure universal design. This means that assessment formats and teaching must be adapted to students’ abilities and needs.

Ways to provide accommodations:

• Use short questions and few answer options per question. This is helpful because there is less text to read and understand.
• Provide extended exam time. Students must apply for exam accommodations.
• Use digital tools such as text-to-speech, which can read questions and answer options aloud.

Literature and sources

This guide is evidence-based. It draws on research literature, guides from other higher education institutions, and the experience of teaching and administrative staff at the University of Oslo. The sources used in this guide are listed below.

Research articles and other publications

Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. New York: McKay.

Haladyna, T. M. & Downing, Steven M. (1993). How many options is enough for a multiple-choice test item? (Validity Studies). Educational and Psychological Measurement, 53(4), 999–1009.

Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education, 15(3), 309–34.

Hecke, V. T. (2015). Initial correction versus negative marking in multiple choice examinations. Teaching Mathematics and its Applications: An International Journal of the IMA, 34(3), 171–178. https://doi.org/10.1093/teamat/hrv011

Holt, A. (2006). An analysis of negative marking in multiple-choice assessment. The School of Computing and Mathematical Sciences, The University of Waikato, Hamilton, New Zealand.

Lesage, E., Valcke, M. & Sabbe, E. (2013). Scoring methods for multiple-choice assessment in higher education – Is it still a matter of number right scoring or negative marking? Studies in Educational Evaluation, 39(3), 188–193.

Nwadinigwe, P. I. & Naibi, L. (2013). The Number of Options in a Multiple-Choice Test Item and the Psychometric Characteristics. Journal of Education and Practice, 28(4), 189–197.

O’Neill, D. L. (2021). Multiple choice questions in teaching. Dansk Universitetspædagogisk Tidsskrift, 16(30), 70–83.

Olsen, S. T. (2009). Multiple-choice questions and digital learning platforms: A qualitative study of preparation for the first part of the written examination in Biology 2. [Master’s thesis, Norwegian University of Science and Technology].

Sirnes, S. (2005). Multiple-choice questions: Construction and analysis. Bergen: Fagbokforlaget.

Woodford, K. & Bancroft, P. (n.d.). Multiple choice questions not considered harmful. School of Software Engineering and Data Communications, Queensland University of Technology.

User guides and guidance documents

CELL. (n.d.). User guide: How to create quizzes in Canvas. Retrieved 19 December 2022 from https://www.jus.uio.no/cell/ressurser/verktoy/brukermanual-krim1500.pdf

Horgen, S. A. (2017, 29 May). How to create effective quizzes in Canvas [Video]. YouTube. https://www.youtube.com/watch?v=kYSULmSqMGU

KU Leuven (2022). Formulating multiple choice questions. Retrieved from Formulating multiple choice questions - Education (kuleuven.be)

NTNU. (2014). Multiple-choice questions. Teaching guide from the Norwegian University of Science and Technology.

Olaussen, J. E. (16 August 2018). REAL teaching seminar. Retrieved 19 December 2022 from https://www.mn.uio.no/kurt/aktuelt/arrangementer/presentasjoner/2018/inkluderende-undervisning-real-uio-plenum-aug2018.pdf

University of Oslo. For staff. Importing and exporting questions. Retrieved 19 December 2022 from https://www.uio.no/for-ansatte/arbeidsstotte/sta/eksamen/digital-eksamen/veiledninger/inspera/importere-og-eksportere-oppgavesett.html#toc4

Rules and guidelines

Dyslexia Norway. (2021). Rights in higher education. Dysleksinorge.no. https://dysleksinorge.no/rettigheter-hoyere-utdanning/

University of Oslo. See the accommodations you have been granted. Retrieved 16 December 2022 from https://www.uio.no/studier/eksamen/tilrettelegging/se-hva-du-har-fatt-innvilget/

University of Oslo. (n.d.). Examinations. Uio.no. https://www.uio.no/studier/emner/sv/sv/SVEXFAC03/#exam

Experience-sharing discussions

Three experience-sharing discussions with University of Oslo staff were conducted in connection with this report.

• Eivind Balsvik, course coordinator for SVEXFAC: discussion of multiple-choice examinations in SVEXFAC, 22 September 2022.
• Eirik Finne, student adviser at IFIKK: discussion of using Inspera for multiple-choice questions, 15 November 2022.
• Francesca Jensenius (ISV): discussion of multiple-choice examinations in STV4021, 30 November 2022.