A comparison of the psychometric properties of three- and four-option multiple-choice questions in nursing assessments

Summary 

In multiple-choice tests, four-option items are the standard in nursing education. There are few evidence-based reasons, however, for MCQs to have four or more options as studies have shown that three-option items perform equally as well and the additional options most often do not improve test reliability and validity. The aim of this study was to examine and compare the psychometric properties of four-option items with the same items rewritten as three-option items. Using item-analysis data to eliminate the distractor with the lowest response rate, we compared three- and four-option versions of 41 multiple-choice items administered to two student cohorts over two subsequent academic years. Removing the non-functioning distractor resulted in minimal changes in item difficulty and discrimination. Three-option items contained more functioning distractors despite having fewer distractors overall. Existing distractors became more discriminating when infrequently selected distractors were removed from items. Overall, three-option items perform equally as well as four-option items. Since three-option items require less time to develop and administer and additional options provide no psychometric advantage, teachers are encouraged to adopt three-option items as the standard on multiple-choice tests.

Keywords: Multiple-choice questions, Multiple-choice tests, Distractors, Item analysis, Test construction, Number of choices per item, Item discrimination, Assessment