The incidence of clueing in multiple choice testbank questions in accounting : some evidence from Australia

In 2014 Central Queensland University (CQU) in Australia banned the use of multiple choice questions (MCQs) as an assessment tool. One of the reasons given for this decision was that MCQs provide an opportunity for students to ‘pass’ by merely guessing their answers. The mathematical likelihood of a student passing by guessing alone can be removed, or at least minimised, by the way the test is administered. Consequently, the real risk associated with guessing rests in the quality of the questions asked; they must not give ‘clues’ that exacerbate any guessing risk. With the widespread use of multiple choice testing, having access to testbanks containing high-quality questions that don’t, in themselves, add to the risk of student guessing is important for time-poor academics who are increasingly likely to rely on them as a source of test items. This study analysed the questions used to assess the same topic from the multiple choice testbanks provided by the publishers of six commonly used Australian financial accounting textbooks. A significant majority of the questions (almost two-thirds) showed evidence of at least one ‘clueing’ flaw that could actually increase the chances of a student guessing the correct answer. The findings provide a degree of support for CQU’s decision and demonstrate some of the risks of adopting MCQ testbanks without adequate scrutiny prior to their use. Given the results of previous studies into the poor quality of MCQ testbanks, and the proliferation of guidelines for high-quality MCQ writing, it is both surprising and disappointing that such a large number of flaws are still being found in testbanks produced by what should be a well-informed academy. Clearly, care needs to be taken but multiple-choice tests that utilize ‘quality’ questions and that are appropriately administered can still be a reliable form of assessment. Perhaps CQU’s decision to ban them entirely is too extreme a position to take in the current climate in higher education.