Combining assessment scores : a variable feast

Background: In medical education grades are often determined by combining the scores of different assessments. While the psychometric literature recommends that the variance in each assessment be identical, there is limited discussion of the impact of different methods of combining scores. Method: Eight mock assessment results were created and used to develop an overall assessment that consisted of four types of assessments that were undertaken twice in a year. Various methods of combining scores were then used. The methods used two mechanisms for standardising results and a number of algorithms for combining scores. The impact of using the standard error of measurement was also investigated. Results: The number of 'students' failing ranged from five (simple addition of scores/8) to 30 (standardisation of scores, add scores/8, plus pass each type of assessment). Discussion: There is a large variation in the number of students failing associated with different methods of combining scores. Conclusion: Choosing a method to produce a combined score is a non-trivial exercise. The impact on student progression can be significant and whatever method is chosen will require justification.