Reliability of single-item ratings of quality in higher education : a replication

Single-item ratings of the quality of instructors or subjects are widely used by higher education institutions, yet such ratings are commonly assumed to have inadequate psychometric properties. Recent research has demonstrated that reliability of such ratings can indeed be estimated, using either the correction for attenuation formula or factor analytic methods. This study replicates prior research on the reliability of single-item ratings of quality of instruction, using a different, more student-focussed approach to teaching and learning evaluation than used by previous researchers. Class average data from 1,097 classes, representing responses from 59,815 students, were analysed. At the "class" level of analysis, both methods of estimation suggested the single item of quality had high reliability: .96 using the correction for attenuation formula, and .94 using the factor analytic method. An alternative method of calculating reliability, which takes into account the hierarchical nature of the data, like-wise suggested high estimated reliability (.92) of the single-item rating. These results indicate the suitability of the overall class rating for quality improvement in higher education, with a large sample.