Exploring varied time intervals on diagnostic performances of radiologists and trainees via test sets

This study aims to investigate how the fluctuation of time intervals between self-assessment test sets influence the performance of radiologists and radiology trainees. The data was collected from 54 radiologists and 92 trainees who completed 260 and 550 readings of 9 mammogram test sets between 2019 and 2023. Readers’ performances were evaluated via case sensitivity, lesion sensitivity, specificity, ROC AUC and JAFROC. There was significant positive correlation between the intervals of test sets and radiologist's improvement in specificity and JAFROC (P<0.05). For separations in test sets exceeding 90 days, radiologists’ performance improved for sensitivity (5.2%), lesion sensitivity (6.6%), ROC (3.1%) and JAFROC (6.3%), with specificity remaining consistent. For trainees who completed test sets within a single day, a significant postive correlation was recorded between the time intervals of test sets and their improvement in ROC AUC (P=0.008) and JAFROC (P=0.02). However, for trainees who needed more than 1 day to complete a test set, this correlation was reversed in sensitivity (P=0.009) and ROC AUC (P=0.02). The most notable progress of trainees was found in sensitivity (6.15%), lesion sensitivity (11.6%), ROC AUC (3.5%) and JAFROC (4.35%) with specificity remained unchanged when the test sets were completed between 31-90 days.