Predicting Pouch of Douglas obliteration using ultrasound and laparoscopic video sets : an interobserver and diagnostic accuracy study

Objectives—To investigate the diagnostic accuracy and interobserver agreement among sonologists when assessing offline ultrasound (US) video sets of the "sliding sign" and among gynecologic surgeons when assessing corresponding laparoscopic video sets to predict pouch of Douglas (POD) obliteration and to compare the performance of the groups. Methods—A diagnostic and reproducibility study was conducted, including 15 observers in 4 groups: (1) senior sonologists, (2) junior sonologists, (3) general gynecologists, and (4) advanced laparoscopists. The sonologists viewed 25 offline preoperative US video sets of the sliding sign, and the surgeons viewed the corresponding intraoperative laparoscopic videos of the same patients. Each observer was asked to classify POD obliteration in the video sets and was compared to the reference standard POD state determined at real-time laparoscopy by a single investigator (G.C.). The interobserver correlation and diagnostic accuracy were evaluated among the 15 observers and 4 groups. The Cohen κ coefficient and Fleiss ê coefficient were used for the analysis. Results—The overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for senior sonologists were 93.3%, 100%, 89.6%, 84.4%, and 100%, respectively; for junior sonologists, 70.0%, 88.9%, 59.4%, 55.2%, and 90.5%; for general gynecologists, 75.2%, 88.1%, 78.1%, 69.8%, and 91.9%; and for advanced laparoscopists, 82.4%, 91.9%, 90.8%, 82.9%, and 95.8%. The overall agreement between senior sonologists was almost perfect (Fleiss κ = 0.876); for junior sonologists and general gynecologists, it was moderate (Fleiss κ = 0.589 and 0.528); and for advanced laparoscopists, it was substantial (Fleiss κ = 0.652). Conclusions—Interobserver agreement was superior among senior sonologists. Prediction of POD obliteration using offline US videos by senior sonologists is comparable to offline assessments of laparoscopic videos by advanced laparoscopists for prediction of POD obliteration.