TY - JOUR
T1 - Simulation-based clinical assessment identifies threshold competence to practise physiotherapy in Australia : a crossover trial
AU - Moss, P.
AU - Barnett-Harris, A.
AU - Lee, D.
AU - Gupta, K.
AU - Pritchard, S.
AU - Sievers, N.
AU - Te, M.
AU - Blackstock, Felicity
N1 - Publisher Copyright:
© 2022, The Author(s).
PY - 2022/12
Y1 - 2022/12
N2 - Background: Although evidence exists for the efficacy of high-fidelity simulation as an educational tool, there is limited evidence for its application in high-stakes professional threshold competency assessment. An alternative model of simulation-based assessment was developed by the Australian Physiotherapy Council (APC), using purpose-written standardised patients, mapped to the appropriate threshold level. The aim of this two-phase study was to investigate whether simulation-based clinical assessments resulted in equivalent outcomes to standard, real-life assessments for overseas-trained physiotherapists seeking registration to practice in Australia. Methods: A randomised crossover trial comparing simulation-based assessment to real-life assessment was completed. Participants were internationally trained physiotherapists applying for registration to practice in Australia, voluntarily recruited from the Australian Physiotherapy Council (APC) assessment waiting list: study 1 n = 25, study 2 n = 144. Study 1 participants completed usual APC real-life assessments in 3 practice areas, completed on different days at APC partner healthcare facilities. Participants also underwent 3 practice area-matched simulation-based assessments, completed on the same day at purpose-designed simulation facilities. Study 2 participants completed 3 simulation-based assessments and 1 real-life assessment that was randomly allocated for order and practice area. Assessment of competency followed the standard APC procedure of 90-minute examinations using The Moderated Assessment Form (MAF). Results: The overall pass rate was higher for real-life assessments in both studies: study 1, 50% versus 42.7%; study 2, 55.6% versus 44.4%. Chi-square analysis showed a high to moderate level of exact matching of pass/fail grades across all assessments: study 1, 73.4% (p < 0.001); study 2, 58.3% (p = 0.027). Binary logistic regression showed that the best predictors of real-life pass/fail grade were simulation-based MAF pass/fail grade (study 1, OR 7.86 p < 0.001; study 2, OR 2.037, p = 0.038) and simulation-based total MAF score (study 1, OR 1.464 p < 0.001; study 2, OR 1.234, p = 0.001). Conclusion: Simulation-based assessment is a significant predictor of clinical performance and can be used to successfully identify high stakes threshold competence to practice physiotherapy in Australia.
AB - Background: Although evidence exists for the efficacy of high-fidelity simulation as an educational tool, there is limited evidence for its application in high-stakes professional threshold competency assessment. An alternative model of simulation-based assessment was developed by the Australian Physiotherapy Council (APC), using purpose-written standardised patients, mapped to the appropriate threshold level. The aim of this two-phase study was to investigate whether simulation-based clinical assessments resulted in equivalent outcomes to standard, real-life assessments for overseas-trained physiotherapists seeking registration to practice in Australia. Methods: A randomised crossover trial comparing simulation-based assessment to real-life assessment was completed. Participants were internationally trained physiotherapists applying for registration to practice in Australia, voluntarily recruited from the Australian Physiotherapy Council (APC) assessment waiting list: study 1 n = 25, study 2 n = 144. Study 1 participants completed usual APC real-life assessments in 3 practice areas, completed on different days at APC partner healthcare facilities. Participants also underwent 3 practice area-matched simulation-based assessments, completed on the same day at purpose-designed simulation facilities. Study 2 participants completed 3 simulation-based assessments and 1 real-life assessment that was randomly allocated for order and practice area. Assessment of competency followed the standard APC procedure of 90-minute examinations using The Moderated Assessment Form (MAF). Results: The overall pass rate was higher for real-life assessments in both studies: study 1, 50% versus 42.7%; study 2, 55.6% versus 44.4%. Chi-square analysis showed a high to moderate level of exact matching of pass/fail grades across all assessments: study 1, 73.4% (p < 0.001); study 2, 58.3% (p = 0.027). Binary logistic regression showed that the best predictors of real-life pass/fail grade were simulation-based MAF pass/fail grade (study 1, OR 7.86 p < 0.001; study 2, OR 2.037, p = 0.038) and simulation-based total MAF score (study 1, OR 1.464 p < 0.001; study 2, OR 1.234, p = 0.001). Conclusion: Simulation-based assessment is a significant predictor of clinical performance and can be used to successfully identify high stakes threshold competence to practice physiotherapy in Australia.
UR - https://hdl.handle.net/1959.7/uws:73522
U2 - 10.1186/s41077-022-00215-2
DO - 10.1186/s41077-022-00215-2
M3 - Article
SN - 2059-0628
VL - 7
JO - Advances in Simulation
JF - Advances in Simulation
IS - 1
M1 - 21
ER -