Assessment of psychology competencies in field placements : standardized vignettes reduce rater bias

Craig J. Gonsalvez, John Bushnell, Russell Blackman, Frank Deane, Vida Bliokas, Kathryn Nicholson-Perry, Alice Shires, Yasmina Nasstasia, Christopher Allan, Roslyn Knight

    Research output: Contribution to journalArticlepeer-review

    51 Citations (Scopus)

    Abstract

    Supervisors’ ratings of psychology trainees’ competence in field settings are a critical component of training assessment. There has been little systematic research regarding the validity of these assessments, but the available evidence suggests we have a problem! Supervisors’ judgments may be affected by systemic biases that pose a serious threat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method—the use of vignettes—against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer-review and revisions, before final vignettes were assigned calibration scores by a group of experts. A catalogue of vignettes (n = 41) that represent various domains of competence across several developmental stages was compiled. University and field supervisors used the conventional rating scale and the vignette-matching procedure (VMP) to evaluate competencies at end-placement. Data from a pilot (n = 20) and a follow-up study (n = 57) suggest that compared with a conventional rating scale, the VMP reduced leniency and halo biases. The VMP has the potential to improve outcomes of competency assessments in field placements and merits further research and development.
    Original languageEnglish
    Pages (from-to)99-111
    Number of pages13
    JournalTraining and Education in Professional Psychology
    Volume7
    Issue number2
    DOIs
    Publication statusPublished - 2013

    Keywords

    • competence
    • evaluation
    • field placements
    • internship programs
    • psychology students
    • rating of

    Fingerprint

    Dive into the research topics of 'Assessment of psychology competencies in field placements : standardized vignettes reduce rater bias'. Together they form a unique fingerprint.

    Cite this