Abstract
Objective
The objective structured examination is one of the most valid, reliable, and effective tools for assessing clinical and communication skills, often by use of standard patients (SPs). SPs can also be assessors of those skills. One of the crucial areas when utilizing SP-based assessment is the quality and consistency assurance of their portrayal of the case and their ability to fill in checklists in an adequate way. The aim of this study was to assess the validity and reliability of SPs’ ability to assess students’ communication skill via a Calgary-Cambridge checklist.
Method
This cross-sectional and correlational study was conducted at the Tehran University of Medical Science. We first analyzed validity; the criterion validity of the SPs’ filling in the checklists was assessed through determining the correlation between the SPs’ completed checklists and the checklists filled in by three physician raters individually and then reproducibility: it was assessed by a test-retest approach inter-rater reliability.
Result
The mean correlation for assessing the validity of SPs’ completed checklists by individual SPs was 0.81. The inter-rater reliability was calculated by kappa coefficient, and the total correlation among the three raters was 0.85. The reliability of the test-retest approach showed no significant differences between the test and re-test results.
Conclusion
The increased number of medical students and different faculties’ responsibilities such as doing educational, research, and health services duties assessing medical student communication skills is a complex issue. The results of our study showed that trained SPs can be used as a valid tool to assess medical students’ communication skills, which is also more cost effective and reduces work load of medical faculties.
References
Baig LA, Violato C, Crutcher RA. Assessing clinical communication skills in physicians: are the skills context specific or generalizable. BMC Med Educ. 2009;9:22.
Maguire P, Pitceathly C. Key communication skills and how to acquire them. Brit Med J. 2002;325:697–700.
Stubbe D. Effective patient centered—communication FOCUS. Winter. 2012;x(1):42–4.
World Health Organization. Doctor-patient interaction and communication. Geneva: WHO; 1993.
Vu NV, Barrows HS. Use of standardized patients in clinical assessments: recent developments and measurement findings. Educational Researcher 1994;23(3):23–30.
Adamo G. Simulated and standardized patients in OSCE achievement and challenges. Med Teach. 2003;25:262–70.
Hodges B et al. Evaluating communication skills in the objective structured clinical examination format: reliability and generalizability. Med Educ. 1996;30:38–43.
Wallace P. Coaching standardized patients: for use in the assessment of clinical competence. In: Wallace P, editor. New York: Springer Publishing Company; 2006.
Schlegel C et al. Validity evidence and reliability of a simulated patient feedback instrument. BMC Med Educ. 2012;12:1–6.
Thompson JC, Kosmorsky GS, Ellis BD. Field of dreamers and dreamed-up fields: functional and fake perimetry. Ophthalmology. 1996;103:11725.
Colliver JA, Swartz MH. Assessing clinical performance with standardized patients. JAMA. 1997;278:7901.
Norman GR et al. Simulated patients. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer Publishing Company; 1985.
Gomez JM, et al. Clinical skills assessment with standardized patients. Med Educ. 1997;31(2):94–8.
Ponzer S, Hylin U. Standardized patients—good help in teaching. They function as stand-ins in training of consultation and clinical skills. Lakartidningen. 2004;101(42):3240–4.
Vu NV. Steward DE, and Marcy M, An assessment of the consistency and accuracy of standardized patients’ simulations. J Med Educ. 1987;62:1002.
Colliver JA SM, Robbs RS, Lofquist M, Cohen D, Verhulst SJ. The effect of using multiple standardized patients on the intercase reliability of a large scale standardized patient examination administered over an extended testing period. Acad Med. 1998;73(10 suppl):S813.
Shirazi M et al. Training and validation of standardized patients for unannounced assessment of physicians’ management of depression. Acad Psychiatr. 2011;35(6):382–7.
Colliver JA. Validation of standardized-patient assessment: a meaning for clinical competence. Acad Med. 1995;70:1062–4.
Swartz MH, Colliver JA. Using standardized patients for assessing clinical performance: an overview. Mt Sinai J Med. 1996;63:241–9.
Chan CSY et al. Communication skill of general practitioners: any room for improvement? How much can it be improved? Med Educ. 2003;37:514–26.
Whelan R et al. Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG Clinical Skills Assessment (CSA®). Med Teach. 2005;27:200–6.
Donnon T, Lee M, Cairncross S. Using item analysis to assess objectively the quality of the Calgary-Cambridge OSCE Checklist. CMEJ, 2011;2(1):e16–e22.
Keen AJA, Klein S, Alexander DA. Assessing the communication skills of doctors in training: reliability and sources of error. Adv Health Sci Educ. 2003;8:5–16.
Zanten VM, Boulet JR, McKinley D. Using standardized patients to assess the interpersonal skills of physicians: six years’ experience with a high-stakes certification examination. Health Commun. 2007;22(3):195–205.
Wasnick JD et al. Do residency applicants know what the ACGME core competencies are? One program’s experience. Acad Med. 2010;85(5):791–3.
Silverman J. Teaching clinical communication: a mainstream activity or just a minority sport? Patient Educ Couns. 2009;76(3):361–7.
Hausberg MC et al. Enhancing medical students’ communication skills: development and evaluation of an undergraduate training program. BMC Med Educ. 2012;12(1):16.
Boulet JR. Using SP-based assessments to make valid inferences concerning student abilities: pitfalls and challenges. Med Teach. 2012;34(9):681–2.
Brinkman WB et al. Evaluation of resident communication skills and professionalism: a matter of perspective? Pediatrics. 2006;118(4):1371–9.
Boulet JR et al. Checklist content on a standardized patient assessment: an ex post facto review. Adv Health Sci Educ. 2008;13:59–69.
Fincher R-ME, Lewis LA. Simulations used to teach clinical skills. In: Norman GR, van der Vleuten CPM, Newble DI, editors. Int Handb Res Med Educ. Dordrecht, The Netherlands: Kluwer Academic Publishers; 2002. p. 499–535.
Parkes J, Sinclair N, McCarty T. Appropriate expertise and training for standardized patient assessment examiners. Acad Psychiatr. 2009;33(4):285–8.
Rothman AI, Cusimano M. A comparison of physician examiners’, standardized patients’, and communication experts’ ratings of international medical graduates’ English proficiency. Acad Med. 2000;75(12):1206–11.
Gutton G et al. Communication skills in standardized-patient assessment of final-year medical students: a psychometric study. Adv Health Sci Educ. 2004;9:179–87.
Disclosure
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shirazi, M., Labaf, A., Monjazebi, F. et al. Assessing Medical Students’ Communication Skills by the Use of Standardized Patients: Emphasizing Standardized Patients’ Quality Assurance. Acad Psychiatry 38, 354–360 (2014). https://doi.org/10.1007/s40596-014-0066-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40596-014-0066-2