An evaluation of the success of a surgical resident learning portfolio

Background: Learning portfolios have gained modest acceptance in graduate medical education because of challenges related to user satisfaction, time and resource commitment, and quality assessment. In 2001, the Department of Surgery implemented the Surgical Learning and Instructional Portfolio (SLIP) to help residents develop a case-based portfolio demonstrating practice-based learning. In 2008, the format was changed to a Web-based platform with open viewing of portfolios for all learners. This study was performed to evaluate the SLIP program using resident and faculty perspectives in the domains of satisfaction, compliance, and educational value.

Methods: Likert scale surveys were distributed to residents to assess satisfaction. Using a semistructured format with subsequent qualitative analysis of the meeting transcript, a focus group discussion was held with the SLIP director, SLIP facilitator, and program coordinator. An analysis of the program compliance was performed by review of SLIP entry dates. Finally, the quality of the SLIP entries (n = 420) was analyzed in a blinded manner using a locally developed standardized SLIP assessment tool. Data analysis was performed using Pearson's correlation and Cronbach's alpha.

Results: Residents were satisfied with the program and felt the Web-based format promoted self-reflection. They perceived that time spent was appropriate. Residents also believed they gained medical knowledge of their own specific entry topics but did not learn routinely from others' entries. Faculty asserted that the Web-based platform eased the administrative burden but did not necessarily alter the quality of the SLIP entries. Compliance with the assignment was 100%. SLIP entry analysis demonstrated the reflection and understanding of the topics chosen. However, the overall quality assessment of entries was hindered by suboptimal interrater reliability (inter-rater reliability (IR) = 0.636).

Conclusions: The SLIP program allows residents to demonstrate practice-based learning and improvement of medical knowledge. The Web-based format provides transparency and ease of administration. Quality assessment of individual portfolio entries remains a challenge to the widespread adoption of portfolios.