You are here

  • Home
  • Archive
  • Volume 57, Issue 9
  • Evaluation of a postgraduate educational programme for occupational physicians on work rehabilitation guidelines for patients with low back pain

Article Text

Article menu

Download PDFPDF

Short report
Evaluation of a postgraduate educational programme for occupational physicians on work rehabilitation guidelines for patients with low back pain
  1. P B A Smitsa,b,
  2. J H A M Verbeekb,
  3. F J H van Dijkb,
  4. J C M Metzc,
  5. Th J ten Cated
  1. aCorvu School of Occupational Medicine Amsterdam, Academic Medical Center, PO Box 22700, 1100 DE Amsterdam, The Netherlands, bCoroneI Institute for Occupational and Environmental Health, cCatholic University of Nijmegen, The Netherlands, dUniversity of Utrecht, The Netherlands
  1. Dr P B A Smitsp.b.smits{at}amc.uva.nl

Abstract

OBJECTIVES The postgraduate educational programme for occupational physicians on guidelines for work rehabilitation of patients with low back pain was evaluated as to what extent did knowledge of the guidelines increase, and did the workers improve their performance at work.

METHODS An experimental group (n=25) attended an educational programme and a reference group did so (n=20) 6 months later. Knowledge and performance were assessed for both groups, before and after education of the experimental group. Knowledge was assessed for the reference group after education.

RESULTS Knowledge increased significantly more in the experimental group. The reference group's score increased further after education. The experimental group's adjusted gain score for performance indicators was significantly positive. Analysis of covariance also showed a significant effect for the experimental group for increased performance score.

CONCLUSIONS The educational programme improved the quality of care because knowledge and performance of occupational physicians improved and complied better with practice guidelines.

  • evaluation
  • postgraduate education
  • occupational physicians
  • low back pain
  • guidelines

https://doi.org/10.1136/oem.57.9.645

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Improvement of quality of care is regarded as a challenge for the medical profession,1 and the development of guidelines is seen as one way of approaching this. Guidelines aim to change the performance of doctors, yet it is obvious that they are also important for education. They provide a clear framework for how to act in practice. Performance indicators can be used to test whether medical performance complies with the guidelines, and thus provide good evaluation of the effects of an educational programme.2There is a need for evaluation of teaching programmes for occupational physicians, which hopefully will lead to more evidence based education.3 4 For this reason, a postgraduate educational programme for occupational physicians on guidelines for rehabilitation of patients with low back pain was evaluated. The research questions were whether the educational programme increased knowledge of the guidelines and whether the programme improved performance of the workers in practice.

    Methods

    The educational programme was evaluated by testing knowledge and performance indicators at the four levels of Miller's pyramid of clinical assessment.5 The content of the educational programme was based on Dutch guidelines for occupational physicians and general practitioners.6 7 The educational programme took a day and a half within a module of 6 days on work rehabilitation. An experimental group of 25 physicians participated in the educational programme. A reference group of 20 physicians did not, but took part in the programme 6 months later. The groups consisted of physicians of two consecutive year groups and were similar for possible confounders. We assessed the knowledge and performance of both groups by testing before and after education.

    Subjects took the paper and pencil knowledge test in the classroom. The tests consisted of 45 true or false questions. Cronbach's α for reliability of the tests was 0.65 before and 0.60 after the test.

    We used 12 performance indicators for the assessment of compliance with the guidelines on work rehabilitation.2 We asked the physicians to randomly collect, from their own practice, five patients with low back pain before the educational programme and five patients after. The physicians reported their performance for each case on specially designed forms. From these forms one researcher scored to what extent the physicians had followed the guidelines based on criteria which indicate the difference between good and bad performance.8 For every physician we calculated a mean percentage of the performance scores before and after the programme.

    Differences in scores before and after (gain scores) were tested with a paired samples t test or independent samplest test where appropriate. To evaluate the possible influence of differences in performance scores at baseline we used analysis of covariance. We calculated adjusted gain scores by subtracting the gain score of the reference group from the gain score of the experimental group.

    Results

    The knowledge test scores at time 1 were low (table). With true or false questions a score of about 80% is supposed to be significantly different from that which can be attained by chance alone. The gain scores were positive for both, the experimental and the reference group. The adjusted gain score was 9% which is significantly different from zero.

    For the performance indicators the gain score was positive for the experimental group only. The adjusted gain score was 15% and significantly different from zero. The results of an analysis of covariance showed a significant effect for the experimental group (F=25.3, p<0.0001) when performance scores before the test were entered as a covariate.

    Discussion

    In this study there was an increase in knowledge and an improvement of performance in practice after an educational programme on low back pain. These results could be biased in several ways. Firstly, we did not use a randomised study design and the groups might differ in other aspects than just the educational programme—such as learning style or location and time of testing. However, these differences would be more likely to bias the results in the direction of not finding a difference between the two groups. The increase in score on the knowledge test of the reference group is probably due to taking the test twice. The adjusted gain scores take this effect into account. Therefore we conclude that the increases in knowledge and performance can be ascribed to the educational programme.

    The validity of the knowledge test was sufficient for evaluation purposes with an internal consistency of 0.65 and 0.60 (Cronbach's α). The performance indicators were based on a physician's self report. However, the participants did not know the criteria we used to assess deviation from the guidelines. It was shown that performing well as measured by the same indicators predicted a better outcome and patient satisfaction.9 Therefore, we conclude that the performance scores do reflect performance in practice.

    We conclude that this study provides evidence for an increase in quality of care after occupational physicians participated in our postgraduate programme.

    References

    1. ↵
      1. Hulshof CTJ,
      2. Verbeek JHAM,
      3. Dijk FJH van,
      4. et al.
      (1998) Evaluation research in Occupational health services: general principles and a model for OHS evaluation. Hulshof CTJ. Prevention and control of adverse effects of whole-body vibration: an evaluation study in occupational health services [dissertation]. (University of Amsterdam, Amsterdam).
    2. ↵
      1. Weide WE van der,
      2. Verbeek JHAM,
      3. Dijk FJH van,
      4. et al.
      (1998) The development and evaluation of a quality assessment instrument for occupational physicians. Occup Environ Med 55:375–382.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Petersen S
      (1999) Time for evidence based medical education. Tomorrow's doctors need informed educators not amateur tutors. BMJ 318:1223–1224.
      OpenUrlFREE Full Text
    4. ↵
      1. Verbeek JHAM,
      2. Kroon PJ
      (1995) Education and training in occupational health and safety. Safety Science 20:iii–iiv.
      OpenUrl
    5. ↵
      1. Miller GE
      (1990) The assessment of clinical skills / competence / performance. Acad Med 65:S63–S67.
      OpenUrlPubMedWeb of Science
    6. ↵
      1. Verbeek JHAM,
      2. Weide WE van der
      (1997) Begeleiding van werknemers met lage rugklachten (Work rehabilitation of workers with low-back pain). Richtlijnen Arbo zorg. (Delwel, Den Haag) . (B11 3–2–1 t/m 34.).
    7. ↵
      1. Smits PBA,
      2. Verbeek JHAM
      (1995) Consultvoering door bedrijfsartsen, een actieve en probleemgerichte werkwijze (Consultation by occupational physicians: an active and problem-oriented working-method). Tijdschr Bedrijfs en Verzekeringsgeneesk 3:129–134.
    8. ↵
      1. AHCPR
      (1995) Using clinical practice guidelines to evaluate quality of care. Vol 2. Methods. (USDHHS, PHS, AHCPR, Rockville, USA) . (AHCPR pub 95–0046.).
    9. ↵
      1. Weide WE,
      2. Verbeek JHAM,
      3. Dijk FJH van
      (1999) Relation between indicators for quality of occupational rehabilitation of employees with low-back pain. Occup Environ Med 56:488–493.
      OpenUrlAbstract/FREE Full Text
    View this table:
    Table 1

    Scores on knowledge test (% correct answers) and performance indicators (% correct indicators) for the experimental and reference groups before and after a training programme for occupational physicians on rehabilitation of patients with low back pain