Free
Research Article  |   April 1993
The Effect of Rater Severity On Person Ability Measure: A Rasch Model Analysis
Author Affiliations
  • Mary E. Lunz, PhD, is Director of Examination Activities and Psychometrician, Board of Registry of the American Society of Clinical Pathologists, 2100 West Harrison Street, Chicago, Illinois 60612
  • John A. Stahl, PhD, is Manager, Evaluation and Research, Board of Registry of the American Society of Clinical Pathologists, Chicago, Illinois
Article Information
Assessment Development and Testing / Practice
Research Article   |   April 1993
The Effect of Rater Severity On Person Ability Measure: A Rasch Model Analysis
American Journal of Occupational Therapy, April 1993, Vol. 47, 311-317. doi:10.5014/ajot.47.4.311
American Journal of Occupational Therapy, April 1993, Vol. 47, 311-317. doi:10.5014/ajot.47.4.311
Abstract

This paper presents a method for analyzing oral examinations with an extended, many-faceted Rasch model that calibrates medical specialty candidates, protocols, and raters. Significant variance was found among protocol difficulties and rater severities. When candidates’ raw scores were compared with calibrated measures corrected for the bias caused by the particular protocols and raters encountered, variation between candidate scores and measures were observed. The data were found to fit the Rasch model well enough to be suitable for making measurement on oral examinations more objective as well as providing specific feedback to oral examination raters. In this example a medical oral examination was used; however, the techniques are applicable to any situation in which trained professionals rate candiate or patient performances. For occupational therapists, potential applications include evaluation of a student’s fieldwork performance or observation of a patient’s task performance.