Poster Session
Issue Date: August 2016
Published Online: August 01, 2016
Updated: January 01, 2021
Interrater Reliability of the Jarrold, Boucher and Smith Play Scenario
Author Affiliations
  • SPIRAL Foundation
Article Information
Autism/Autism Spectrum Disorder / Pediatric Evaluation and Intervention / Assessment/Measurement
Poster Session   |   August 01, 2016
Interrater Reliability of the Jarrold, Boucher and Smith Play Scenario
American Journal of Occupational Therapy, August 2016, Vol. 70, 7011500051.
American Journal of Occupational Therapy, August 2016, Vol. 70, 7011500051.

Date Presented 4/8/2016

This study examined the interrater reliability of trained raters for the Jarrold, Boucher, and Smith play scenario by viewing deidentified video of 5 children with autism spectrum disorder and 4 typical children, contributing evidence for the use of this measure in assessment of the childhood occupation of pretend play.

Primary Author and Speaker: Teresa A. May-Benton

Contributing Authors: Alison Teasdale, Michele Hung, Naomi Lin, Jennifer Quinn, Pranjali Savla, Dorothy Smith

QUESTION: Can trained raters achieve adequate interrater reliability on the Jarrold, Boucher, and Smith (1996) Play Scale?
RATIONALE: Current literature indicates that children with ASD have limited imaginative play skills (Doody & Mertz, 2013). Jarrold et al.’s (1996) study strengthened methodology utilized in prior studies to further develop a play scenario to examine whether children with autism spectrum disorder (ASD) can engage in pretend play under specific structured circumstances. However, no published scoring criteria was available for this play scenario. The current study developed a formal scoring rubric for this measure and examined its interrater reliability.
DESIGN: Quantitative, reliability study
PARTICIPANTS: Raters were 5 female graduate occupational therapy students ages 23–35 yr and videos of 9 male children ages 3–5 yr (4 typical and 5 with ASD).
MEASURE: The Jarrold et al. (1996) play scenario is a measure for pretend play that examines a child’s ability to demonstrate pretend play in structured and unstructured settings by observing children in spontaneous play, elicited play, and instructed play conditions. Within these play conditions, the play acts of pretend play, intermediate pretend play, functional play, manipulative play, and periods of no play were observed.
METHOD: Raters trained to administer and rate the Jarrold et al. (1996) play scenario were given a set of deidentified videos of children with ASD and typical children collected as part of a larger project examining praxis and play in children with ASD. The children’s diagnostic group and age were blind to raters until all scoring was completed.
After training, raters viewed and scored two sample videos collectively and individually to establish criteria for scoring, create a scoring rubric, and determine and resolve scoring discrepancies. A third video was viewed and scored individually to establish preliminary agreement between the raters with 100% agreement within a range of two scores for spontaneous and elicited play and 90% absolute agreement for instructed play. Raters then individually watched and scored nine videos.
ANALYSIS: Cronbach’s α was used to determine interrater reliability for the continuous data spontaneous and elicited play conditions, and percentage of agreement was used for the categorical instructed play conditions.
RESULTS: Cronbach’s αs were as follows: spontaneous play, α = .80; elicited play, α = .68; and spontaneous and elicited play combined, α = .84. Percentage of agreement for instructed play = 82%. Reliability of individual play acts ranged from .01 to .90.
DISCUSSION: Good to strong interrater reliability was found in spontaneous play, spontaneous and elicited play combined, and instructed play conditions. Thus, play conditions were able to be reliably rated with this scenario, and individual play acts were less reliable largely because of a lack of observations in some acts.
This study developed a training program and scoring rubric for raters. Areas for further exploration in the relationship between children with ASD and the capacity for pretend play in terms of identification of specific types of play acts were noted.
IMPACT STATEMENT: This research contributes to understanding of pretend play in children with ASD and typical peers and furthers use of this measure as a potentially reliable assessment of play. Better understanding of play in children with ASD may lead to development of interventions to promote participation in this most important childhood occupation.
Doody, K. R., & Mertz, J. (2013). Preferred play activities of children with autism spectrum disorder in naturalistic settings. North American Journal of Medical Science, 6, 128–133.
Jarrold, C., Boucher, J., & Smith, P. K. (1996). Generativity deficits in pretend play in autism. British Journal of Developmental Psychology, 14, 275–300.