Free
Research Article  |   November 2012
Assessing the Sensitivity of the Evaluation of Social Interaction: Comparing Social Skills in Children With and Without Disabilities
Author Affiliations
  • Lou Ann Griswold, PhD, OTR/L, FAOTA, is Associate Professor, Department of Occupational Therapy, University of New Hampshire, Hewitt Hall, Durham, NH 03824; louann.griswold@unh.edu
  • Samantha Townsend, MS, OTR/L, is Occupational Therapist, Fairfax County Public Schools, Fairfax County, VA. At the time of the study, she was an undergraduate student, Department of Occupational Therapy, University of New Hampshire, Durham
Article Information
Rehabilitation, Participation, and Disability / Children and Youth
Research Article   |   November 2012
Assessing the Sensitivity of the Evaluation of Social Interaction: Comparing Social Skills in Children With and Without Disabilities
American Journal of Occupational Therapy, November/December 2012, Vol. 66, 709-717. doi:10.5014/ajot.2012.004051
American Journal of Occupational Therapy, November/December 2012, Vol. 66, 709-717. doi:10.5014/ajot.2012.004051
Abstract

OBJECTIVE. We examined the sensitivity of the Evaluation of Social Interaction (ESI) as a measure of the overall quality of social interaction in children as they engage in social exchanges in a natural context with typical social partners.

METHOD. We compared the ESI measures of 23 children with disabilities with those of 23 children without disabilities, matched by age and gender.

RESULTS. Paired t-test analysis revealed a statistically significant difference, t(22) = −4.065, p = .001, in the quality of social interaction for the two groups of children, indicating sensitivity of the ESI.

CONCLUSION. The results support the use of the ESI to measure quality of social interaction in a natural context.

Social skills are an essential component of children’s occupations—the meaningful activities in which they engage. Social skills enable children to interact with others for a variety of purposes and enhance performance at home, in school, on the playground, and in many other settings. Elliott, Sheridan, and Gresham (1989)  reported that children who have deficits in social interaction skills experience immediate and long-lasting negative consequences. More specifically, Hartup (1988)  emphasized the importance of social skills and noted that “the single best childhood predictor of adult adaptation is not school grades, and not classroom behavior, but rather, the adequacy with which the child gets along with other children” (p. 1). Wight and Chapparo (2008)  emphasized the relevance of addressing social interaction to support children’s full participation in learning activities in school and urged occupational therapists to provide intervention to address social interaction. Lim, Rodger, and Brown’s (2010)  study of the perspective of teachers corroborated the importance of social interaction skills in children to support both learning and relationships with others.
Difficulty with social interaction skills is not diagnosis or condition specific. Challenges with social interaction have been documented for children with learning disabilities (Kavale & Forness, 1996), language impairment (McCabe & Marshall, 2006), anxiety disorders (Strauss, Lease, Kazdin, Dulcan, & Last, 1989), and autism (American Psychiatric Association, 1994; Beresford, Tozer, Rabiee, & Sloper, 2007).
Social Interaction as an Area of Concern for Occupational Therapy
Occupational therapy practitioners have been concerned with social interaction for many years. Kielhofner (1985)  included communication and social interaction in the performance subsystem of the Model of Human Occupation. Fisher and Kielhofner (1995)  operationally defined communication and interaction skills in Kielhofner’s 1995  version of the Model of Human Occupation. Doble, Bonnell, and Magill-Evans (1991)  reported that occupational therapists working in a variety of settings in Canada stated that they supported their clients’ needs for social interaction but lacked an evaluation tool with which to plan intervention or measure progress. Doble et al. (1991)  argued that relying on observations that do not offer quantifiable results provides incomplete and potentially biased information. Doble and Magill-Evans (1992)  introduced a conceptual framework for occupational therapists to use when considering social interaction performance during the natural contexts of work, play, and self-care tasks. They suggested specific observable social interaction skills that could be used in an assessment.
Approaches to Assessing Social Interaction
Social interaction is typically assessed using one of two primary methods: (1) report by others, including parents, teachers, or peers, or (2) observation of a child’s interactions. Sheridan, Hungelmann, and Maughan (1999)  conceptualized these two methods of learning about a child’s interaction as very different from one another. They explained that reports by others indicate a child’s social competence, reflecting their judgment of the child’s behavior, whereas observation of a child during interactions enables the evaluator to identify specific behaviors in social interaction that may lead to competence.
Assessment Based on Report by Others
A number of assessments of social interaction involve reporting by parents or teachers. One criticism of this approach is that the evaluator’s perspective may vary across environments. When parents or teachers complete a questionnaire on a child’s social interaction, their reports are applicable only to the specific settings and situations in which the observation occurred, such as home, school, or playground (McConnell & Odom, 1999; Wight & Chapparo, 2008). McCabe and Marshall (2006)  found that teacher and parent ratings of social interaction in children who had a diagnosis of a specific language impairment differed from one another on the basis of whether the children’s social interaction was assessed in the school or home environment. Expectations, as well as the peers with whom children interact, may differ in the two environments. McCabe and Marshall also questioned whether teachers and parents clearly understood the terms used in the rating scale, raising concerns about raters’ understanding of the concepts being evaluated.
Peer acceptance ratings are another observation-based indicator of a child’s social competence. According to Elliott et al. (1989), peer acceptance ratings reflect peers’ acceptance or rejection of the child rather than the child’s social interaction skills. McConnell and Odom (1999)  claimed that peer acceptance ratings capture the social impact of a child’s social performance within a given setting but are limited to specific situations. Elliott et al. further pointed out that peer acceptance ratings do not identify the specific behaviors on which the peer judgment is based.
Assessment Based on Observed Interaction
Observing a child during interaction with others is the second approach to assessing social interaction (Sheridan et al., 1999). Sheridan et al. (1999)  described social skills as discrete, learned actions that children do when interacting with another person. Michelson, Sugai, Wood, and Kazdin (1983)  noted that social skills include specific and discrete verbal and nonverbal behaviors that can be observed, yet they emphasized that social skills are complex and occur within a context that includes both the environment in which the interaction takes place and the activity in which the person and social partners are engaging. Michelson et al. explained that social skills are interactive; that is, a person must respond to a social partner. Recognizing the complexity of social skills and the influence of context, particularly the social partner and environment, Elliott et al. (1989)  recommended assessing social skills by observing a child interacting with others with whom the child typically interacts in a natural environment.
Because the social interaction skills that support an interaction are complex, evaluators need to rate multiple behaviors. Doble and Magill-Evans (1992)  proposed evaluating social interaction behaviors that include skills related to acknowledging a social partner, sending messages to a social partner, and timing and coordinating messages. Many social interaction studies, however, have measured a limited number of behaviors, such as eye gaze, without considering the effects of the context of social interaction and the influence of the environment or social partner (Cruice, Worrall, & Hickson, 2005; Turkstra, 2005). Other measures of social interaction are limited in focus—for example, counting the number of utterances or number of words used during a given interaction (Girolametto, Hoaken, Weitzman, & van Leishout, 2000). Although these behaviors contribute to social interaction, they do not represent the range of skills that indicate overall quality of social interaction. Mere counting of certain behaviors does not take the context and actions of the social partner into consideration.
Occupational Nature of Social Interaction
Occupational therapists frequently include observation of a person in their evaluation. Assessments based on observation may be either contrived to elicit certain behaviors or conducted in a natural context (Griswold, 2010). Doble and Magill-Evans (1992)  advocated for evaluating a person’s social interaction in the natural context of work, play, or leisure to gather information to later provide intervention to support participation in these contexts. More recently, others (Fisher, 2009; Law, 2002) have emphasized the importance of assessing a person’s performance in a natural context. In a 2002 meeting at the National Institutes of Health focused on research for people with autism, participating researchers advocated for assessment tools that would measure the effectiveness of programs in the real world rather than in a clinical setting (Lord et al., 2005). Wight and Chapparo (2008)  argued that planning intervention that is relevant to children’s social skill needs requires that assessment occur in the natural context.
Lim and Rodger (2008)  identified elements related to context and outlined an ideal evaluation of social interaction for children. First, the interaction should be observed during an activity that is relevant for the child and in which the child is motivated to engage. Second, the interaction should occur within a natural context, such as during play activities or schoolwork. Finally, the interaction should occur with social partners with whom the child would typically interact—for example, classmates, siblings, or adults with whom they have frequent contact.
Evaluation of Social Interaction
We chose the Evaluation of Social Interaction (ESI; Fisher & Griswold, 2010) for this study because it matches the criteria indicated in the literature as important in assessing quality of social interaction. The evaluator, an occupational therapist, observes the person as he or she engages in desired, relevant activities in a natural context with typical social partners, elements that Lim and Rodger (2008)  emphasized. The ESI is used to rate both verbal and nonverbal behaviors that support social interaction. It includes 27 skills that relate to initiating and ending a social interaction, producing the interaction, physically supporting the interaction, shaping the content and maintaining the flow of the interaction, verbally supporting the interaction, and adapting to problems that might arise during the interaction. The range of behaviors observed and number of skills rated using the ESI reflect the complexity of social interaction noted by Doble and Magill-Evans (1992) . The criterion used to rate the person’s performance of each skill is written in a way that reflects the context of the social interaction and the need for certain behaviors as indicated by the social partner during the natural flow of the interaction, identified as important by Fisher (2009), Lord et al. (2005), Michelson et al. (1983), and Wight and Chapparo (2008) .
After observing as the child engages in two or more social exchanges in a natural context, the occupational therapist rates each of the 27 skills using a 4-point rating scale indicating level of competence in performance. The occupational therapist enters the raw scores for each skill for all observations into a computer software program developed using many-faceted Rasch analysis. The ESI software generates a measure of the quality of social interaction that takes into account the hierarchies of difficulty of the social interaction skills, the intended purpose of the social interaction, and the severity of the rater (Fisher & Griswold, 2010). In other words, the skill ratings are not summed or totaled but rather are weighted according to the level of relative difficulty of each skill. Likewise, the relative challenge of the intended purpose of the social interaction provides a way to consider nearly any interaction observed within a natural context. Coster (2008)  advocated for assessments that use Rasch analysis to enhance one’s understanding of complex constructs such as social interaction.
Although the ESI meets the desired qualities for an assessment of social interaction, the ESI’s validity, reliability, and sensitivity need to be examined as part of the standardization of this tool. Using data from an early sample of 128 people aged 4–73 yr, Simmons, Griswold, and Berg (2010)  reported excellent internal scale validity, with 95% of the observations fitting the Rasch model. Moreover, they reported person separation reliability of .89 and item separation reliability of .98 (Simmons et al., 2010, p. 14). Fisher and Griswold (2010)  reported similar findings using a larger standardization sample of 1,140 people.
Sensitivity of the ESI has been evaluated only in studies on adults. Simmons et al. (2010) reported that ESI measures for adults with a disability were significantly different from those for adults without a disability. Søndergaard (as cited in Fisher & Griswold, 2010) found that 32 people with a neurological condition and 32 people with a psychiatric disorder had significantly lower ESI measures compared with a group of 32 people without a diagnosis or condition. Following Søndergaard’s study, Fisher and Griswold (2010) used the data in the ESI database for people aged 16–64 yr and found significant differences among three groups: people with a psychiatric disorder, people with a neurological condition, and people without a diagnosis or condition.
Because the ESI is a relatively new assessment for occupational therapists and addresses an important need for many children whom occupational therapists assess and provide services for, determination of the tool’s sensitivity would be beneficial to support its clinical application with children. The focus of our research project was to evaluate the sensitivity of the ESI for children. The research question was, Is the ESI able to differentiate quality of social interaction of children who have a disability from that of those who do not?
Method
Research Design
To address the research question, we used a quasi-experimental design to compare the quality of social interaction as measured by the ESI for two groups of children: (1) those with known disabilities and (2) same-age and -gender peers without a disability. The institutional review board of the University of New Hampshire approved this study.
Participants
We recruited participants by sending a letter to parents of children who attended preschool, kindergarten, readiness, first-grade, and second-grade classrooms at a local elementary school in northern New England; the letter was sent home with the children. In the letter, we asked parents if their child could be included in this study. Additionally, we asked parents of children whom author Townsend knew through personal contact for permission to include their children in the research. A total of 119 permission slips were received (89 from the school and 30 from the personal contacts). All children for whom a parent provided consent were eligible for inclusion in the study.
Townsend identified 25 children with a disability as reported by parents on the consent form. She selected children to include in the study to obtain a sample reflecting a variety of disabilities and range of ages. She then identified children without disabilities for whom she had consent who were of the same age and gender as those with disabilities. When she had more than 1 child from whom to choose, she selected a child on the basis of convenience of observing (e.g., she was already observing other children in that child’s classroom).
Procedure
We observed the children in the natural context of school, child care, or play areas (e.g., park or beach) as they engaged in activities of schoolwork, play, or instrumental activities of daily living with their typical social partners. Social partners included peers, siblings, parents, and teachers. We did not ask the children to do any particular task but rather observed them as they engaged in regular social interactions. We chose the interactions to observe on the basis of what the child’s teacher or parent indicated was relevant and challenging for the child—for example, working on a group project with peers. We observed the children without disabilities during relevant social interactions on the basis of what the child had chosen to do—for example, playing a game or talking with friends. The ESI categorizes social interactions by their intended purpose; the categories are gathering information, sharing information, problem solving or decision making, collaborating or producing, acquiring goods and services, and conversing socially or making small talk (Fisher & Griswold, 2010). We determined the intended purpose of the observed interactions using the criteria in the ESI manual; specific examples of children’s interactions and their intended purpose are in Table 1. We observed each child twice, with the two observations possibly differing in social partners, environment, or intended purpose of the social interaction.
Table 1.
Intended Purpose of Observed Social Interactions and Selected Specific Examples
Intended Purpose of Observed Social Interactions and Selected Specific Examples×
PurposeExamples
Gathering information (13%)
  • Getting instructions from a teacher
  • Receiving assistance from a teacher
  • Learning how to operate a kiln
  • Finding out what items are needed at the grocery store
Sharing information (8%)
  • Teaching a classmate how to complete a project
  • Helping a sibling read an invitation
  • Telling a teacher about a toy
  • Telling a friend about the playground equipment
  • Tattling on a sibling
  • Showing a sibling a photo album
Problem solving or decision making (4%)
  • Working through a disagreement with peers
  • Planning the rules for a hide-and-seek game with friends
  • Planning where to set up the boundaries for a chase game with friends
Collaborating or producing (51%)
  • Working on a group project with classmates
  • Working with a partner on an assignment
  • Working with classmates to organize the books on a shelf
  • Playing a game in the pool with friends
  • Playing a board game with a sibling
  • Building a sand castle with a friend
  • Building with Legos™ with a sibling
  • Playing a computer game with a friend
  • Doing a puzzle with a friend
  • Collecting rocks with a friend
  • Playing a fishing game with the class
  • Catching fish and bugs in a pond with a friend
Acquiring goods and services (2%)Getting an ice cream sundae
Conversing socially or making small talk (22%)
  • Talking with peers while eating a snack or meal
  • Talking while on the swings
Table 1.
Intended Purpose of Observed Social Interactions and Selected Specific Examples
Intended Purpose of Observed Social Interactions and Selected Specific Examples×
PurposeExamples
Gathering information (13%)
  • Getting instructions from a teacher
  • Receiving assistance from a teacher
  • Learning how to operate a kiln
  • Finding out what items are needed at the grocery store
Sharing information (8%)
  • Teaching a classmate how to complete a project
  • Helping a sibling read an invitation
  • Telling a teacher about a toy
  • Telling a friend about the playground equipment
  • Tattling on a sibling
  • Showing a sibling a photo album
Problem solving or decision making (4%)
  • Working through a disagreement with peers
  • Planning the rules for a hide-and-seek game with friends
  • Planning where to set up the boundaries for a chase game with friends
Collaborating or producing (51%)
  • Working on a group project with classmates
  • Working with a partner on an assignment
  • Working with classmates to organize the books on a shelf
  • Playing a game in the pool with friends
  • Playing a board game with a sibling
  • Building a sand castle with a friend
  • Building with Legos™ with a sibling
  • Playing a computer game with a friend
  • Doing a puzzle with a friend
  • Collecting rocks with a friend
  • Playing a fishing game with the class
  • Catching fish and bugs in a pond with a friend
Acquiring goods and services (2%)Getting an ice cream sundae
Conversing socially or making small talk (22%)
  • Talking with peers while eating a snack or meal
  • Talking while on the swings
×
We scored the children’s social interaction performance using ESI criteria specified in the manual for each of the 27 skills (Fisher & Griswold, 2010). The ESI uses the following criteria for competence:
  • A score of 4 indicates competent or mature interactions that are respectful and well timed and supportive of the interaction.

  • A score of 3 indicates questionable skill performance; the skills might be considered undesirable, but no clear problem is noted.

  • A score of 2 indicates that the person demonstrated ineffective or immature social interaction skills; the interaction was somewhat impolite, improper, irrelevant, or disruptive to the overall intended purpose.

  • A score of 1 indicates that the person had very immature or markedly inappropriate social interaction skill performance indicating that the performed skill was unacceptable, the intended purpose was not supported by the skill, or the person needed assistance from another person to continue in the interaction.

Data Analysis
We submitted all data to the AMPS Project International for analysis using many-faceted Rasch (MFR) analysis. The data for the children in this study were compared with other data in the Rasch model at the time of analysis to ensure reliable scoring and internal validity. Rasch analysis converts the ordinal ESI skill item raw scores into linear performance measures, referred to as logits (Fisher & Griswold, 2010) or log-odd probability units (Bond & Fox, 2007). The MFR analysis calculated a social interaction measure on the basis of the calibrated challenge of the intended purpose, the relative difficulty of each of the 27 ESI skill items, and the severity of the rater (Fisher & Griswold, 2010). The Rasch measure indicates the child’s quality of social interaction performance. Higher performance measures reflect children with higher abilities, and lower performance measures reflect children with lower abilities (Fisher & Griswold, 2010).
Fisher and Griswold (2010)  identified 1.0 logit as a cutoff to differentiate competent social interaction from less than competent social interaction (p. 8-7). That is, people whose ESI measure is ≥1.0 logit have socially competent, mature social interaction, and those whose measure is <1.0 logit have less than competent performance. Lower measures on the ESI scale indicate poorer quality of social interaction skill. Fisher and Griswold (2010)  reported a significant positive curvilinear relationship between ESI measure and age for 703 people who were typically developing (p. 3-6).
We compared the ESI measures for children in each group, those with and without a disability, using a paired t test and set the level of significance at p < .05.
Results
The data analysis indicated that one of the participants with a disability had a social interaction measure that was unusually high, indicating invalid results, likely attributable to error in task selection. Therefore, we removed this child and her matched partner from the data. During data analysis, we also discovered that we were missing a same-age matched peer for a child with a disability. Thus, data from 23 pairs of children, a total of 46 children, were included in the study results.
Of the 46 children included in the study, 34 were boys and 12 were girls. The higher number of boys identified with a disability reflects the difference in incidence by gender for many childhood conditions (Rogers, 2010). The children ranged in age from 2 to 12 yr, with a mean age of 7 yr. We categorized the children’s primary diagnoses using the diagnostic groups used for the standardization sample (Fisher & Griswold, 2010). Nine of the 23 children with disabilities had a mild disability, including learning disability, attention deficit disorder, sensory processing disorder, or speech and language disability. Seven of the children had a developmental disability, including Down syndrome, and seven of the children had a psychiatric disorder, including autism spectrum disorder, anxiety disorder, and behavioral disorder (Table 2).
Table 2.
Age, Gender, and Disability Group of Participants
Age, Gender, and Disability Group of Participants×
Number in Each Disability Group
Age, yrBoysGirlsTotalMildDevelopmentalPsychiatric
3213210
4000000
5303300
633003
7123102
8516042
9112110
10011100
11101100
12101010
Total17623977
Table 2.
Age, Gender, and Disability Group of Participants
Age, Gender, and Disability Group of Participants×
Number in Each Disability Group
Age, yrBoysGirlsTotalMildDevelopmentalPsychiatric
3213210
4000000
5303300
633003
7123102
8516042
9112110
10011100
11101100
12101010
Total17623977
×
The mean ESI measure for the children with a disability was −.4435 logit; for children without a disability, it was .0557 logit. The paired t test revealed a significant difference between children with a disability and children without a disability, t(22) = −4.065, p = .001. Paired t-test statistics are presented in Table 3. The results indicate a significant difference in overall quality of social interaction between children with and without disability matched for age and gender, demonstrating the ESI’s ability to differentiate between children with and without a disability on the basis of quality of social interaction.
Table 3.
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability×
GroupMean LogitSDMean DifferenceSDtdfp
Children with disability−.4435.62111
Children without disability.0557.59689−.49913.58890−4.06522.001
Table Footer NoteNote. ESI = Evaluation of Social Interaction; SD = standard deviation.
Note. ESI = Evaluation of Social Interaction; SD = standard deviation.×
Table 3.
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability×
GroupMean LogitSDMean DifferenceSDtdfp
Children with disability−.4435.62111
Children without disability.0557.59689−.49913.58890−4.06522.001
Table Footer NoteNote. ESI = Evaluation of Social Interaction; SD = standard deviation.
Note. ESI = Evaluation of Social Interaction; SD = standard deviation.×
×
Figure 1 shows the ESI measures by age. Most of the children with disabilities had ESI measures lower than their same-age peer without a disability; the exception was the 3-yr-old pairs. Figure 1 indicates a general trend of increasing quality of social interaction with age for children without a disability, reflecting the change in quality of social interaction with age reported by Fisher and Griswold (2010) . Likewise, this sample of children with a disability also showed a general trend of improved quality of social interaction with age, although at a lower level of quality than their peers without a diagnosis.
Figure 1.
Children’s Evaluation of Social Interaction (ESI) measures, by age.
Figure 1.
Children’s Evaluation of Social Interaction (ESI) measures, by age.
×
Discussion
The results of this study suggest that the ESI is able to differentiate between children with and without a disability on the basis of quality of social interaction, as observed in two social exchanges in the natural contexts of school, play, and activities of daily living when interacting with usual social partners. The only exception was the 3-yr-olds, perhaps because 3-yr-olds are still very young and may have greater variation in quality of social interaction in the early stage of developing such skills. This hypothesis is supported by the spread of ESI measures for the three 3-yr-olds (see Figure 1); however, the number of 3-yr-olds is too low to enable us to reach any firm conclusions. The sensitivity of the ESI in differentiating children with and without disability supports similar research with adults by Simmons et al. (2010) . In summary, our findings contribute to existing support for the validity of the ESI as a measure of the quality of social interaction.
Results from this study indicate that occupational therapists can use the ESI to obtain an objective measure of a child’s quality of social interaction using a standardized, criterion-referenced assessment. As an observational assessment in natural contexts, the ESI captures the quality of a person’s social interaction during occupations in which he or she needs or wants to participate. The criterion used to rate the person’s performance of each skill enables the occupational therapist to consider the contextual expectations of the social interaction and the need for certain behaviors indicated by the social partner during the natural flow of the interaction. Thus, the ESI provides a measure of social interaction that acknowledges the complexity of behaviors making up social interaction as identified by Doble and Magill-Evans (1992), Fisher (2009), Lord et al. (2005), Michelson et al. (1983), and Wight and Chapparo (2008) .
As a criterion-referenced assessment, the ESI may enable occupational therapists to better plan intervention to focus on social interaction skills that are challenging. Because observations are conducted in natural contexts, use of the ESI provides occupational therapists with a better understanding of the challenges in social interaction for which a child needs support. Because the tool is not tied to any one area of occupation, the ESI can be used in any social context, enabling occupational therapy practitioners to evaluate and then address the interactions during occupations that are of highest priority for the child, family, and teacher. As a sensitive measure of the quality of social interaction, the ESI might be used to determine the effectiveness of different approaches used in intervention.
Implications for Occupational Therapy Practice
The results of this study support occupational therapy’s role in evaluating the quality of social interaction in children during social interactions occurring in natural context. Specifically, this study indicates that the ESI
  • Provides a valid assessment tool to assess the quality of social interaction in children to establish baseline and to measure change in their social interaction performance and

  • Provides a tool to enable occupational therapy practitioners to plan intervention to address specific social interaction skill deficits for children during desired activities in natural contexts.

Limitations and Future Research
Limitations of this study include the sample’s small size and lack of demographic diversity. The sample was taken from one school and from personal contacts, potentially leading to selection bias. Future research with a larger sample reflecting greater diversity and random selection would further support the sensitivity of the ESI. Additionally, the children who had a diagnosis or disability in this study had a range of disabilities and diagnoses from mild to significant. A larger sample size in a follow-up study would allow for greater examination of the ESI’s ability to differentiate children who are experiencing varying levels of disabilities. Examination of the ESI’s ability to differentiate children who have more subtle difficulties in social interaction from typically developing peers would further support its sensitivity. For example, children who are at risk for a disability because of environmental circumstances might demonstrate early signs of poorer quality of social interaction than their same-age peers.
Because our two observations of each child often took place in different natural contexts or with different social partners, we questioned whether and how the environment, activity context, and social partners may have influenced the children’s quality of social interaction. Hence, several new research questions emerged regarding the influence of these factors. Future research could examine the influence of characteristics of the social partner on a child’s quality of social interaction. Specifically, researchers could examine the influence of age of the social partner (adult vs. child), the status of or relationship to the social partner (e.g., parent, teacher, sibling, friend), the social partner’s quality of social interaction, and the influence of the structure of the activity in which the social exchange occurs (e.g., schoolwork activities vs. play activities). The results of such studies could provide further guidance for intervention to support improved social interaction performance.
Conclusion
The ESI addresses the need for an observation-based assessment of the quality of social interaction for children as they engage in the occupations of play, education, and activities of daily living with typical social partners. By observing children during natural social exchanges that are important and relevant, occupa-tional therapists can identify strengths and challenges in social interaction behaviors reflecting the quality of performance during desired occupations. Such assessment results can more clearly guide intervention to support social interaction, a skill area that is essential throughout life. Simmons et al. (2010)  noted that as occupational therapy practitioners increase their intervention to more deliberately include social interaction, they need assessment tools to guide intervention planning and measure progress. This study supports the sensitivity of the ESI with children, complementing Simmons et al.’s previous findings with adults.
Acknowledgments
The authors appreciate the support of Brett Berg at the AMPS Project International for analyzing the study data. We thank the Hamel Center for Undergraduate Research at the University of New Hampshire for funding this research project.
References
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.×
Beresford, B., Tozer, R., Rabiee, P., & Sloper, P. (2007). Desired outcomes for children and adolescents with autistic spectrum disorders. Children and Society, 21, 4–16.
Beresford, B., Tozer, R., Rabiee, P., & Sloper, P. (2007). Desired outcomes for children and adolescents with autistic spectrum disorders. Children and Society, 21, 4–16.×
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Erlbaum.
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Erlbaum.×
Coster, W. J. (2008). Embracing ambiguity: Facing the challenge of measurement (Eleanor Clarke Slagle Lecture). American Journal of Occupational Therapy, 62, 743–752. http://dx.doi.org/10.5014/ajot.62.6.743 [Article] [PubMed]
Coster, W. J. (2008). Embracing ambiguity: Facing the challenge of measurement (Eleanor Clarke Slagle Lecture). American Journal of Occupational Therapy, 62, 743–752. http://dx.doi.org/10.5014/ajot.62.6.743 [Article] [PubMed]×
Cruice, M., Worrall, L., & Hickson, L. (2005). Personal factors, communication and vision predict social participation in older adults. International Journal of Speech–Language Pathology, 7, 220–232. http://dx.doi.org/10.1080/14417040500337088 [Article]
Cruice, M., Worrall, L., & Hickson, L. (2005). Personal factors, communication and vision predict social participation in older adults. International Journal of Speech–Language Pathology, 7, 220–232. http://dx.doi.org/10.1080/14417040500337088 [Article] ×
Doble, S. E., Bonnell, J. E., & Magill-Evans, J. (1991). Evaluation of social skills: A survey of current practice. Canadian Journal of Occupational Therapy, 58, 241–249. [Article]
Doble, S. E., Bonnell, J. E., & Magill-Evans, J. (1991). Evaluation of social skills: A survey of current practice. Canadian Journal of Occupational Therapy, 58, 241–249. [Article] ×
Doble, S. E., & Magill-Evans, J. (1992). A model of social interaction to guide occupational therapy practice. Canadian Journal of Occupational Therapy, 59, 141–150. [Article]
Doble, S. E., & Magill-Evans, J. (1992). A model of social interaction to guide occupational therapy practice. Canadian Journal of Occupational Therapy, 59, 141–150. [Article] ×
Elliott, S. N., Sheridan, S. M., & Gresham, F. M. (1989). Assessing and treating social skill deficits: A case study for the scientist–practitioner. Journal of School Psychology, 27, 197–222. http://dx.doi.org/10.1016/0022-4405(89)90007-1 [Article]
Elliott, S. N., Sheridan, S. M., & Gresham, F. M. (1989). Assessing and treating social skill deficits: A case study for the scientist–practitioner. Journal of School Psychology, 27, 197–222. http://dx.doi.org/10.1016/0022-4405(89)90007-1 [Article] ×
Fisher, A. G. (2009). Occupational therapy intervention process model: A model for planning and implementing top-down, client-centered, and occupation-based occupational therapy interventions. Fort Collins, CO: Three Star Press.
Fisher, A. G. (2009). Occupational therapy intervention process model: A model for planning and implementing top-down, client-centered, and occupation-based occupational therapy interventions. Fort Collins, CO: Three Star Press.×
Fisher, A. G., & Griswold, L. A. (2010). Evaluation of Social Interaction (2nd ed.). Fort Collins, CO: Three Star Press.
Fisher, A. G., & Griswold, L. A. (2010). Evaluation of Social Interaction (2nd ed.). Fort Collins, CO: Three Star Press.×
Fisher, A. G., & Kielhofner, G. (1995). Skill in occupational performance. In Kielhofner, G., A Model of Human Occupation: Theory and application (2nd ed., pp. 113–137). Baltimore: Williams & Wilkins.
Fisher, A. G., & Kielhofner, G. (1995). Skill in occupational performance. In Kielhofner, G., A Model of Human Occupation: Theory and application (2nd ed., pp. 113–137). Baltimore: Williams & Wilkins.×
Girolametto, L., Hoaken, L., Weitzman, E., & van Leishout, R. (2000). Patterns of adult–child linguistic interaction in integrated day care groups. Language, Speech, and Hearing Services in Schools, 31, 155–168.
Girolametto, L., Hoaken, L., Weitzman, E., & van Leishout, R. (2000). Patterns of adult–child linguistic interaction in integrated day care groups. Language, Speech, and Hearing Services in Schools, 31, 155–168.×
Griswold, L. A. S. (2010). Evaluation in the intervention planning process. In Hinojosa, J., Kramer, P., & Crist, P. (Eds.), Evaluation: Obtaining and interpreting data (3rd ed., pp. 75–102). Bethesda, MD: AOTA Press.
Griswold, L. A. S. (2010). Evaluation in the intervention planning process. In Hinojosa, J., Kramer, P., & Crist, P. (Eds.), Evaluation: Obtaining and interpreting data (3rd ed., pp. 75–102). Bethesda, MD: AOTA Press.×
Hartup, W. W. (1988). Early peer relations: Developmental significance and prognostic implications (ERIC Document Reproduction Service No. ED297885)
Hartup, W. W. (1988). Early peer relations: Developmental significance and prognostic implications (ERIC Document Reproduction Service No. ED297885)×
Kavale, K. A., & Forness, S. R. (1996). Social skill deficits and learning disabilities: A meta-analysis. Journal of Learning Disabilities, 29, 226–237. http://dx.doi.org/10.1177/002221949602900301 [Article] [PubMed]
Kavale, K. A., & Forness, S. R. (1996). Social skill deficits and learning disabilities: A meta-analysis. Journal of Learning Disabilities, 29, 226–237. http://dx.doi.org/10.1177/002221949602900301 [Article] [PubMed]×
Kielhofner, G. (1985). A Model of Human Occupation: Theory and application. Baltimore: Williams & Wilkins.
Kielhofner, G. (1985). A Model of Human Occupation: Theory and application. Baltimore: Williams & Wilkins.×
Kielhofner, G. (1995). A Model of Human Occupation: Theory and application (2nd ed.). Baltimore: Williams & Wilkins.
Kielhofner, G. (1995). A Model of Human Occupation: Theory and application (2nd ed.). Baltimore: Williams & Wilkins.×
Law, M. (2002). Participation in the occupations of everyday life. American Journal of Occupational Therapy, 56, 640–649. http://dx.doi.org/10.5014/ajot.56.6.640 [Article] [PubMed]
Law, M. (2002). Participation in the occupations of everyday life. American Journal of Occupational Therapy, 56, 640–649. http://dx.doi.org/10.5014/ajot.56.6.640 [Article] [PubMed]×
Lim, S. M., & Rodger, S. (2008). An occupational perspective on the assessment of social competence in children. British Journal of Occupational Therapy, 71, 469–481.
Lim, S. M., & Rodger, S. (2008). An occupational perspective on the assessment of social competence in children. British Journal of Occupational Therapy, 71, 469–481.×
Lim, S. M., Rodger, S., & Brown, T. (2010). Learning related and interpersonal social skills constructs in two existing social skills assessments. Occupational Therapy in Mental Health, 26, 131–150. http://dx.doi.org/10.1080/01642121003736069 [Article]
Lim, S. M., Rodger, S., & Brown, T. (2010). Learning related and interpersonal social skills constructs in two existing social skills assessments. Occupational Therapy in Mental Health, 26, 131–150. http://dx.doi.org/10.1080/01642121003736069 [Article] ×
Lord, C., Wagner, A., Rogers, S., Szatmari, P., Aman, M., Charman, T.et al (2005). Challenges in evaluating psychosocial interventions for autistic spectrum disorders. Journal of Autism and Developmental Disorders, 35, 695–708, discussion 709–711. http://dx.doi.org/10.1007/s10803-005-0017-6 [Article] [PubMed]
Lord, C., Wagner, A., Rogers, S., Szatmari, P., Aman, M., Charman, T.et al (2005). Challenges in evaluating psychosocial interventions for autistic spectrum disorders. Journal of Autism and Developmental Disorders, 35, 695–708, discussion 709–711. http://dx.doi.org/10.1007/s10803-005-0017-6 [Article] [PubMed]×
McCabe, P. C., & Marshall, D. J. (2006). Measuring the social competence of preschool children with specific language impairment: Correspondence among informant ratings and behavioral observations. Topics in Early Childhood Special Education, 26, 234–246. http://dx.doi.org/10.1177/02711214060260040401 [Article]
McCabe, P. C., & Marshall, D. J. (2006). Measuring the social competence of preschool children with specific language impairment: Correspondence among informant ratings and behavioral observations. Topics in Early Childhood Special Education, 26, 234–246. http://dx.doi.org/10.1177/02711214060260040401 [Article] ×
McConnell, S. R., & Odom, S. L. (1999). A multimeasure performance-based assessment of social competence in young children with disabilities. Topics in Early Childhood Special Education, 19, 67–74. [Article]
McConnell, S. R., & Odom, S. L. (1999). A multimeasure performance-based assessment of social competence in young children with disabilities. Topics in Early Childhood Special Education, 19, 67–74. [Article] ×
Michelson, L., Sugai, D. P., Wood, R. P., & Kazdin, A. E. (1983). Social skills assessment and training with children: An empirically based handbook. New York: Plenum Press.
Michelson, L., Sugai, D. P., Wood, R. P., & Kazdin, A. E. (1983). Social skills assessment and training with children: An empirically based handbook. New York: Plenum Press.×
Rogers, S. L. (2010). Common conditions that influence children’s participation. In Case-Smith, J., & O’Brien, J. C. (Eds.), Occupational therapy for children (6th ed., pp. 146–192). Maryland Heights, MO: Mosby/Elsevier.
Rogers, S. L. (2010). Common conditions that influence children’s participation. In Case-Smith, J., & O’Brien, J. C. (Eds.), Occupational therapy for children (6th ed., pp. 146–192). Maryland Heights, MO: Mosby/Elsevier.×
Sheridan, S. M., Hungelmann, A., & Maughan, D. P. (1999). A contextualized framework for social skills assessment, intervention, and generalization. School Psychology Review, 28, 84–103.
Sheridan, S. M., Hungelmann, A., & Maughan, D. P. (1999). A contextualized framework for social skills assessment, intervention, and generalization. School Psychology Review, 28, 84–103.×
Simmons, C. D., Griswold, L. A., & Berg, B. (2010). Evaluation of social interaction during occupational engagement. American Journal of Occupational Therapy, 64, 10–17. http://dx.doi.org/10.5014/ajot.64.1.10 [Article] [PubMed]
Simmons, C. D., Griswold, L. A., & Berg, B. (2010). Evaluation of social interaction during occupational engagement. American Journal of Occupational Therapy, 64, 10–17. http://dx.doi.org/10.5014/ajot.64.1.10 [Article] [PubMed]×
Strauss, C. C., Lease, C. A., Kazdin, A. E., Dulcan, M. K., & Last, C. G. (1989). Multimethod assessment of the social competence of children with anxiety disorders. Journal of Clinical Child Psychology, 18, 184–189. http://dx.doi.org/10.1207/s15374424jccp1802_10 [Article]
Strauss, C. C., Lease, C. A., Kazdin, A. E., Dulcan, M. K., & Last, C. G. (1989). Multimethod assessment of the social competence of children with anxiety disorders. Journal of Clinical Child Psychology, 18, 184–189. http://dx.doi.org/10.1207/s15374424jccp1802_10 [Article] ×
Turkstra, L. S. (2005). Looking while listening and speaking: Eye-to-face gaze in adolescents with and without traumatic brain injury. Journal of Speech, Language, and Hearing Research, 48, 1429–1441. http://dx.doi.org/10.1044/1092-4388(2005/099) [Article]
Turkstra, L. S. (2005). Looking while listening and speaking: Eye-to-face gaze in adolescents with and without traumatic brain injury. Journal of Speech, Language, and Hearing Research, 48, 1429–1441. http://dx.doi.org/10.1044/1092-4388(2005/099) [Article] ×
Wight, M., & Chapparo, C. (2008). Social competence and learning difficulties: Teacher perceptions. Australian Occupational Therapy Journal, 55, 256–265. http://dx.doi.org/10.1111/j.1440-1630.2007.00706.x [Article] [PubMed]
Wight, M., & Chapparo, C. (2008). Social competence and learning difficulties: Teacher perceptions. Australian Occupational Therapy Journal, 55, 256–265. http://dx.doi.org/10.1111/j.1440-1630.2007.00706.x [Article] [PubMed]×
Figure 1.
Children’s Evaluation of Social Interaction (ESI) measures, by age.
Figure 1.
Children’s Evaluation of Social Interaction (ESI) measures, by age.
×
Table 1.
Intended Purpose of Observed Social Interactions and Selected Specific Examples
Intended Purpose of Observed Social Interactions and Selected Specific Examples×
PurposeExamples
Gathering information (13%)
  • Getting instructions from a teacher
  • Receiving assistance from a teacher
  • Learning how to operate a kiln
  • Finding out what items are needed at the grocery store
Sharing information (8%)
  • Teaching a classmate how to complete a project
  • Helping a sibling read an invitation
  • Telling a teacher about a toy
  • Telling a friend about the playground equipment
  • Tattling on a sibling
  • Showing a sibling a photo album
Problem solving or decision making (4%)
  • Working through a disagreement with peers
  • Planning the rules for a hide-and-seek game with friends
  • Planning where to set up the boundaries for a chase game with friends
Collaborating or producing (51%)
  • Working on a group project with classmates
  • Working with a partner on an assignment
  • Working with classmates to organize the books on a shelf
  • Playing a game in the pool with friends
  • Playing a board game with a sibling
  • Building a sand castle with a friend
  • Building with Legos™ with a sibling
  • Playing a computer game with a friend
  • Doing a puzzle with a friend
  • Collecting rocks with a friend
  • Playing a fishing game with the class
  • Catching fish and bugs in a pond with a friend
Acquiring goods and services (2%)Getting an ice cream sundae
Conversing socially or making small talk (22%)
  • Talking with peers while eating a snack or meal
  • Talking while on the swings
Table 1.
Intended Purpose of Observed Social Interactions and Selected Specific Examples
Intended Purpose of Observed Social Interactions and Selected Specific Examples×
PurposeExamples
Gathering information (13%)
  • Getting instructions from a teacher
  • Receiving assistance from a teacher
  • Learning how to operate a kiln
  • Finding out what items are needed at the grocery store
Sharing information (8%)
  • Teaching a classmate how to complete a project
  • Helping a sibling read an invitation
  • Telling a teacher about a toy
  • Telling a friend about the playground equipment
  • Tattling on a sibling
  • Showing a sibling a photo album
Problem solving or decision making (4%)
  • Working through a disagreement with peers
  • Planning the rules for a hide-and-seek game with friends
  • Planning where to set up the boundaries for a chase game with friends
Collaborating or producing (51%)
  • Working on a group project with classmates
  • Working with a partner on an assignment
  • Working with classmates to organize the books on a shelf
  • Playing a game in the pool with friends
  • Playing a board game with a sibling
  • Building a sand castle with a friend
  • Building with Legos™ with a sibling
  • Playing a computer game with a friend
  • Doing a puzzle with a friend
  • Collecting rocks with a friend
  • Playing a fishing game with the class
  • Catching fish and bugs in a pond with a friend
Acquiring goods and services (2%)Getting an ice cream sundae
Conversing socially or making small talk (22%)
  • Talking with peers while eating a snack or meal
  • Talking while on the swings
×
Table 2.
Age, Gender, and Disability Group of Participants
Age, Gender, and Disability Group of Participants×
Number in Each Disability Group
Age, yrBoysGirlsTotalMildDevelopmentalPsychiatric
3213210
4000000
5303300
633003
7123102
8516042
9112110
10011100
11101100
12101010
Total17623977
Table 2.
Age, Gender, and Disability Group of Participants
Age, Gender, and Disability Group of Participants×
Number in Each Disability Group
Age, yrBoysGirlsTotalMildDevelopmentalPsychiatric
3213210
4000000
5303300
633003
7123102
8516042
9112110
10011100
11101100
12101010
Total17623977
×
Table 3.
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability×
GroupMean LogitSDMean DifferenceSDtdfp
Children with disability−.4435.62111
Children without disability.0557.59689−.49913.58890−4.06522.001
Table Footer NoteNote. ESI = Evaluation of Social Interaction; SD = standard deviation.
Note. ESI = Evaluation of Social Interaction; SD = standard deviation.×
Table 3.
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability
Paired t-Test Analysis for ESI Logit Measures for Children With and Without a Disability×
GroupMean LogitSDMean DifferenceSDtdfp
Children with disability−.4435.62111
Children without disability.0557.59689−.49913.58890−4.06522.001
Table Footer NoteNote. ESI = Evaluation of Social Interaction; SD = standard deviation.
Note. ESI = Evaluation of Social Interaction; SD = standard deviation.×
×