SIGNIFICANCE: In this study, we present preliminary validity and reliability evidence for the Assessment of Compared Qualities—Social Interaction (ACQ–SI), which is an innovative new tool that can be used in research and practice to measure the extent of the discrepancy between two perspectives: a person’s self-report and direct observation. The ACQ–SI is occupation-focused and supports occupation-centered practice.

APPROACH: Hypotheses for this study included the following: (1) the ACQ–SI rating scale demonstrates sound psychometric properties (i.e., ordered category measures and goodness of fit of the rating scale categories; M² ≤ 1.5); (2) the ACQ–SI items and social interaction intended purposes define a single unidimensional construct (i.e., goodness of fit; M² ≤ 1.4 and/or z < 2); (3) the ACQ–SI intended purposes/items separate people by differing levels of discrepancy between self-reported and observed quality of social interaction (separation index ≥ 2; separation reliability ≥ .80); and (4) the ACQ–SI measures are reliable (i.e., low standard error; SE ≤ 0.4).

Developing a new instrument requires evaluating preliminary data to ensure that the new tool demonstrates sound psychometric qualities. Rasch measurement methods are preferred for generating such evidence.

METHOD: This study was a cross-sectional design. Recruitment of participants was based on convenience sampling procedures. The participants were 80 adults, aged 21 to 67 yr (M = 33.5 yr, SD = 11.0), with a variety of developmental disabilities (e.g., intellectual disability, cerebral palsy, spina bifida) and living in northern Sweden. Slightly more than 50% were women, 57% were independent living, and 43% needed minimal to substantial assistance for community living. All data were gathered in community settings.

When the ACQ–SI is administered, the person is first evaluated with the Evaluation of Social Interaction (ESI). More specifically, the person is observed during two prioritized social exchanges with typical social partners. Immediately following each ESI observation, the ACQ–SI interview is administered by asking 11 open-ended questions designed to encourage the person to reflect on and report any perceived problems with social interaction. Following the ACQ–SI interview, the occupational therapist scores the ESI and compares the ESI results to the responses that the person gave during the ACQ–SI interview. The 11 ACQ–SI items are then scored, with a four-category rating scale, on the basis of the extent of discrepancy between the two perspectives: observed and reported.

Each person’s raw ACQ–SI scores were analyzed with many-facet Rasch (MFR) analysis that adjusted each person’s ACQ–SI measure to account for rater severity and challenge of the observed social exchange. MFR analyses routinely generate the statistics specified in the earlier research questions.

RESULTS: The MFR analysis revealed ordered category measures, and all categories except “no discrepancy” demonstrated acceptable fit. All of the intended purposes and 10 of 11 items demonstrated acceptable goodness of fit; one rater accounted for the misfit. The ACQ–SI intended purposes and items separated people into at least four different levels (separation index = 3.7; separation reliability = .93). The SE was 0.4.

CONCLUSION: The results support preliminary validity and good reliability of the ACQ–SI. The small sample and limited heterogeneity suggest a need to collect additional data to ensure validity of the ACQ–SI.