Research Article
Issue Date: July/August 2020
Published Online: May 15, 2020
Updated: June 22, 2020
Development of a Computerized Adaptive Testing System for Assessing Social Knowledge in People With Schizophrenia
Author Affiliations
  • Hsin-Yu Chiang, ScD, is Associate Professor, Department of Occupational Therapy, College of Medicine, Fu Jen Catholic University, New Taipei City, Taiwan
  • Shih-Chieh Lee, PhD, is Postdoctoral Researcher, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
  • Po-Han Lin, PhD, is Assistant Professor, International Master’s Program of Learning and Instruction, National Taipei University of Education, Taipei, Taiwan
  • Chia-Yeh Chou, MS, is Associate Professor, Department of Occupational Therapy, College of Medicine, Fu Jen Catholic University, New Taipei City, Taiwan
  • Ching-Lin Hsieh, PhD, OTR, is Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, and Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan; clhsieh@ntu.edu.tw
Article Information
Assessment Development and Testing / Mental Health / Research Articles
Research Article   |   May 15, 2020
Development of a Computerized Adaptive Testing System for Assessing Social Knowledge in People With Schizophrenia
American Journal of Occupational Therapy, May 2020, Vol. 74, 7404205050. https://doi.org/10.5014/ajot.2020.036293
American Journal of Occupational Therapy, May 2020, Vol. 74, 7404205050. https://doi.org/10.5014/ajot.2020.036293
Abstract

Importance: A psychometrically sound measure of social knowledge (SK) is necessary to assess people with schizophrenia because they tend to have moderate to severe deficits in SK.

Objective: To develop a computerized adaptive test (CAT) for assessing SK in people with schizophrenia.

Design: Two phases, consisting of (1) development and validation of an SK item bank and (2) determination of the best stopping rules for the CAT.

Setting: Two psychiatric hospitals.

Participants: Two hundred thirty-six people diagnosed with schizophrenia through convenience sampling.

Measure: Computerized Adaptive Test–Social Knowledge (CAT–SK).

Results: The SK items were examined using Rasch analysis. A CAT simulation was performed to determine the best set of stopping rules for achieving high reliability and efficiency. After unsuitable items were removed, 71 items remained with acceptable model fit (infit and outfit mean square <1.4) and no gender bias. Two suboptimal alternative sets of rules were identified. The most efficient set used 21 items to achieve acceptable Rasch reliability (.81). The most reliable set used 40 items to achieve satisfactory Rasch reliability (.88). High correlations (r > .93) between CAT–SK scores and scores on the SK item bank support the concurrent validity of the CAT–SK.

Conclusions and Relevance: The CAT–SK appears to be a valid assessment that can provide reliable or efficient measures of SK. If high reliability is needed, examiners can adopt the most reliable set of 40 items. If efficiency is the primary concern, they can adopt the most efficient set of 21 items.

What This Article Adds: The CAT–SK is a valid measure of SK with flexibility to meet examiners’ needs.