Katharine Preissner, Kathryn Bernice Duke, Catherine Killian, Rachel Luangdilok Ouyang, Eva D. Jarek, Anders Kottorp; The Revised American Occupational Therapy Association Fieldwork Performance Evaluations: Evaluation of Content Validity—Part 1. Am J Occup Ther 2020;74(6):7406205090. https://doi.org/10.5014/ajot.2020.044180
Download citation file:
© 2021 American Occupational Therapy Association
Importance: Occupational therapy education and practice has changed over time; however, items on the American Occupational Therapy Association’s (AOTA’s) Fieldwork Performance Evaluations (FWPEs) for the Occupational Therapy Student (OTS) and Occupational Therapy Assistant Student (OTAS) have not been updated in more than two decades.
Objective: To explore evidence of validity in relation to test content of the revised FWPEs for the OTS and OTAS
Design: A qualitative study using cognitive interviews was conducted to gather perspectives on the revised FWPEs, including updated items and a proposed rating scale. A content analysis approach was used to link patterns in responses to stable, meaningful constructs to further align and refine content of the tool before further validation.
Setting: Fieldwork sites and academic settings.
Participants: Eighteen fieldwork educators (FWEs) and academic fieldwork coordinators
Results: Seven areas for refinement were identified: (1) relevance to a variety of practice settings, (2) overlapping and redundant items, (3) long item statements, (4) double- and triple-barreled item statements, (5) alignment between OTS and OTAS items, (6) further alignment with AOTA and Accreditation Council for Occupational Therapy Education documents and contemporary practice, and (7) wording and features of the proposed rating scale. The findings guided revisions of FWPE content.
Conclusions and Relevance: Cognitive interviewing was a critical step in refining the FWPE items to support content validity. The issues identified may not have been detected using traditional approaches to survey development and therefore were critical in maximizing the validity and usefulness of the final items, which will ultimately benefit fieldwork educators and students.
What This Article Adds: This study resulted in refinements to FWPE items before the next step in the validation process, ultimately improving the final FWPE items. In addition, this article outlines a process that other researchers can use to validate similar tools.
Any instrument development process should include methods such as cognitive interviews that incorporate the viewpoints of potential users—in this case, stakeholders such as FWEs and AFWCs—who serve to identify and address concerns with the proposed tools before further quantitative validation processes are undertaken.
We recommend aligning occupational therapy educational tools with ACOTE standards, because these standards represent critical knowledge and skills for occupational therapy practice. This alignment also promotes consistency in competence across all OTS and OTAS in the United States. Although this study was initiated before the most recent ACOTE (2018) standards were published, we strongly argue for ongoing alignment between the FWPEs and both current and future standards on a regular basis.
Finally, we suggest that the process used in this study be used in other countries that do not yet have standardized and validated fieldwork evaluation tools.
This PDF is available to Subscribers Only
For full access to this pdf, sign in to an existing account, or purchase an annual subscription.