Rasch Analysis of the Assessment of Quality of Activities (A-QOA), an Observational Tool for Clients With Dementia

Masahiro Ogawa; Haruna Shirai; Seiji Nishida; Hitoshi Tanimukai

doi:10.5014/ajot.2021.039917

Abstract

Importance: The Assessment of Quality of Activities (A-QOA) is an observation-based tool for assessing the strength of engagement in an activity by the person performing it in a natural context. By quantifying the quality of engagement, the A-QOA can help occupational therapy practitioners be better able to select meaningful activities and more clearly understand the effectiveness of various choices.

Objective: To examine use of the A-QOA as a valid unidimensional scale and to clarify preliminary results on its internal scale validity and item reliability using the Rasch model.

Participants: One hundred thirty-one participants with dementia performing 262 activities.

Outcomes and Measures: We used the Rasch model to clarify the psychometric properties of A-QOA’s measurement quality.

Results: Rasch analysis revealed that 21 of the 25 items reached an acceptable level of fit, and 4 did not. After eliminating the 4 misfitting items, the resulting A-QOA was determined to have both acceptable internal scale validity and item separation reliability, which are fundamental psychometric properties of a clinical observational instrument.

Conclusions and Relevance: The A-QOA can be used to quantitatively assess the strength of engagement in an activity by the person performing it by using the observational method.

What This Article Adds: In clinical settings, the A-QOA can be used both to select activities for clients with dementia and to quantitatively show the effects of occupational therapy interventions.

In recent decades, both life expectancy and the proportion of the population composed of older people have been gradually increasing worldwide. As a result, the prevalence of dementia has also been rising. The National Institute on Aging (2017) has defined dementia as loss of cognitive functioning and behavioral abilities to the extent that it affects a person’s daily living and activities. According to this definition, all people suffering from dementia have significant problems in their everyday functioning. Because the primary goal of occupational therapy is to enable people to participate in their daily activities (World Federation of Occupational Therapy, 2012), health care professionals need to clearly assess and effectively enhance the occupations that people with dementia engage in. Successful participation in occupations by people with dementia has been shown to result in positive outcomes (Beerens et al., 2018; Gitlin et al., 2008; Graff et al., 2007). In contrast, the loss of ability to engage in occupations is a strong manifestation of unmet human needs and a significant threat to the health and well-being of people with dementia (Perrin, 1997a).

In clinical settings, assessing occupations and making a decision as to which activities are suitable for clients with dementia are often challenging. Dementia frequently removes both a person’s ability to make choices about occupations and the linguistic capability to express feelings about engaging in an activity. Dementia-related cognitive impairments include memory and orientation disturbances, loss of decision-making capacity, and decreases in language comprehension and expression (Bayles, 2003). As a result, occupational therapy practitioners often face challenges in helping clients with dementia select and perform activities.

In addition, determining the effectiveness of occupational therapy interventions for clients with dementia is difficult, particularly when assessing client-reported outcomes. One of the essential results of occupational therapy is client satisfaction, measured in terms of occupational participation. Several assessments of client satisfaction have been developed, such as the Canadian Occupational Performance Measure (Carswell et al., 2004; Law et al., 2014). However, clients with dementia might struggle with formal self-report measures that examine the effects of engaging in occupations because of difficulty in verbally expressing their feelings and experiences while performing activities. When a client’s ability to communicate is greatly limited, the choice of activities may also be limited, and therefore some practitioners use proxy assessments that ask a caregiver (e.g., spouse, adult child) about the client’s preferences. However, significant differences were found between client and proxy ratings when using the EuroQoL health questionnaire as a quality of life (QOL) assessment, especially for clients with severe dementia (Jönsson et al., 2006). Therefore, assessment by observation has been added to client- and proxy-report measures as an essential skill for occupational therapy practitioners to use in assessing and selecting meaningful activities for clients with dementia.

During the past several decades, it has been recognized that people with dementia are important agents for facilitating a broader understanding of client experiences of well-being and QOL during observation (Algar et al., 2016). Therefore, several assessments were developed to assess the effectiveness of activities for people with dementia using the observational method (Jarrott et al., 2008; Lawton et al., 1996, 1999; Perrin, 1997b; Rentz, 2002; van Weert et al., 2005). Most of these assessments have three features in common. First, they have specific observational items classified into several domains, such as mood, verbal expression, behavior in response to an activity, and relationship with other people. Second, many were developed to assess specific activities, such as those commonly undertaken in Snoezelen (van Weert et al., 2005) and Montessori contexts (Jarrott et al., 2008), to determine how these activities affect people with dementia. Third, these observational assessments are scored according to a rigorous time frame, such as every 5 or 10 min; within each time segment, scores are determined according to the duration of the behavior of the targeted activity.

Unlike previous studies, our earlier qualitative study to summarize what constituted effective assessments found that experienced practitioners regularly observed various aspects of an activity from start to finish before determining its effectiveness (Ogawa et al., 2017). Therefore, leveraging this observation, we committed to the idea that when assessing the effectiveness of activities, it is crucial to observe not just a part of the activity but the whole activity. We used this framework in developing the Assessment of Quality of Activities (A-QOA) to help guide an observational assessment of the effectiveness of activities performed by clients with dementia. For the A-QOA, activity is operationally defined as an action or sphere of action that a person engages in or is expected to engage in.

Description of the Assessment of Quality of Activities

The A-QOA is an observation-based tool for assessing the quality of engagement in activities a client performs under normal circumstances. It is used to measure the strength of engagement in an activity by the person who performs it. This metric is based on multiple factors, including the state of engaging in an activity, verbal and emotional expression, social interaction, and the effect on the person performing the activity. Although the A-QOA can be used to assess virtually any activity over a short observation, we recommend that practitioners using the assessment include observation of the activity from start to finish.

Before the current study, the A-QOA contained 25 observational items (Table 1). Twenty-two items are scored on the following 4-point scale: 1 = not observed, 2 = observed to a limited or questionable extent, 3 = observed, and 4 = observed as a strong or exceptional tendency; higher scores indicate more positive reactions to the activity. Three items (1, 11, and 15) assess negative feelings and conditions and are scored on the following 4-point scale: 1 = observed strong or exceptional state to prevent performing the activity, 2 = observed state to prevent performing the activity, 3 = partly observed state to prevent performing the activity, and 4 = not observed at all; higher scores indicate less negative responses. For example, Item 16, “Showing a smile,” is defined as “observing an expression that appears pleased with a smiling face.” The following are examples of scoring:

Client bursts into loud laughter during an activity: scored 4 points, “observed as a strong or exceptional tendency.”
Client smiles temporarily during an activity, but the duration of the smile is short: scored 2 points, “observed to a limited or questionable extent.”

Table 1.

Original 25 Items of the Assessment of Quality of Activities

Category	Item
Engagement in activity	1. No conditions to prevent participation in the activity^a 2. Preparing for the activity^a 3. Initiating the activity 4. Directing one’s gaze at the activity 5. Positioning one’s body toward the activity 6. Continuing the activity 7. Concentrating on the activity 8. Demonstrating activity-related knowledge and techniques 9. Making selections or showing preferences during the activity 10. Working out a way to make the activity progress smoothly
Results of activity	11. No negative feelings after the activity^a 12. Expressing satisfaction as a result of the activity 13. Expressing a sense of capability 14. Showing willingness toward the next activity
Expression of emotions	15. No negative emotions during the activity^a 16. Showing a smile 17. Showing excitement
Social interaction through the activity	18. Interacting with others through the activity 19. Cooperating with others in the activity 20. Sharing activity-related knowledge and techniques with others 21. Conveying one’s intention to others 22. Being considerate of others 23. Sharing activity-induced emotions with others
Verbal expressions	24. Increasing utterance 25. Reminiscing

^aMean square met the criteria to remove items.

View Large

Table 1.

Original 25 Items of the Assessment of Quality of Activities

Category	Item
Engagement in activity	1. No conditions to prevent participation in the activity^a 2. Preparing for the activity^a 3. Initiating the activity 4. Directing one’s gaze at the activity 5. Positioning one’s body toward the activity 6. Continuing the activity 7. Concentrating on the activity 8. Demonstrating activity-related knowledge and techniques 9. Making selections or showing preferences during the activity 10. Working out a way to make the activity progress smoothly
Results of activity	11. No negative feelings after the activity^a 12. Expressing satisfaction as a result of the activity 13. Expressing a sense of capability 14. Showing willingness toward the next activity
Expression of emotions	15. No negative emotions during the activity^a 16. Showing a smile 17. Showing excitement
Social interaction through the activity	18. Interacting with others through the activity 19. Cooperating with others in the activity 20. Sharing activity-related knowledge and techniques with others 21. Conveying one’s intention to others 22. Being considerate of others 23. Sharing activity-induced emotions with others
Verbal expressions	24. Increasing utterance 25. Reminiscing

^aMean square met the criteria to remove items.

Development of the Assessment of Quality of Activities

We began a series of four studies to develop the A-QOA at the beginning of 2014. In the first study, to gather observational items, we performed a quantitative investigation based on interviews with 10 occupational therapists and their experiences with 47 clients who had received occupational therapy services (Ogawa et al., 2017). The interview comprised open-ended questions aimed at soliciting information around the topic related to the question, “What do you look for in clients with dementia when you assess the effects of an activity on them?” From this study, we obtained 19 observational items.

We then explored other observational items with seven occupational therapists who were experts on children with multiple severe disabilities, interviewing them using the same methodology as the first study. The interviews, based on their experiences with 37 clients, yielded 10 observational items that overlapped with those of the previous study. In the third study, we confirmed the content validity of the observational items and definitions obtained from the first two studies using the Delphi method. The 153 occupational therapists who participated achieved more than 75% agreement on all the items. The names and definitions of the items were modified on the basis of feedback received in the sessions. In the course of the first three studies, we collected many observational examples to help in the scoring of each item, which contributed significantly to the development of the user manual for the instrument. Finally, we merged these results with those of several additional previous studies and created a draft version of the assessment manual for the A-QOA. Repeated pilot tests of this manual were performed over 2 yr to modify its contents on the basis of what we learned from observing 35 videos of people with dementia engaging in a wide variety of activities.

After we developed the A-QOA, a remaining problem was that summing the qualitative ordinal counts to create a total score did not result in a number that was a valid means of making quantitative comparisons. To make such inferences, the numbers must be of equal intervals (Fisher, 1993). In solving this problem, we sought to contribute to the field of clinical occupational therapy and care for people with dementia. By providing a quantification of the A-QOA, we hoped to help occupational therapy practitioners be better able to select meaningful activities and more clearly understand the effectiveness of various choices. To this end, the objective of this study was to examine the use of the A-QOA as a valid unidimensional scale and to clarify preliminary results on its internal scale validity and item reliability using the Rasch model.

Method

Participants

All participants were people diagnosed with dementia at the hospitals and facilities in Japan with which the members of our research team were affiliated. Our collaborators in this effort met three criteria: They (1) were occupational therapists, (2) had regular contact with patients diagnosed with dementia in clinical settings, and (3) were affiliated with a facility that agreed to cooperate with this study. The participants were selected by our collaborators at their affiliated hospital or facility. Prior to data collection, the collaborators received the A-QOA assessment manual and an 8-hr training that described the use of the instrument and the observational items and provided practice in assessing four filmed activity scenes with feedback.

Data Collection

The collaborators selected two activities for each participant, one considered meaningful and the other considered less valuable and engaging. The purpose of choosing activities in this way was to avoid skewed assessment data and to rate both high-scoring and low-scoring activities on a broad scale. In addition, the collaborators were asked to choose situations that encouraged interpersonal exchanges and activities that would induce emotional expressions. Moreover, we collected data on age, gender, and dementia severity, assessed using the Functional Assessment Staging Test (FAST; Reisberg, 1984).

The collaborators, their affiliated facilities, and the participants were informed of the purpose and methods of the study, and consent was obtained from all of them. This research was approved by the Ethics Review Committee of Bukkyo University (H30–6-B).

Statistical Analysis

The Rasch model was used to clarify the psychometric properties of the measurement quality of the A-QOA and to test the observational items for goodness of fit. Any misfitting items were removed, and the test was repeated until the goodness of fit reached an acceptable level for all the remaining items. The Rasch model produces two fit indexes, namely, infit and outfit (Linacre, 2012a). Each index is expressed as a mean square (MnSq) and a z score, and both statistics can be used to set standards, such as 0.75 ≤ MnSq ≤ 1.3, and z < 2 (Bond & Fox, 2007). At present, there is no clear agreement about how to set these standards. Because the A-QOA is used for clinical observations, we followed a criterion best suited for this application (Wright & Linacre, 1994) and decided to set the acceptable range for infit and outfit MnSq values at between 0.5 and 1.7. We stopped the item removal process when the items had infit and outfit MnSq values that met the acceptable range.

After our item analysis was complete, we evaluated the assessment of reliability, which involved examining the person and item separation indexes and strata. Reliable results are indicated by reliability coefficients close to 1.0, which in turn generally require a separation index of >2.0 on both the item and person dimensions (Linacre, 2012b). The Rasch analysis with two facets, items and activities (persons), was performed using Winsteps^® Version 4.3.2 (Winsteps.com; Beaverton, OR), with the joint maximum likelihood estimation and the normal approximation algorithm as the estimation method.

Results

Participants

Eighteen study collaborators generated observational data from 131 participants undertaking 262 activities. The correlation (r) between two scores on the A-QOA for each participant was .531 (p < .001). The participants were 88 women and 43 men with a mean age of 84.4 (SD = 7.8). Table 2 shows participants’ FAST stages and types of observed activities.

Table 2.

Participants’ FAST Stages and Types of Activities Observed

Characteristic	n
FAST stage
Stage 2 (age appropriate)	4
Stage 3 (borderline)	4
Stage 4 (mild)	26
Stage 5 (moderate)	36
Stage 6 (slightly severe)	54
Stage 7 (severe)	6
Unknown	1
Total participants	131
Types of activities observed
Static activities (e.g., watching television, reading, playing board games)	39
Dynamic activities (e.g., sports, exercises)	68
Creative and expressive activities (e.g., handicrafts, singing, dancing)	106
Conversational activities (e.g., reminiscing)	22
Daily living activities (e.g., ADLs, IADLs)	26
Unknown	1
Total activities	262

Note. ADLs = activities of daily living; FAST = Functional Assessment Staging Test; IADLs = instrumental activities of daily living.

View Large

Table 2.

Participants’ FAST Stages and Types of Activities Observed

Characteristic	n
FAST stage
Stage 2 (age appropriate)	4
Stage 3 (borderline)	4
Stage 4 (mild)	26
Stage 5 (moderate)	36
Stage 6 (slightly severe)	54
Stage 7 (severe)	6
Unknown	1
Total participants	131
Types of activities observed
Static activities (e.g., watching television, reading, playing board games)	39
Dynamic activities (e.g., sports, exercises)	68
Creative and expressive activities (e.g., handicrafts, singing, dancing)	106
Conversational activities (e.g., reminiscing)	22
Daily living activities (e.g., ADLs, IADLs)	26
Unknown	1
Total activities	262

Note. ADLs = activities of daily living; FAST = Functional Assessment Staging Test; IADLs = instrumental activities of daily living.

Goodness of Fit of Observational Items

The initial Rasch model analysis revealed that 21 of the 25 observational items reached an acceptable level of fit (Table 3). The four outliers were “Preparing for the activity” (difficulty = 2.51, infit MnSq = 1.59, outfit MnSq = 1.99), “No negative emotions during the activity” (difficulty = −3.06, infit MnSq = 1.86, outfit MnSq = 1.75), “No negative feelings after the activity” (difficulty = −3.70, infit MnSq = 1.92, outfit MnSq = 1.61), and “No conditions to prevent participation in the activity” (difficulty = −4.57, infit MnSq = 1.71, outfit MnSq = 4.46).

Table 3.

Rasch Analysis of the Original 25-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
2. Preparing for the activity	2.51	0.12	1.59^a	5.14	1.99^a	4.29
20. Sharing activity-related knowledge and techniques with others	2.00	0.11	1.45	4.40	1.24	1.56
17. Showing excitement	1.89	0.11	0.83	−2.04	0.73	−2.06
25. Reminiscing	1.48	0.10	1.46	4.83	1.37	2.78
13. Expressing a sense of capability	1.41	0.10	0.79	−2.70	0.72	−2.61
22. Being considerate of others	0.99	0.10	0.98	−0.20	1.08	0.81
10. Working out a way to make the activity progress smoothly	0.97	0.10	1.41	4.43	1.42	3.61
14. Showing willingness toward the next activity	0.91	0.10	0.81	−2.47	0.81	−1.90
19. Cooperating with others in the activity	0.89	0.10	1.12	1.47	1.09	0.87
9. Making selections or showing preferences during the activity	0.87	0.10	1.20	2.27	1.24	2.25
23. Sharing activity-induced emotions with others	0.75	0.10	0.69	−4.20	0.67	−3.75
8. Demonstrating activity-related knowledge and techniques	0.52	0.10	1.37	4.05	1.34	3.27
24. Increasing utterance	0.42	0.10	0.64	−4.99	0.68	−3.83
16. Showing a smile	0.41	0.10	0.70	−4.00	0.70	−3.53
12. Expressing satisfaction as a result of the activity	0.28	0.10	0.82	−2.26	0.79	−2.39
18. Interacting with others through the activity	0.28	0.10	0.74	−3.39	0.79	−2.47
21. Conveying one’s intention to others	0.20	0.10	0.81	−2.39	0.85	−1.64
7. Concentrating on the activity	−0.79	0.10	0.62	−5.01	0.62	−4.93
3. Initiating the activity	−1.00	0.10	0.72	−3.56	0.71	−3.64
6. Continuing the activity	−1.03	0.10	0.62	−5.09	0.59	−5.49
5. Positioning one’s body toward the activity	−1.29	0.10	0.55	−6.15	0.53	−6.38
4. Directing one’s gaze at the activity	−1.35	0.10	0.55	−6.07	0.58	−5.57
15. No negative emotions during the activity	−3.06	0.12	1.86^a	7.83	1.75^a	5.89
11. No negative feelings after the activity	−3.70	0.13	1.92^a	7.87	1.61	3.88
1. No conditions to prevent participation in the activity	−4.57	0.15	1.71^a	5.42	4.46^a	9.50

Note. A-QOA = Assessment of Quality of Activities.

^aMean square met the criteria to remove items.

View Large

Table 3.

Rasch Analysis of the Original 25-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
2. Preparing for the activity	2.51	0.12	1.59^a	5.14	1.99^a	4.29
20. Sharing activity-related knowledge and techniques with others	2.00	0.11	1.45	4.40	1.24	1.56
17. Showing excitement	1.89	0.11	0.83	−2.04	0.73	−2.06
25. Reminiscing	1.48	0.10	1.46	4.83	1.37	2.78
13. Expressing a sense of capability	1.41	0.10	0.79	−2.70	0.72	−2.61
22. Being considerate of others	0.99	0.10	0.98	−0.20	1.08	0.81
10. Working out a way to make the activity progress smoothly	0.97	0.10	1.41	4.43	1.42	3.61
14. Showing willingness toward the next activity	0.91	0.10	0.81	−2.47	0.81	−1.90
19. Cooperating with others in the activity	0.89	0.10	1.12	1.47	1.09	0.87
9. Making selections or showing preferences during the activity	0.87	0.10	1.20	2.27	1.24	2.25
23. Sharing activity-induced emotions with others	0.75	0.10	0.69	−4.20	0.67	−3.75
8. Demonstrating activity-related knowledge and techniques	0.52	0.10	1.37	4.05	1.34	3.27
24. Increasing utterance	0.42	0.10	0.64	−4.99	0.68	−3.83
16. Showing a smile	0.41	0.10	0.70	−4.00	0.70	−3.53
12. Expressing satisfaction as a result of the activity	0.28	0.10	0.82	−2.26	0.79	−2.39
18. Interacting with others through the activity	0.28	0.10	0.74	−3.39	0.79	−2.47
21. Conveying one’s intention to others	0.20	0.10	0.81	−2.39	0.85	−1.64
7. Concentrating on the activity	−0.79	0.10	0.62	−5.01	0.62	−4.93
3. Initiating the activity	−1.00	0.10	0.72	−3.56	0.71	−3.64
6. Continuing the activity	−1.03	0.10	0.62	−5.09	0.59	−5.49
5. Positioning one’s body toward the activity	−1.29	0.10	0.55	−6.15	0.53	−6.38
4. Directing one’s gaze at the activity	−1.35	0.10	0.55	−6.07	0.58	−5.57
15. No negative emotions during the activity	−3.06	0.12	1.86^a	7.83	1.75^a	5.89
11. No negative feelings after the activity	−3.70	0.13	1.92^a	7.87	1.61	3.88
1. No conditions to prevent participation in the activity	−4.57	0.15	1.71^a	5.42	4.46^a	9.50

Note. A-QOA = Assessment of Quality of Activities.

^aMean square met the criteria to remove items.

The next round of Rasch analysis was performed without the data for the four outlier items. All 21 remaining items showed acceptable goodness of fit as determined by both infit and outfit MnSq (Table 4).

Table 4.

Rasch Analysis of the Final 21-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
20. Sharing activity-related knowledge and techniques with others	2.86	0.11	1.49	4.73	1.34	1.92
17. Showing excitement	2.75	0.11	0.86	−1.60	0.77	−1.54
25. Reminiscing	2.30	0.11	1.53	5.35	1.39	2.70
13. Expressing a sense of capability	2.22	0.11	0.87	−1.53	0.81	−1.58
22. Being considerate of others	1.76	0.10	1.06	0.76	1.20	1.72
10. Working out a way to make the activity progress smoothly	1.74	0.10	1.49	5.14	1.47	3.80
14. Showing willingness toward the next activity	1.67	0.10	0.89	−1.37	0.93	−0.66
19. Cooperating with others in the activity	1.65	0.10	1.27	3.08	1.24	2.09
9. Making selections or showing preferences during the activity	1.63	0.10	1.29	3.21	1.27	2.41
23. Sharing activity-induced emotions with others	1.49	0.10	0.76	−3.18	0.72	−3.00
8. Demonstrating activity-related knowledge and techniques	1.23	0.10	1.50	5.30	1.46	4.14
24. Increasing utterance	1.12	0.10	0.70	−3.97	0.73	−3.09
16. Showing a smile	1.11	0.10	0.81	−2.37	0.83	−1.87
12. Expressing satisfaction as a result of the activity	0.96	0.10	0.93	−0.79	0.92	−0.85
18. Interacting with others through the activity	0.96	0.10	0.80	−2.51	0.86	−1.50
21. Conveying one’s intention to others	0.88	0.10	0.88	−1.45	0.92	−0.81
7. Concentrating on the activity	−0.26	0.11	0.76	−2.92	0.76	−2.72
3. Initiating the activity	−0.52	0.11	0.85	−1.65	0.85	−1.66
6. Continuing the activity	−0.54	0.11	0.74	−3.10	0.69	−3.60
5. Positioning one’s body toward the activity	−0.85	0.11	0.63	−4.56	0.59	−4.97
4. Directing one’s gaze at the activity	−0.93	0.11	0.64	−4.40	0.66	−4.03

Note. A-QOA = Assessment of Quality of Activities.

View Large

Table 4.

Rasch Analysis of the Final 21-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
20. Sharing activity-related knowledge and techniques with others	2.86	0.11	1.49	4.73	1.34	1.92
17. Showing excitement	2.75	0.11	0.86	−1.60	0.77	−1.54
25. Reminiscing	2.30	0.11	1.53	5.35	1.39	2.70
13. Expressing a sense of capability	2.22	0.11	0.87	−1.53	0.81	−1.58
22. Being considerate of others	1.76	0.10	1.06	0.76	1.20	1.72
10. Working out a way to make the activity progress smoothly	1.74	0.10	1.49	5.14	1.47	3.80
14. Showing willingness toward the next activity	1.67	0.10	0.89	−1.37	0.93	−0.66
19. Cooperating with others in the activity	1.65	0.10	1.27	3.08	1.24	2.09
9. Making selections or showing preferences during the activity	1.63	0.10	1.29	3.21	1.27	2.41
23. Sharing activity-induced emotions with others	1.49	0.10	0.76	−3.18	0.72	−3.00
8. Demonstrating activity-related knowledge and techniques	1.23	0.10	1.50	5.30	1.46	4.14
24. Increasing utterance	1.12	0.10	0.70	−3.97	0.73	−3.09
16. Showing a smile	1.11	0.10	0.81	−2.37	0.83	−1.87
12. Expressing satisfaction as a result of the activity	0.96	0.10	0.93	−0.79	0.92	−0.85
18. Interacting with others through the activity	0.96	0.10	0.80	−2.51	0.86	−1.50
21. Conveying one’s intention to others	0.88	0.10	0.88	−1.45	0.92	−0.81
7. Concentrating on the activity	−0.26	0.11	0.76	−2.92	0.76	−2.72
3. Initiating the activity	−0.52	0.11	0.85	−1.65	0.85	−1.66
6. Continuing the activity	−0.54	0.11	0.74	−3.10	0.69	−3.60
5. Positioning one’s body toward the activity	−0.85	0.11	0.63	−4.56	0.59	−4.97
4. Directing one’s gaze at the activity	−0.93	0.11	0.64	−4.40	0.66	−4.03

Note. A-QOA = Assessment of Quality of Activities.

Reliability Analysis

The reliability analysis revealed that the item separation index was 10.27, and the strata were 7.18. The item separation reliability was .99. For the participants, the person separation index was 4.11, and the strata were 3.07. The person separation reliability was .94.

Discussion

One of the key objectives of this study was to apply Rasch analysis to examine the psychometric properties of the A-QOA as a rating scale and to indicate the outcome of activities. We found that this instrument can be used as a valid unidimensional scale. The 21-item A-QOA was shown to have acceptable internal scale validity and item separation reliability for a valid clinical observational instrument. Six of the remaining 21 items had z scores >2.0, indicating that these items might have unexpected outlying observations and inlying patterns. According to the Winsteps manual (Linacre, 2012a), MnSq is preferred over the z score in evaluating statistical fit because the former shows the size and the latter shows the significance of being out of expectation according to the statistical analysis. Moreover, in this context, a property of the z score is that a bigger sample size leads to a bigger z score. For these reasons, using the MnSq (i.e., an index invariant with sample size) to omit items in this study seems to be a valid methodology.

However, high z scores indicated some unexpected patterns and suggest that more analysis is needed. The Rasch model is generally used for measures that assess a person’s ability, such as the Assessment of Motor and Process Skills (Fisher, 1993). In contrast, the A-QOA evaluates a series of responses to engaging in an activity rather than assessing an ability. In addition, the observational items of the A-QOA involve social interaction, emotion, and verbal expression, which are induced by social contact. Therefore, different patterns were observed because behaviors might be dependent on the type of activity and the number of people engaged in the activity. More analysis of the misfitting data would help in developing improved guidelines regarding the scope of activities that can be assessed by the A-QOA.

The four items found to be misfitted in the first Rasch analysis are worthy of discussion regarding their selection for further revision of the items. Three were negative items, meaning that their difficulty levels were substantially lower than those of the other items. In addition, these three items were scored differently from the positive items. In other words, negative feeling and attitudes during activities would be rare and haphazardly observed, regardless of positive items in this assessment situation. For example, one can show some negative behaviors while participating in an activity but still fully engage in it. However, these items could be important to assess the effect of activities. Therefore, these negative aspects might be assessed by another axis that could be different from the positive axis.

The fourth misfitted item was “Preparing for the activity,” which was the most difficult item in the entire initial analysis. In clinical settings for people with dementia, occupational therapy practitioners and care staff often prepare the activities to be undertaken because of clients’ limited ability and need for increased time to prepare. Because clients typically do not participate in preparing for activities, this fourth item was misfitted. However, “Preparing for the activity” is expected to be a critical test item that should not simply be deleted. We will examine the possibility that “Preparing for the activity” can be incorporated into the item “Initiating the activity,” which has similar concepts, in future research.

Implications for Occupational Therapy Practice

The findings of this study have the following implications for occupational therapy practice:

The A-QOA can be used to assess, verbalize, and quantitatively show the effectiveness of activities using an observational method with a client-centered viewpoint.
The A-QOA can help occupational therapists in clinical practice select optimal activities for clients with dementia and demonstrate the success of the corresponding interventions.

Study Limitations

This study of the A-QOA confirmed its internal scale validity and item reliability through the use of small samples of participants with dementia in an Asian country. In the future, additional analysis should be conducted with larger samples and using other Rasch methods such as analysis of standardized residuals and analysis of unexpected responses to verify unidimensionality. The A-QOA is a criterion-referenced test that has been refined in light of analysis using the Rasch model. A broader reference data set should be developed to help produce more stable results of assessments. Similarly, application of the A-QOA with different diagnostic groups in various cultural contexts could also profitably be studied.

Conclusion

The A-QOA can be used to quantitatively assess the strength of engagement in an activity by the person who performs it. The results of this study enhance the value of the A-QOA in occupational therapy practice and as an index of the effectiveness of occupational therapy in research settings. Therefore, we recommend the wide use of the A-QOA across settings in the future.

Acknowledgments

This work was supported by KAKENHI Grants-in-Aid for Young Scientists (16K20818,19K19704) from the Japan Society for the Promotion of Science.

References

Algar, K., Woods, R. T., & Windle, G. (2016). Measuring the quality of life and well-being of people with dementia: A review of observational measures. Dementia, 15, 832–857. https://doi.org/10.1177/1471301214540163 [Article] [PubMed]

Bayles, K. A. (2003). Effects of working memory deficits on the communicative functioning of Alzheimer’s dementia patients. Journal of Communication Disorders, 36, 209–219. https://doi.org/10.1016/S0021-9924(03)00020-0 [Article] [PubMed]

Beerens, H. C., Zwakhalen, S. M. G., Verbeek, H., Tan, F. E. S., Jolani, S., Downs, M., . . . Hamers, J. P. H. (2018). The relation between mood, activity, and interaction in long-term dementia care. Aging and Mental Health, 22, 26–32. https://doi.org/10.1080/13607863.2016.1227766 [Article] [PubMed]

Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Erlbaum.

Carswell, A., McColl, M. A., Baptiste, S., Law, M., Polatajko, H., & Pollock, N. (2004). The Canadian Occupational Performance Measure: A research and clinical literature review. Canadian Journal of Occupational Therapy, 71, 210–222. https://doi.org/10.1177/000841740407100406 [Article]

Fisher, A. G. (1993). The assessment of IADL motor skills: An application of many-faceted Rasch analysis. American Journal of Occupational Therapy, 47, 319–329. https://doi.org/10.5014/ajot.47.4.319 [Article]

Gitlin, L. N., Winter, L., Burke, J., Chernett, N., Dennis, M. P., & Hauck, W. W. (2008). Tailored activities to manage neuropsychiatric behaviors in persons with dementia and reduce caregiver burden: A randomized pilot study. American Journal of Geriatric Psychiatry, 16, 229–239. https://doi.org/10.1097/01.JGP.0000300629.35408.94 [Article]

Graff, M. J. L., Vernooij-Dassen, M. J. M., Thijssen, M., Dekker, J., Hoefnagels, W. H. L., & Olderikkert, M. G. (2007). Effects of community occupational therapy on quality of life, mood, and health status in dementia patients and their caregivers: A randomized controlled trial. Journals of Gerontology, Series A: Biological Sciences and Medical Sciences, 62, 1002–1009. https://doi.org/10.1093/gerona/62.9.1002 [Article]

Jarrott, S. E., Gozali, T., & Gigliotti, G. M. (2008). Montessori programming for persons with dementia in the group setting: An analysis of engagement and affect. Dementia, 7, 109–125. https://doi.org/10.1177/1471301207085370 [Article]

Jönsson, L., Andreasen, N., Kilander, L., Soininen, H., Waldemar, G., Nygaard, H., . . . Wimo, A. (2006). Patient- and proxy-reported utility in Alzheimer disease using the EuroQoL. Alzheimer Disease and Associated Disorders, 20, 49–55. https://doi.org/10.1097/01.wad.0000201851.52707.c9 [Article] [PubMed]

Law, M., Baptiste, S., Carswell, A., McColl, M. A., Polatajko, H., & Pollock, N. (2014). Canadian Occupational Performance Measure (5th ed.). CAOT Publications.

Lawton, M. P., Van Haitsma, K., & Klapper, J. (1996). Observed affect in nursing home residents with Alzheimer’s disease. Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 51, P3–P14. https://doi.org/10.1093/geronb/51B.1.P3 [Article]

Lawton, M. P., Van Haitsma, K., Perkinson, M., & Ruckdeschel, K. (1999). Observed affect and quality of life in dementia: Further affirmations and problems. Journal of Mental Health and Aging, 5, 69–81.

Linacre, M. (2012a). Winsteps Rasch Tutorial 2: Fit analysis and measurement models. https://www.winsteps.com/a/winsteps-tutorial-2.pdf

Linacre, M. (2012b). Winsteps Rasch Tutorial 3: Partial credit scales, reliability and anchoring. https://www.winsteps.com/a/winsteps-tutorial-3.pdf

National Institute on Aging. (2017). Basics of Alzheimer’s disease and dementia: What is dementia? Symptoms, types, and diagnosis. https://www.nia.nih.gov/health/what-dementia-symptoms-types-and-diagnosis

Ogawa, M., Nishida, S., & Shirai, H. (2017). A qualitative study to explore ways to observe the results of engaging activities in clients with dementia. Occupational Therapy International, 2017, 7513875. https://doi.org/10.1155/2017/7513875

Perrin, T. (1997a). Occupational need in severe dementia: A descriptive study. Journal of Advanced Nursing, 25, 934–941. https://doi.org/10.1046/j.1365-2648.1997.1997025934.x [Article]

Perrin, T. (1997b). The Positive Response Schedule for Severe Dementia. Aging and Mental Health, 1, 184–191. https://doi.org/10.1080/13607869757290 [Article]

Reisberg, B. (1984). Functional staging of dementia of the Alzheimer type. Annals of the New York Academy of Sciences, 435, 481–483. https://doi.org/10.1111/j.1749-6632.1984.tb13859.x [Article]

Rentz, C. A. (2002). Memories in the making: Outcome-based evaluation of an art program for individuals with dementing illnesses. American Journal of Alzheimer’s Disease and Other Dementias, 17, 175–181. https://doi.org/10.1177/153331750201700310 [Article] [PubMed]

Van Weert, J. C. M., van Dulmen, A. M., Spreeuwenberg, P. M. M., Ribbe, M. W., & Bensing, J. M. (2005). Behavioral and mood effects of Snoezelen integrated into 24-hour dementia care. Journal of the American Geriatrics Society, 53, 24–33. https://doi.org/10.1111/j.1532-5415.2005.53006.x [Article] [PubMed]

World Federation of Occupational Therapy. (2012). About occupational therapy. https://wfot.org/about/about-occupational-therapy

Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. https://www.rasch.org/rmt/rmt83b.htm

Table 1.

Original 25 Items of the Assessment of Quality of Activities

Category	Item
Engagement in activity	1. No conditions to prevent participation in the activity^a 2. Preparing for the activity^a 3. Initiating the activity 4. Directing one’s gaze at the activity 5. Positioning one’s body toward the activity 6. Continuing the activity 7. Concentrating on the activity 8. Demonstrating activity-related knowledge and techniques 9. Making selections or showing preferences during the activity 10. Working out a way to make the activity progress smoothly
Results of activity	11. No negative feelings after the activity^a 12. Expressing satisfaction as a result of the activity 13. Expressing a sense of capability 14. Showing willingness toward the next activity
Expression of emotions	15. No negative emotions during the activity^a 16. Showing a smile 17. Showing excitement
Social interaction through the activity	18. Interacting with others through the activity 19. Cooperating with others in the activity 20. Sharing activity-related knowledge and techniques with others 21. Conveying one’s intention to others 22. Being considerate of others 23. Sharing activity-induced emotions with others
Verbal expressions	24. Increasing utterance 25. Reminiscing

^aMean square met the criteria to remove items.

View Large

Table 1.

Original 25 Items of the Assessment of Quality of Activities

Category	Item
Engagement in activity	1. No conditions to prevent participation in the activity^a 2. Preparing for the activity^a 3. Initiating the activity 4. Directing one’s gaze at the activity 5. Positioning one’s body toward the activity 6. Continuing the activity 7. Concentrating on the activity 8. Demonstrating activity-related knowledge and techniques 9. Making selections or showing preferences during the activity 10. Working out a way to make the activity progress smoothly
Results of activity	11. No negative feelings after the activity^a 12. Expressing satisfaction as a result of the activity 13. Expressing a sense of capability 14. Showing willingness toward the next activity
Expression of emotions	15. No negative emotions during the activity^a 16. Showing a smile 17. Showing excitement
Social interaction through the activity	18. Interacting with others through the activity 19. Cooperating with others in the activity 20. Sharing activity-related knowledge and techniques with others 21. Conveying one’s intention to others 22. Being considerate of others 23. Sharing activity-induced emotions with others
Verbal expressions	24. Increasing utterance 25. Reminiscing

^aMean square met the criteria to remove items.

Table 2.

Participants’ FAST Stages and Types of Activities Observed

Characteristic	n
FAST stage
Stage 2 (age appropriate)	4
Stage 3 (borderline)	4
Stage 4 (mild)	26
Stage 5 (moderate)	36
Stage 6 (slightly severe)	54
Stage 7 (severe)	6
Unknown	1
Total participants	131
Types of activities observed
Static activities (e.g., watching television, reading, playing board games)	39
Dynamic activities (e.g., sports, exercises)	68
Creative and expressive activities (e.g., handicrafts, singing, dancing)	106
Conversational activities (e.g., reminiscing)	22
Daily living activities (e.g., ADLs, IADLs)	26
Unknown	1
Total activities	262

Note. ADLs = activities of daily living; FAST = Functional Assessment Staging Test; IADLs = instrumental activities of daily living.

View Large

Table 2.

Participants’ FAST Stages and Types of Activities Observed

Characteristic	n
FAST stage
Stage 2 (age appropriate)	4
Stage 3 (borderline)	4
Stage 4 (mild)	26
Stage 5 (moderate)	36
Stage 6 (slightly severe)	54
Stage 7 (severe)	6
Unknown	1
Total participants	131
Types of activities observed
Static activities (e.g., watching television, reading, playing board games)	39
Dynamic activities (e.g., sports, exercises)	68
Creative and expressive activities (e.g., handicrafts, singing, dancing)	106
Conversational activities (e.g., reminiscing)	22
Daily living activities (e.g., ADLs, IADLs)	26
Unknown	1
Total activities	262

Note. ADLs = activities of daily living; FAST = Functional Assessment Staging Test; IADLs = instrumental activities of daily living.

Table 3.

Rasch Analysis of the Original 25-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
2. Preparing for the activity	2.51	0.12	1.59^a	5.14	1.99^a	4.29
20. Sharing activity-related knowledge and techniques with others	2.00	0.11	1.45	4.40	1.24	1.56
17. Showing excitement	1.89	0.11	0.83	−2.04	0.73	−2.06
25. Reminiscing	1.48	0.10	1.46	4.83	1.37	2.78
13. Expressing a sense of capability	1.41	0.10	0.79	−2.70	0.72	−2.61
22. Being considerate of others	0.99	0.10	0.98	−0.20	1.08	0.81
10. Working out a way to make the activity progress smoothly	0.97	0.10	1.41	4.43	1.42	3.61
14. Showing willingness toward the next activity	0.91	0.10	0.81	−2.47	0.81	−1.90
19. Cooperating with others in the activity	0.89	0.10	1.12	1.47	1.09	0.87
9. Making selections or showing preferences during the activity	0.87	0.10	1.20	2.27	1.24	2.25
23. Sharing activity-induced emotions with others	0.75	0.10	0.69	−4.20	0.67	−3.75
8. Demonstrating activity-related knowledge and techniques	0.52	0.10	1.37	4.05	1.34	3.27
24. Increasing utterance	0.42	0.10	0.64	−4.99	0.68	−3.83
16. Showing a smile	0.41	0.10	0.70	−4.00	0.70	−3.53
12. Expressing satisfaction as a result of the activity	0.28	0.10	0.82	−2.26	0.79	−2.39
18. Interacting with others through the activity	0.28	0.10	0.74	−3.39	0.79	−2.47
21. Conveying one’s intention to others	0.20	0.10	0.81	−2.39	0.85	−1.64
7. Concentrating on the activity	−0.79	0.10	0.62	−5.01	0.62	−4.93
3. Initiating the activity	−1.00	0.10	0.72	−3.56	0.71	−3.64
6. Continuing the activity	−1.03	0.10	0.62	−5.09	0.59	−5.49
5. Positioning one’s body toward the activity	−1.29	0.10	0.55	−6.15	0.53	−6.38
4. Directing one’s gaze at the activity	−1.35	0.10	0.55	−6.07	0.58	−5.57
15. No negative emotions during the activity	−3.06	0.12	1.86^a	7.83	1.75^a	5.89
11. No negative feelings after the activity	−3.70	0.13	1.92^a	7.87	1.61	3.88
1. No conditions to prevent participation in the activity	−4.57	0.15	1.71^a	5.42	4.46^a	9.50

Note. A-QOA = Assessment of Quality of Activities.

^aMean square met the criteria to remove items.

View Large

Table 3.

Rasch Analysis of the Original 25-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
2. Preparing for the activity	2.51	0.12	1.59^a	5.14	1.99^a	4.29
20. Sharing activity-related knowledge and techniques with others	2.00	0.11	1.45	4.40	1.24	1.56
17. Showing excitement	1.89	0.11	0.83	−2.04	0.73	−2.06
25. Reminiscing	1.48	0.10	1.46	4.83	1.37	2.78
13. Expressing a sense of capability	1.41	0.10	0.79	−2.70	0.72	−2.61
22. Being considerate of others	0.99	0.10	0.98	−0.20	1.08	0.81
10. Working out a way to make the activity progress smoothly	0.97	0.10	1.41	4.43	1.42	3.61
14. Showing willingness toward the next activity	0.91	0.10	0.81	−2.47	0.81	−1.90
19. Cooperating with others in the activity	0.89	0.10	1.12	1.47	1.09	0.87
9. Making selections or showing preferences during the activity	0.87	0.10	1.20	2.27	1.24	2.25
23. Sharing activity-induced emotions with others	0.75	0.10	0.69	−4.20	0.67	−3.75
8. Demonstrating activity-related knowledge and techniques	0.52	0.10	1.37	4.05	1.34	3.27
24. Increasing utterance	0.42	0.10	0.64	−4.99	0.68	−3.83
16. Showing a smile	0.41	0.10	0.70	−4.00	0.70	−3.53
12. Expressing satisfaction as a result of the activity	0.28	0.10	0.82	−2.26	0.79	−2.39
18. Interacting with others through the activity	0.28	0.10	0.74	−3.39	0.79	−2.47
21. Conveying one’s intention to others	0.20	0.10	0.81	−2.39	0.85	−1.64
7. Concentrating on the activity	−0.79	0.10	0.62	−5.01	0.62	−4.93
3. Initiating the activity	−1.00	0.10	0.72	−3.56	0.71	−3.64
6. Continuing the activity	−1.03	0.10	0.62	−5.09	0.59	−5.49
5. Positioning one’s body toward the activity	−1.29	0.10	0.55	−6.15	0.53	−6.38
4. Directing one’s gaze at the activity	−1.35	0.10	0.55	−6.07	0.58	−5.57
15. No negative emotions during the activity	−3.06	0.12	1.86^a	7.83	1.75^a	5.89
11. No negative feelings after the activity	−3.70	0.13	1.92^a	7.87	1.61	3.88
1. No conditions to prevent participation in the activity	−4.57	0.15	1.71^a	5.42	4.46^a	9.50

Note. A-QOA = Assessment of Quality of Activities.

^aMean square met the criteria to remove items.

Table 4.

Rasch Analysis of the Final 21-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
20. Sharing activity-related knowledge and techniques with others	2.86	0.11	1.49	4.73	1.34	1.92
17. Showing excitement	2.75	0.11	0.86	−1.60	0.77	−1.54
25. Reminiscing	2.30	0.11	1.53	5.35	1.39	2.70
13. Expressing a sense of capability	2.22	0.11	0.87	−1.53	0.81	−1.58
22. Being considerate of others	1.76	0.10	1.06	0.76	1.20	1.72
10. Working out a way to make the activity progress smoothly	1.74	0.10	1.49	5.14	1.47	3.80
14. Showing willingness toward the next activity	1.67	0.10	0.89	−1.37	0.93	−0.66
19. Cooperating with others in the activity	1.65	0.10	1.27	3.08	1.24	2.09
9. Making selections or showing preferences during the activity	1.63	0.10	1.29	3.21	1.27	2.41
23. Sharing activity-induced emotions with others	1.49	0.10	0.76	−3.18	0.72	−3.00
8. Demonstrating activity-related knowledge and techniques	1.23	0.10	1.50	5.30	1.46	4.14
24. Increasing utterance	1.12	0.10	0.70	−3.97	0.73	−3.09
16. Showing a smile	1.11	0.10	0.81	−2.37	0.83	−1.87
12. Expressing satisfaction as a result of the activity	0.96	0.10	0.93	−0.79	0.92	−0.85
18. Interacting with others through the activity	0.96	0.10	0.80	−2.51	0.86	−1.50
21. Conveying one’s intention to others	0.88	0.10	0.88	−1.45	0.92	−0.81
7. Concentrating on the activity	−0.26	0.11	0.76	−2.92	0.76	−2.72
3. Initiating the activity	−0.52	0.11	0.85	−1.65	0.85	−1.66
6. Continuing the activity	−0.54	0.11	0.74	−3.10	0.69	−3.60
5. Positioning one’s body toward the activity	−0.85	0.11	0.63	−4.56	0.59	−4.97
4. Directing one’s gaze at the activity	−0.93	0.11	0.64	−4.40	0.66	−4.03

Note. A-QOA = Assessment of Quality of Activities.

View Large

Table 4.

Rasch Analysis of the Final 21-Item A-QOA

Item	Difficulty	SE	Infit		Outfit
Item	Difficulty	SE	MnSq	z	MnSq	z
20. Sharing activity-related knowledge and techniques with others	2.86	0.11	1.49	4.73	1.34	1.92
17. Showing excitement	2.75	0.11	0.86	−1.60	0.77	−1.54
25. Reminiscing	2.30	0.11	1.53	5.35	1.39	2.70
13. Expressing a sense of capability	2.22	0.11	0.87	−1.53	0.81	−1.58
22. Being considerate of others	1.76	0.10	1.06	0.76	1.20	1.72
10. Working out a way to make the activity progress smoothly	1.74	0.10	1.49	5.14	1.47	3.80
14. Showing willingness toward the next activity	1.67	0.10	0.89	−1.37	0.93	−0.66
19. Cooperating with others in the activity	1.65	0.10	1.27	3.08	1.24	2.09
9. Making selections or showing preferences during the activity	1.63	0.10	1.29	3.21	1.27	2.41
23. Sharing activity-induced emotions with others	1.49	0.10	0.76	−3.18	0.72	−3.00
8. Demonstrating activity-related knowledge and techniques	1.23	0.10	1.50	5.30	1.46	4.14
24. Increasing utterance	1.12	0.10	0.70	−3.97	0.73	−3.09
16. Showing a smile	1.11	0.10	0.81	−2.37	0.83	−1.87
12. Expressing satisfaction as a result of the activity	0.96	0.10	0.93	−0.79	0.92	−0.85
18. Interacting with others through the activity	0.96	0.10	0.80	−2.51	0.86	−1.50
21. Conveying one’s intention to others	0.88	0.10	0.88	−1.45	0.92	−0.81
7. Concentrating on the activity	−0.26	0.11	0.76	−2.92	0.76	−2.72
3. Initiating the activity	−0.52	0.11	0.85	−1.65	0.85	−1.66
6. Continuing the activity	−0.54	0.11	0.74	−3.10	0.69	−3.60
5. Positioning one’s body toward the activity	−0.85	0.11	0.63	−4.56	0.59	−4.97
4. Directing one’s gaze at the activity	−0.93	0.11	0.64	−4.40	0.66	−4.03

Note. A-QOA = Assessment of Quality of Activities.