Development of Additional Tasks for the Executive Function Performance Test

Bridget Hahn; Carolyn Baum; Jennifer Moore; Linda Ehrlich-Jones; Susan Spoeri; Meghan Doherty; Timothy J. Wolf

doi:10.5014/ajot.2014.008565

Free

Brief Report | November 2014

Development of Additional Tasks for the Executive Function Performance Test

Bridget Hahn; Carolyn Baum; Jennifer Moore; Linda Ehrlich-Jones; Susan Spoeri; Meghan Doherty; Timothy J. Wolf

Author Affiliations

Bridget Hahn, MS, OTR/L, is Clinical Occupational Therapist, Day Rehabilitation Program, Rehabilitation Institute of Chicago, 307 West Grand Avenue, Chicago, IL 60654; bhahn@ric.org
Carolyn Baum, PhD, OTR/L, FAOTA, is Professor, Departments of Occupational Therapy and Neurology, and Elias Michael Director, Program in Occupational Therapy, Washington University School of Medicine, St. Louis, MO
Jennifer Moore, PT, DHS, NCS, is Clinical Practice Leader, Rehabilitation Institute of Chicago, Chicago, IL
Linda Ehrlich-Jones, PhD, RN, is Research Scientist, Center for Rehabilitation Outcomes Research, Rehabilitation Institute of Chicago, Chicago, IL
Susan Spoeri, MS, OTR/L, is Education Program Manager, Rehabilitation Institute of Chicago Academy, and Clinical Occupational Therapist, Rehabilitation Institute of Chicago, Chicago, IL
Meghan Doherty, OTR/L, MSOT, is Research Coordinator, Performance, Participation, and Neurorehabilitation Laboratory, Program in Occupational Therapy, Washington University School of Medicine, St. Louis, MO. At the time of the study, she was Clinical Occupational Therapist, Rehabilitation Institute of Chicago, Chicago, IL
Timothy J. Wolf, OTD, MSCI, OTR/L, is Assistant Professor, Department of Neurology, Program in Occupational Therapy, Washington University School of Medicine, St. Louis, MO

Article Information

American Journal of Occupational Therapy, November/December 2014, Vol. 68, e241-e246. doi:10.5014/ajot.2014.008565

Abstract

OBJECTIVE. The Executive Function Performance Test (EFPT) is a reliable and valid performance-based assessment of executive function for people with stroke. The objective of this study was to enhance the clinical utility of the EFPT by developing and testing additional tasks for the EFPT in the Alternate EFPT (aEFPT).

METHOD. We performed a cross-sectional study with poststroke participants (n = 25) and healthy control participants (n = 25). All participants completed a neuropsychological assessment battery and both the EFPT and the aEFPT.

RESULTS. No statistically significant differences were found between the EFPT and the aEFPT when examining total scores, construct scores, and two overall task scores. Correlations between the aEFPT and the neuropsychological measures were adequate to strong (r^2s = .59–.83).

CONCLUSION. The aEFPT tasks are comparable to the original EFPT tasks, providing occupational therapy practitioners with additional tasks that can be used clinically to identify performance-based executive function deficits in people with stroke.

Executive functions are necessary to successfully complete instrumental activities of daily living (IADLs; e.g., grocery shopping, public transportation use, work- or school-related tasks) and are essential for completion of goal-directed behavior. Components of executive functions include initiation, sequencing, the ability to ignore irrelevant stimuli, identification and achievement of goals, and working memory (Connor & Maeir, 2011). Deficits in executive functions have been demonstrated in people with a wide range of diagnoses, including brain injury, stroke, multiple sclerosis, chronic heart failure, and Parkinson’s disease (Ferreira, 2010; Konrad et al., 2011; McKinlay, Grace, Dalrymple-Alford, & Roger, 2010; Nys et al., 2007; Pressler, 2008). These deficits are correlated with decreased performance in both activities of daily living and IADLs (Diamond, Felsenthal, Macciocchi, Butler, & Lally-Cassady, 1996; McKinlay et al., 2010; Tatemichi et al., 1994)

Seventy-four percent of people with stroke demonstrate acute cognitive impairment, with disorders in executive functioning being most common (Nys et al., 2007). Stroke survivors with cognitive impairments have greater functional impairment and a higher frequency of discharge to a dependent living environment in comparison to survivors without cognitive impairments (Diamond et al., 1996; Leśniak, Bak, Czepiel, Seniów, & Członkowska, 2008; Tatemichi et al., 1994). Even people with mild neurological impairment resulting from stroke (i.e., National Institutes of Health Stroke Scale score of <6) have reported decreased participation in IADLs, despite rarely being referred for rehabilitation services (Edwards, Hahn, Baum, & Dromerick, 2006; Hildebrand, Brewer, & Wolf, 2012; O’Brien & Wolf, 2010; Rochette, Desrosiers, Bravo, St-Cyr-Tribble, & Bourget, 2007; Wolf, Baum, & Conner, 2009).

Occupational therapy practitioners help people with executive function deficits after stroke improve their ability to perform and participate in their chosen occupations through the use of learned strategies associated with tasks that have meaning to each person. They are skilled in treating people with executive dysfunction using interventions such as metacognitive strategy training (including self-awareness training, goal management training, and cognitive orientation to daily occupational performance), functional task training, and environmental adaptations (Cicerone et al., 2011; Connor & Maeir, 2011; Goverover, Johnston, Toglia, & Deluca, 2007; Henshaw, Polatajko, McEwen, Ryan, & Baum, 2011; McEwen, Polatajko, Huijbregts, & Ryan, 2010).

Before occupational therapy practitioners can intervene with clients who have poststroke executive dysfunction, they need valid and reliable tools to identify the specific impairments that can limit performance in everyday life activities. Traditionally, neuropsychological assessments have been considered the gold standard for the identification of executive function deficits; however, the health care community has long known that people can perform within normal limits on neuropsychological assessments and still have performance-based impairments in activities that require executive function (Burgess et al., 2006; Shallice & Burgess, 1991). This discrepancy is a result of limitations in traditional pencil-and-paper assessments and contradicts current definitions of the components of executive function (Lezak, 1982). Therefore, it is critical for practitioners to use performance-based assessments of executive function with people with neurological injury.

An example of an established performance-based measure of executive function is the Executive Function Performance Test (EFPT; Baum & Wolf, 2013). The EFPT is easily administered and available in the public domain (Baum & Wolf, 2013). However, only one form of each EFPT task is currently available, which limits its use as an outcome measure because a potential learning effect may occur after administration.

This pilot study to develop the Alternate EFPT (aEFPT) had four objectives: (1) to develop additional tasks to complement those currently part of the EFPT, (2) to evaluate the alternate form reliability of the aEFPT, (3) to evaluate the discriminant validity of the aEFPT, and (4) to evaluate the concurrent validity of the aEFPT compared with the EFPT and standardized neuropsychological measures of executive function.

Method

Research Design

This cross-sectional study included poststroke participants (n = 25) and healthy control participants (n = 25). One of the three authors who were occupational therapists at the Rehabilitation Institute of Chicago obtained the written informed consent and administered all study screens and measures. All participants completed the EFPT and the aEFPT in two sessions that were approximately 1 wk apart. During the initial visit, we obtained written informed consent from all potential participants and then screened participants for inclusion. After it was determined that a person met the study criteria, we obtained demographic information, and the participant completed a short neuropsychological assessment battery. After completing the neuropsychological battery, participants were administered either the EFPT or the aEFPT on the basis of an a priori randomization sequence. One week after the initial testing session, we administered the form of the assessment (either the EFPT or the aEFPT) not administered during the initial visit. A local university institutional review board reviewed and approved the study.

Participants

We recruited people with stroke through a local stroke registry that includes information about approximately 500 people with stroke who consented to being approached about participation in research studies. In addition, we recruited participants with stroke and healthy control participants using flyers posted at the Rehabilitation Institute of Chicago’s day rehabilitation, inpatient, and outpatient sites.

Inclusion criteria included diagnosis of unilateral stroke at least 6 mo before study enrollment (poststroke participants only), ability to speak English, and ability to provide informed consent. Exclusion criteria were evidence of dementia on study enrollment as determined by a score of >12 on the Short Blessed Test (Katzman et al., 1983); severe depressive symptoms reflected by a score of >16 on the Center for Epidemiologic Studies–Depression Scale (Radloff, 1977); and current participation in any rehabilitation services addressing cognitive dysfunction, including any occupational or speech therapy.

Measures

All raters completed training on all the measures and were deemed proficient. Administrators were in contact with EFPT developers, and before study enrollment, all raters reviewed multiple videotapes of test administrations, scored the clients separately, and compared their scores to ensure consistency among raters.

Neuropsychological Battery.

The Trail Making Test Parts A and B (Trails A and B; Reitan & Wolfson, 1995) and the Wechsler Memory Scale–Revised Digit Span Forward and Backward Tests (Wechsler, 1987) were used to establish concurrent validity with performance-based assessments. Both Digit Span Tests were administered; however, we used data only from the Backward Test because of its increased complexity.

Executive Function Performance Test.

The EFPT evaluates five executive function constructs—(1) initiation, (2) organization, (3) sequencing, (4) safety and judgment, and (5) completion—as the client completes the four tasks of (1) cooking, (2) telephone use, (3) medication management, and (4) money management. The assessment is scored using a graded cueing system for each of the five constructs and is designed to measure the level of assistance the client needs to be successful with the task. The maximum score after all four tasks are administered is 100 points (25 points for each of the four tasks). The higher the score, the higher the degree of assistance required.

The EFPT has established interrater reliability (ICC = .91 for total score); internal consistency (α = .94); and validity with gold standards Trails B, Digit Span Backward Test and Story Recall from the Wechsler Memory Scale–Revised, Animal Fluency, and Short Blessed Test for the stroke population (all ps < .05; Baum et al., 2008). Reliability and validity of the EFPT have additionally been established in people with multiple sclerosis and schizophrenia (Goverover et al., 2005; Katz, Tadmore, Felzen, & Hartman-Maeir, 2007). Detailed instructions on how to administer and score the EFPT can be found in the EFPT manual available online (Baum & Wolf, 2013).

Alternate Executive Function Performance Test.

The aEFPT was developed to complement the original EFPT by providing additional tasks. The aEFPT tasks were developed by the authors of this article who were occupational therapists at the Rehabilitation Institute of Chicago (Hahn, Spoeri, & Doherty) in collaboration with Carolyn Baum, one of the original authors of the EFPT. The four additional tasks were designed to be similar to the four original tasks but with a novel component to prevent a learning effect from the original form. The additional tasks are all activities within the four original task categories (cooking, telephone use, medication management, and money management), which have been identified as essential skills for independent living (Lysack, Neufeld, Mast, MacNeill, & Lichtenberg, 2003). The aEFPT cooking task involves making pasta instead of oatmeal, as in the EFPT. The telephone use task involves calling a doctor’s office instead of a grocery store. Medication management involves sorting medications into a 7-day pill sorter instead of taking a medication. The money management task involves ordering a specific item from a catalog instead of paying two bills. Administration of the additional tasks is completed using the original EFPT structure. All tasks are scored using the same graded cueing system examining five constructs: (1) initiation, (2) organization, (3) sequencing, (4) safety and judgment, and (5) completion. Time taken to complete each task is recorded.

Data Analysis

All data analysis was completed with IBM SPSS for Windows (Version 19; IBM Corporation, Armonk, NY). Nonparametric statistical analyses were used because of the distribution and nature of the data. The Mann–Whitney U test and χ² analysis were used to test for demographic differences between the control and poststroke participants. To evaluate the discriminant validity of the aEFPT, the Mann–Whitney U test was used to examine differences between groups on the EFPT and aEFPT. To evaluate the alternate form reliability of the aEFPT, the Wilcoxon signed-rank test was used to examine differences between EFPT and aEFPT scores from the poststroke group. To evaluate the concurrent validity of the aEFPT, Spearman’s rank order coefficients were calculated between scores on the aEFPT, the EFPT, and the neuropsychological measures (α was set at .05).

Results

Sixty-one participants were recruited. One healthy control participant and 7 poststroke participants were screened out because of depression, and 1 poststroke participant was excluded because of dementia. One participant from each group dropped out because of scheduling conflicts. In total, 50 participants, 25 in each group, completed the study.

Demographic characteristics of the study sample are presented in Table 1. Significant differences were found in race (Z = –2.52, p = .01), with poststroke participants being more diverse and having a higher percentage of African-Americans and Asian-Americans than the control group. The control group had a higher percentage of White and Hispanic participants than the poststroke group. Gender distribution was also different between the two groups, with the control group having a larger percentage of women. No significant difference between groups was found in age or education.

Table 1.

Demographic Characteristics, by Group

	Group
Characteristic	Control, n (%) or M (SD)	Poststroke, n (%) or M (SD)	Difference Between Groups, Z (p)
Race			−2.52 (.01)*
African-American	5 (20)	14 (56)
White	18 (72)	8 (32)
Hispanic	2 (8)	1 (4)
Asian-American	0 (0)	2 (8)
Gender			−2.33 (.02)*
Male	5 (20)	13 (52)
Female	20 (80)	12 (48)
Age and education
Age, yr	56.96 (7.13)	57.00 (8.97)	–0.38 (.70)
Education, yr	15.72 (2.92)	14.16 (2.44)	−1.80 (.07)

Note. n = 25 in each group. M = mean; SD = standard deviation.

*p < .05.

View Large

Table 1.

Demographic Characteristics, by Group

	Group
Characteristic	Control, n (%) or M (SD)	Poststroke, n (%) or M (SD)	Difference Between Groups, Z (p)
Race			−2.52 (.01)*
African-American	5 (20)	14 (56)
White	18 (72)	8 (32)
Hispanic	2 (8)	1 (4)
Asian-American	0 (0)	2 (8)
Gender			−2.33 (.02)*
Male	5 (20)	13 (52)
Female	20 (80)	12 (48)
Age and education
Age, yr	56.96 (7.13)	57.00 (8.97)	–0.38 (.70)
Education, yr	15.72 (2.92)	14.16 (2.44)	−1.80 (.07)

Note. n = 25 in each group. M = mean; SD = standard deviation.

*p < .05.

Discriminant Validity

Task scores were significantly different between the control group and the poststroke group on all tasks for both the EFPT and the aEFPT (Table 2). Construct scores and completion times for the healthy control group and the poststroke group for both performance-based measures are listed in Table 3. Total aEFPT scores (Z = –5.28, p = .00) and times (Z = –5.38, p = .00) were significantly different between the control and poststroke groups. Total EFPT scores (Z = –5.44, p = .00) and times (Z = –5.07, p = .00) were also significantly different between the groups.

Table 2.

Task Scores, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT tasks
Cooking	0.48 (0.59)	2.56 (2.33)	−4.58 (.00)*
Telephone use	0.04 (0.20)	1.16 (1.31)	−4.00 (.00)*
Medication management	0.24 (0.52)	0.80 (1.12)	−2.16 (.03)*
Money management	0.88 (0.60)	2.92 (3.05)	−3.53 (.00)*
aEFPT tasks
Cooking	0.44 (0.58)	2.92 (2.33)	−4.47 (.00)*
Telephone use	0.00 (0.00)	0.60 (1.04)	−3.04 (.00)*
Medication management	0.56 (0.77)	2.08 (1.17)	−3.61 (.00)*
Money management	0.24 (0.52)	2.76 (1.69)	−5.24 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

View Large

Table 2.

Task Scores, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT tasks
Cooking	0.48 (0.59)	2.56 (2.33)	−4.58 (.00)*
Telephone use	0.04 (0.20)	1.16 (1.31)	−4.00 (.00)*
Medication management	0.24 (0.52)	0.80 (1.12)	−2.16 (.03)*
Money management	0.88 (0.60)	2.92 (3.05)	−3.53 (.00)*
aEFPT tasks
Cooking	0.44 (0.58)	2.92 (2.33)	−4.47 (.00)*
Telephone use	0.00 (0.00)	0.60 (1.04)	−3.04 (.00)*
Medication management	0.56 (0.77)	2.08 (1.17)	−3.61 (.00)*
Money management	0.24 (0.52)	2.76 (1.69)	−5.24 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

Table 3.

Construct Scores and Completion Times, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT constructs
Initiation	0.00 (0.00)	0.00 (0.00)	0.00 (1.00)
Organization	0.04 (0.20)	1.04 (1.81)	−3.10 (.00)*
Sequencing	0.60 (0.71)	3.44 (2.63)	−4.69 (.00)*
Safety and judgment	0.96 (0.79)	2.48 (1.80)	−3.40 (.00)*
Completion	0.04 (0.79)	0.48 (1.36)	−1.45 (.15)
EFPT total score	1.64 (1.04)	7.44 (5.47)	−5.44 (.00)*
EFPT time, min	22.41 (4.32)	35.08 (10.05)	−5.07 (.00)*
aEFPT constructs
Initiation	0.04 (0.20)	0.08 (0.28)	−0.59 (.56)
Organization	0.12 (0.33)	0.52 (0.71)	−2.33 (.02)*
Sequencing	0.68 (0.85)	4.20 (3.58)	−4.51 (.00)*
Safety and judgment	0.28 (0.46)	3.16 (1.86)	−5.39 (.00)*
Completion	0.08 (0.28)	0.36 (0.70)	−1.61 (.10)
aEFPT total score	1.24 (1.05)	8.36 (5.35)	−5.28 (.00)*
aEFPT time, min	28.07 (5.13)	47.19 (11.78)	−5.38 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

View Large

Table 3.

Construct Scores and Completion Times, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT constructs
Initiation	0.00 (0.00)	0.00 (0.00)	0.00 (1.00)
Organization	0.04 (0.20)	1.04 (1.81)	−3.10 (.00)*
Sequencing	0.60 (0.71)	3.44 (2.63)	−4.69 (.00)*
Safety and judgment	0.96 (0.79)	2.48 (1.80)	−3.40 (.00)*
Completion	0.04 (0.79)	0.48 (1.36)	−1.45 (.15)
EFPT total score	1.64 (1.04)	7.44 (5.47)	−5.44 (.00)*
EFPT time, min	22.41 (4.32)	35.08 (10.05)	−5.07 (.00)*
aEFPT constructs
Initiation	0.04 (0.20)	0.08 (0.28)	−0.59 (.56)
Organization	0.12 (0.33)	0.52 (0.71)	−2.33 (.02)*
Sequencing	0.68 (0.85)	4.20 (3.58)	−4.51 (.00)*
Safety and judgment	0.28 (0.46)	3.16 (1.86)	−5.39 (.00)*
Completion	0.08 (0.28)	0.36 (0.70)	−1.61 (.10)
aEFPT total score	1.24 (1.05)	8.36 (5.35)	−5.28 (.00)*
aEFPT time, min	28.07 (5.13)	47.19 (11.78)	−5.38 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

Alternate Form Reliability

Wilcoxon signed-rank test was computed from poststroke participants’ scores on the EFPT compared with the aEFPT to establish alternate form reliability (Table 4). We examined scores for each task and each construct, as well as total score and total time. Total scores were not different between the two tests (Z = –0.768, p = .442). Similarly, construct scores were not different between the two tests. Total time was significantly longer on the aEFPT (Z = 3.579, p = .000). Two tasks, cooking (Z = –0.959, p = .337) and money management (Z = –0.262, p = .793), had no significant differences between tests. However, telephone use (Z = –2.074, p = .038) and medication management (Z = –2.495, p = .013) had significant differences between tests.

Table 4.

Alternate Form Reliability of the aEFPT Compared With the EFPT

Measure	Wilcoxon Signed-Rank Test
Tasks
Cooking	−0.959 (.337)
Telephone use	−2.074 (.038)*
Medication management	−2.495 (.013)*
Money management	−0.262 (.793)
Constructs
Initiation	−1.414 (.157)
Organization	−1.216 (.224)
Sequencing	−1.757 (.079)
Safety and judgment	−1.305 (.192)
Completion	−0.052 (.959)
Total scores	−0.768 (.442)
Total time, min	−3.579 (.000)*

Note. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test.

*p < .05.

View Large

Table 4.

Alternate Form Reliability of the aEFPT Compared With the EFPT

Measure	Wilcoxon Signed-Rank Test
Tasks
Cooking	−0.959 (.337)
Telephone use	−2.074 (.038)*
Medication management	−2.495 (.013)*
Money management	−0.262 (.793)
Constructs
Initiation	−1.414 (.157)
Organization	−1.216 (.224)
Sequencing	−1.757 (.079)
Safety and judgment	−1.305 (.192)
Completion	−0.052 (.959)
Total scores	−0.768 (.442)
Total time, min	−3.579 (.000)*

Note. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test.

*p < .05.

Concurrent Validity

Spearman’s rank order correlation coefficient was computed for concurrent validity testing using only scores from poststroke participants. Significant correlations (p < .01) existed between the aEFPT and Trails A (r² = .53), Trails B (r² = .55), Short Blessed Test (r² = .69), and EFPT (r² = .63). The correlation between the aEFPT and the Digit Span Backward Test (r² = –.38) was not significant.

Discussion

The aim of this study was to develop additional tasks for the EFPT and investigate the psychometric properties in poststroke and healthy control participants. The statistical objectives of this pilot study were to evaluate the alternate form reliability of the aEFPT, the discriminant validity of the aEFPT, and the concurrent validity of the aEFPT compared with the EFPT and standardized neuropsychological measures of executive function.

The total scores on the EFPT and the aEFPT were not significantly different, indicating that the two instruments can be used at two separate points in time to measure an outcome. However, scores on the telephone use and medication management tasks were significantly different between the two instruments. This difference could be attributed to the lower mean test scores and low variability on both these items; for example, 14 participants had a score of zero on the telephone use task, indicating that they required no assistance to complete the task. Another factor contributing to differences in the medication management task could be the increased complexity of the alternate medication management task (i.e., sorting two medications into a pill sorter) compared with the original version (i.e., simply taking a medication). Future research could further study the telephone use and medication management tasks with a larger sample size or a population with more severe impairments to clarify the relationship with the other tasks. Our data thus indicate that the two instruments should be administered in their entirety to provide comparable scores for an outcome. Additionally, the aEFPT tasks of cooking and money management could be used to measure change from the EFPT.

Comparisons between poststroke and control group scores demonstrated a significant difference between the groups on task scores, total scores, and time on both the EFPT and aEFPT. These findings indicate that the EFPT and the aEFPT are sensitive to detecting executive function deficits in the stroke population and are consistent with published research on the EFPT (Baum et al., 2008).

Initiation and completion were the only constructs not significantly different between groups (Table 3). This finding could be the result of poststroke participants having fewer deficits in initiation and completion than in the other constructs, with mean scores ranging from .00 to .36. These findings are consistent with previous research in which Baum and colleagues (2008) found that participants with mild stroke required little assistance with initiation and completion (mean scores were 0.83 and 0.88, respectively).

Task scores for poststroke participants (see Table 2) fell within the “mild stroke” category according to Baum and colleagues (2008), whose participants with mild stroke had mean task scores ranging from 0.92 to 2.98. In the current study, poststroke participants’ mean scores ranged from 0.80 to 2.92 on the EFPT and 0.60 to 2.92 on the aEFPT. The majority of participants in our study had mild executive function deficits, resulting in low variability in scores, particularly with the initiation and completion constructs and the telephone use and medication tasks.

Results of validity testing indicated that the aEFPT significantly correlated with all neuropsychological measures, with the exception of the Digit Span Backward Test. These results are consistent with EFPT validity established by Baum and colleagues (2008) and suggest that these performance-based measures correlate with gold standards for assessing executive functions.

Study limitations include the small sample size and the classification of the majority of the poststroke participants as having mild stroke, which could limit generalizability. Additionally, the small sample size limited our ability to complete a more advanced statistical procedure (i.e., multivariate analysis of covariance). Future research with a larger sample size is warranted to ensure that the differences between healthy control and poststroke participants’ test scores could not be attributed to any of the demographic characteristics. Average years of education was relatively high for all participants, which may have affected test scores; poststroke participants had an average of 14 yr of total schooling and control participants an average of 16 yr.

The increased time participants needed to complete the aEFPT could limit the clinical utility of the additional tasks if they are used collectively. The majority of the additional time was attributed to the money management task. However, the money management task in the aEFPT was generally more complex than the original money management task and therefore would provide additional information beyond what may be gained from the original money management task.

Suggestions for Future Research

Future research should focus on whether the aEFPT would identify executive function impairments in a poststroke population that is more heterogeneous. Further testing and development of the medication management and telephone use tasks should be completed to determine their relationship with the other tasks. Further research to revise the money management portion of the aEFPT and decrease the amount of time required to complete the aEFPT would enhance its clinical utility. Finally, additional tasks for the EFPT should be developed to further establish the clinical utility of this performance-based measure.

Implications for Occupational Therapy Practice

The results of this pilot study have the following implications for occupational therapy practice:

The aEFPT offers a promising way for clinicians to measure an outcome of clinical treatment of executive function deficits.
If administered in its entirety, the aEFPT is not significantly different than the EFPT and is able to detect change in deficits in executive function in the poststroke population.
The tasks of making pasta and ordering from a catalog could be used to measure change after administration of the original EFPT.
The aEFPT is a valid tool to capture performance-based executive function deficits in the poststroke population.

Acknowledgments

This study was carried out with the financial support of The Buchanan Family Fellowship through the Rehabilitation Institute of Chicago (RIC). We additionally thank Jason Raad for his statistical consultation and Donna Zahara, director, RIC Academy, Internal Staff Development, for her continued support of this project.

References

Baum, C. M., Connor, L. T., Morrison, T., Hahn, M., Dromerick, A. W., & Edwards, D. F. (2008). Reliability, validity, and clinical utility of the Executive Function Performance Test: A measure of executive function in a sample of people with stroke. American Journal of Occupational Therapy, 62, 446–455. http://dx.doi.org/10.5014/ajot.62.4.446 [Article] [PubMed]

Baum, C., & Wolf, T. (2013). Executive Function Performance Test (EFPT). Retrieved from http://www.ot.wustl.edu/about/resources/executive-function-performance-test-efpt-308

Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates, L. M., Dawson, D. R., … Channon, S. (2006). The case for the development and use of “ecologically valid” measures of executive function in experimental and clinical neuropsychology. Journal of the International Neuropsychological Society, 12, 194–209. http://dx.doi.org/10.1017/S1355617706060310 [Article] [PubMed]

Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Fraas, M., … Ashman, T. (2011). Evidence-based cognitive rehabilitation: Updated review of the literature from 2003 through 2008. Archives of Physical Medicine and Rehabilitation, 92, 519–530. http://dx.doi.org/10.1016/j.apmr.2010.11.015 [Article] [PubMed]

Connor, L. T., & Maeir, A. (2011). Putting executive performance in a theoretical context. OTJR: Occupation, Participation and Health, 31, S3–S7. http://dx.doi.org/10.3928/15394492-20101108-02 [Article] [PubMed]

Diamond, P. T., Felsenthal, G., Macciocchi, S. N., Butler, D. H., & Lally-Cassady, D. (1996). Effect of cognitive impairment on rehabilitation outcome. American Journal of Physical Medicine and Rehabilitation, 75, 40–43. http://dx.doi.org/10.1097/00002060-199601000-00011 [Article] [PubMed]

Edwards, D. F., Hahn, M., Baum, C., & Dromerick, A. W. (2006). The impact of mild stroke on meaningful activity and life satisfaction. Journal of Stroke and Cerebrovascular Diseases, 15, 151–157. http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2006.04.001 [Article] [PubMed]

Ferreira, M. L. (2010). Cognitive deficits in multiple sclerosis: A systematic review. Arquivos de Neuro-Psiquiatria, 68, 632–641. http://dx.doi.org/10.1590/S0004-282X2010000400029 [Article] [PubMed]

Goverover, Y., Johnston, M. V., Toglia, J., & Deluca, J. (2007). Treatment to improve self-awareness in persons with acquired brain injury. Brain Injury, 21, 913–923. http://dx.doi.org/10.1080/02699050701553205 [Article] [PubMed]

Goverover, Y., Kalmar, J., Gaudino-Goering, E., Shawaryn, M., Moore, N. B., Halper, J., & DeLuca, J. (2005). The relation between subjective and objective measures of everyday life activities in persons with multiple sclerosis. Archives of Physical Medicine and Rehabilitation, 86, 2303–2308. http://dx.doi.org/10.1016/j.apmr.2005.05.016 [Article] [PubMed]

Henshaw, E., Polatajko, H., McEwen, S., Ryan, J. D., & Baum, C. M. (2011). Cognitive approach to improving participation after stroke: Two case studies. American Journal of Occupational Therapy, 65, 55–63. http://dx.doi.org/10.5014/ajot.2011.09010 [Article] [PubMed]

Hildebrand, M., Brewer, M., & Wolf, T. (2012). The impact of mild stroke on participation in physical fitness activities. Stroke Research and Treatment, 2012, 548682. http://dx.doi.org/10.1155/2012/548682

Katz, N., Tadmore, I., Felzen, B., & Hartman-Maeir, A. (2007). Validity of the Executive Function Performance Test in individuals with schizophrenia. OTJR: Occupation, Participation and Health, 27, 1–8.

Katzman, R., Brown, T., Fuld, P., Peck, A., Schechter, R., & Schimmel, H. (1983). Validation of a short Orientation–Memory–Concentration Test of cognitive impairment. American Journal of Psychiatry, 140, 734–739. [PubMed]

Konrad, C., Geburek, A. J., Rist, F., Blumenroth, H., Fischer, B., Husstedt, I., & Lohmann, H. (2011). Long-term cognitive and emotional consequences of mild traumatic brain injury. Psychological Medicine, 41, 1197–1211. [Article] [PubMed]

Leśniak, M., Bak, T., Czepiel, W., Seniów, J., & Członkowska, A. (2008). Frequency and prognostic value of cognitive disorders in stroke patients. Dementia and Geriatric Cognitive Disorders, 26, 356–363. http://dx.doi.org/10.1159/000162262 [Article] [PubMed]

Lezak, M. D. (1982). The problem of assessing executive functions. International Journal of Psychology, 17, 281–297. http://dx.doi.org/10.1080/00207598208247445 [Article]

Lysack, C. L., Neufeld, S., Mast, B. T., MacNeill, S. E., & Lichtenberg, P. A. (2003). After rehabilitation: An 18-month follow-up of elderly inner-city women. American Journal of Occupational Therapy, 57, 298–306. http://dx.doi.org/10.5014/ajot.57.3.298 [Article] [PubMed]

McEwen, S. E., Polatajko, H. J., Huijbregts, M. P., & Ryan, J. D. (2010). Inter-task transfer of meaningful, functional skills following a cognitive-based treatment: Results of three multiple baseline design experiments in adults with chronic stroke. Neuropsychological Rehabilitation, 20, 541–561. http://dx.doi.org/10.1080/09602011003638194 [Article] [PubMed]

McKinlay, A., Grace, R. C., Dalrymple-Alford, J. C., & Roger, D. (2010). Characteristics of executive function impairment in Parkinson’s disease patients without dementia. Journal of the International Neuropsychological Society, 16, 268–277. http://dx.doi.org/10.1017/S1355617709991299 [Article] [PubMed]

Nys, G. M., van Zandvoort, M. J., de Kort, P. L., Jansen, B. P., de Haan, E. H., & Kappelle, L. J. (2007). Cognitive disorders in acute stroke: Prevalence and clinical determinants. Cerebrovascular Diseases, 23, 408–416. http://dx.doi.org/10.1159/000101464 [Article] [PubMed]

O’Brien, A. N., & Wolf, T. J. (2010). Determining work outcomes in mild to moderate stroke survivors. Work, 36, 441–447. [PubMed]

Pressler, S. J. (2008). Cognitive functioning and chronic heart failure: A review of the literature (2002–July 2007). Journal of Cardiovascular Nursing, 23, 239–249. http://dx.doi.org/10.1097/01.JCN.0000305096.09710.ec [Article] [PubMed]

Radloff, S. (1977). The CES–D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. http://dx.doi.org/10.1177/014662167700100306 [Article]

Reitan, R. M., & Wolfson, D. (1995). Category Test and Trail Making Test as measures of frontal lobe functions. Clinical Neuropsychologist, 9, 50–56. http://dx.doi.org/10.1080/13854049508402057 [Article]

Rochette, A., Desrosiers, J., Bravo, G., St-Cyr-Tribble, D., & Bourget, A. (2007). Changes in participation after a mild stroke: Quantitative and qualitative perspectives. Topics in Stroke Rehabilitation, 14, 59–68. http://dx.doi.org/10.1310/tsr1403-59 [Article] [PubMed]

Shallice, T., & Burgess, P. W. (1991). Deficits in strategy application following frontal lobe damage in man. Brain, 114, 727–741. http://dx.doi.org/10.1093/brain/114.2.727 [Article] [PubMed]

Tatemichi, T. K., Desmond, D. W., Stern, Y., Paik, M., Sano, M., & Bagiella, E. (1994). Cognitive impairment after stroke: Frequency, patterns, and relationship to functional abilities. Journal of Neurology, Neurosurgery, and Psychiatry, 57, 202–207. http://dx.doi.org/10.1136/jnnp.57.2.202 [Article] [PubMed]

Wechsler, D. (1987). Wechsler Memory Scale–Revised. San Antonio, TX: Psychological Corporation.

Wolf, T. J., Baum, C., & Conner, L. T. (2009). Changing face of stroke: Implications for occupational therapy practice. American Journal of Occupational Therapy, 63, 621–625. http://dx.doi.org/10.5014/ajot.63.5.621 [Article] [PubMed]

Table 1.

Demographic Characteristics, by Group

	Group
Characteristic	Control, n (%) or M (SD)	Poststroke, n (%) or M (SD)	Difference Between Groups, Z (p)
Race			−2.52 (.01)*
African-American	5 (20)	14 (56)
White	18 (72)	8 (32)
Hispanic	2 (8)	1 (4)
Asian-American	0 (0)	2 (8)
Gender			−2.33 (.02)*
Male	5 (20)	13 (52)
Female	20 (80)	12 (48)
Age and education
Age, yr	56.96 (7.13)	57.00 (8.97)	–0.38 (.70)
Education, yr	15.72 (2.92)	14.16 (2.44)	−1.80 (.07)

Note. n = 25 in each group. M = mean; SD = standard deviation.

*p < .05.

View Large

Table 1.

Demographic Characteristics, by Group

	Group
Characteristic	Control, n (%) or M (SD)	Poststroke, n (%) or M (SD)	Difference Between Groups, Z (p)
Race			−2.52 (.01)*
African-American	5 (20)	14 (56)
White	18 (72)	8 (32)
Hispanic	2 (8)	1 (4)
Asian-American	0 (0)	2 (8)
Gender			−2.33 (.02)*
Male	5 (20)	13 (52)
Female	20 (80)	12 (48)
Age and education
Age, yr	56.96 (7.13)	57.00 (8.97)	–0.38 (.70)
Education, yr	15.72 (2.92)	14.16 (2.44)	−1.80 (.07)

Note. n = 25 in each group. M = mean; SD = standard deviation.

*p < .05.

Table 2.

Task Scores, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT tasks
Cooking	0.48 (0.59)	2.56 (2.33)	−4.58 (.00)*
Telephone use	0.04 (0.20)	1.16 (1.31)	−4.00 (.00)*
Medication management	0.24 (0.52)	0.80 (1.12)	−2.16 (.03)*
Money management	0.88 (0.60)	2.92 (3.05)	−3.53 (.00)*
aEFPT tasks
Cooking	0.44 (0.58)	2.92 (2.33)	−4.47 (.00)*
Telephone use	0.00 (0.00)	0.60 (1.04)	−3.04 (.00)*
Medication management	0.56 (0.77)	2.08 (1.17)	−3.61 (.00)*
Money management	0.24 (0.52)	2.76 (1.69)	−5.24 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

View Large

Table 2.

Task Scores, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT tasks
Cooking	0.48 (0.59)	2.56 (2.33)	−4.58 (.00)*
Telephone use	0.04 (0.20)	1.16 (1.31)	−4.00 (.00)*
Medication management	0.24 (0.52)	0.80 (1.12)	−2.16 (.03)*
Money management	0.88 (0.60)	2.92 (3.05)	−3.53 (.00)*
aEFPT tasks
Cooking	0.44 (0.58)	2.92 (2.33)	−4.47 (.00)*
Telephone use	0.00 (0.00)	0.60 (1.04)	−3.04 (.00)*
Medication management	0.56 (0.77)	2.08 (1.17)	−3.61 (.00)*
Money management	0.24 (0.52)	2.76 (1.69)	−5.24 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

Table 3.

Construct Scores and Completion Times, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT constructs
Initiation	0.00 (0.00)	0.00 (0.00)	0.00 (1.00)
Organization	0.04 (0.20)	1.04 (1.81)	−3.10 (.00)*
Sequencing	0.60 (0.71)	3.44 (2.63)	−4.69 (.00)*
Safety and judgment	0.96 (0.79)	2.48 (1.80)	−3.40 (.00)*
Completion	0.04 (0.79)	0.48 (1.36)	−1.45 (.15)
EFPT total score	1.64 (1.04)	7.44 (5.47)	−5.44 (.00)*
EFPT time, min	22.41 (4.32)	35.08 (10.05)	−5.07 (.00)*
aEFPT constructs
Initiation	0.04 (0.20)	0.08 (0.28)	−0.59 (.56)
Organization	0.12 (0.33)	0.52 (0.71)	−2.33 (.02)*
Sequencing	0.68 (0.85)	4.20 (3.58)	−4.51 (.00)*
Safety and judgment	0.28 (0.46)	3.16 (1.86)	−5.39 (.00)*
Completion	0.08 (0.28)	0.36 (0.70)	−1.61 (.10)
aEFPT total score	1.24 (1.05)	8.36 (5.35)	−5.28 (.00)*
aEFPT time, min	28.07 (5.13)	47.19 (11.78)	−5.38 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

View Large

Table 3.

Construct Scores and Completion Times, by Group

	Group
Test	Control, M (SD)	Poststroke, M (SD)	Difference Between Groups, Z (p)
EFPT constructs
Initiation	0.00 (0.00)	0.00 (0.00)	0.00 (1.00)
Organization	0.04 (0.20)	1.04 (1.81)	−3.10 (.00)*
Sequencing	0.60 (0.71)	3.44 (2.63)	−4.69 (.00)*
Safety and judgment	0.96 (0.79)	2.48 (1.80)	−3.40 (.00)*
Completion	0.04 (0.79)	0.48 (1.36)	−1.45 (.15)
EFPT total score	1.64 (1.04)	7.44 (5.47)	−5.44 (.00)*
EFPT time, min	22.41 (4.32)	35.08 (10.05)	−5.07 (.00)*
aEFPT constructs
Initiation	0.04 (0.20)	0.08 (0.28)	−0.59 (.56)
Organization	0.12 (0.33)	0.52 (0.71)	−2.33 (.02)*
Sequencing	0.68 (0.85)	4.20 (3.58)	−4.51 (.00)*
Safety and judgment	0.28 (0.46)	3.16 (1.86)	−5.39 (.00)*
Completion	0.08 (0.28)	0.36 (0.70)	−1.61 (.10)
aEFPT total score	1.24 (1.05)	8.36 (5.35)	−5.28 (.00)*
aEFPT time, min	28.07 (5.13)	47.19 (11.78)	−5.38 (.00)*

Note. n = 25 in each group. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test; M = mean; SD = standard deviation.

*p < .05.

Table 4.

Alternate Form Reliability of the aEFPT Compared With the EFPT

Measure	Wilcoxon Signed-Rank Test
Tasks
Cooking	−0.959 (.337)
Telephone use	−2.074 (.038)*
Medication management	−2.495 (.013)*
Money management	−0.262 (.793)
Constructs
Initiation	−1.414 (.157)
Organization	−1.216 (.224)
Sequencing	−1.757 (.079)
Safety and judgment	−1.305 (.192)
Completion	−0.052 (.959)
Total scores	−0.768 (.442)
Total time, min	−3.579 (.000)*

Note. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test.

*p < .05.

View Large

Table 4.

Alternate Form Reliability of the aEFPT Compared With the EFPT

Measure	Wilcoxon Signed-Rank Test
Tasks
Cooking	−0.959 (.337)
Telephone use	−2.074 (.038)*
Medication management	−2.495 (.013)*
Money management	−0.262 (.793)
Constructs
Initiation	−1.414 (.157)
Organization	−1.216 (.224)
Sequencing	−1.757 (.079)
Safety and judgment	−1.305 (.192)
Completion	−0.052 (.959)
Total scores	−0.768 (.442)
Total time, min	−3.579 (.000)*

Note. aEFPT = Alternate Executive Performance Test; EFPT = Executive Function Performance Test.

*p < .05.