DVD Training for Depression Identification and Treatment in Older Adults: A Two-Group, Randomized, Wait-List Control Study

Cathy Lysack; Carrie Leach; Theresa Russo; Daniel Paulson; Peter A. Lichtenberg

doi:10.5014/ajot.2013.008060

Free

Research Article | September 2013

DVD Training for Depression Identification and Treatment in Older Adults: A Two-Group, Randomized, Wait-List Control Study

Cathy Lysack; Carrie Leach; Theresa Russo; Daniel Paulson; Peter A. Lichtenberg

Author Affiliations

Article Information

American Journal of Occupational Therapy, September/October 2013, Vol. 67, 584-593. doi:10.5014/ajot.2013.008060

Abstract

OBJECTIVE. To test the effectiveness of an educational intervention aimed at improving mental health knowledge and skills in occupational therapists working with older rehabilitation patients.

METHOD. The DVD-format educational intervention was evaluated using a two-group randomized wait-list control design. Occupational therapists (n = 75) completed a 32-item knowledge questionnaire at three time points. Patient charts were reviewed (n = 960) at 3 months before and 3 and 6 months after DVD training to evaluate clinical practice change.

RESULTS. A two-way analysis of variance showed knowledge scores increased significantly for both groups after DVD training. A significant Group × Time interaction and significant main effects for time and group were found. Chart review data also showed significant increases in desired clinical behaviors in both groups after training. The greatest single item of clinical practice change was use of a standardized depression screen.

CONCLUSION. DVD-based training can significantly improve mental health practice.

Occupational therapists are trained in mental health assessment and intervention, but their skills are underutilized in clinical practice with older adults in rehabilitation (Lysack, Lichtenberg, & Schneider, 2011). The explanations for this underuse are varied and include the complexity of geriatric care; therapists’ roles, which tend to emphasize activities of daily living (ADLs) and home and environmental adaptation (Alexopoulos & Bruce, 2009); and shorter patient lengths of stay. Yet mood disorders, especially depression (the most prevalent mood disorder), are more common and have greater consequences than many health care professionals realize. An influential report from the U.S. Surgeon General found that depression may be as high as 25%–35% in geriatric rehabilitation (U.S. Department of Health and Human Services, 1999). A large study of older adults admitted to two community-based long-term care programs in Michigan showed that 24.5% had probable depression (Li & Conwell, 2007). The consequences of untreated depression are serious, including increased risk of disability, hospitalization, institutionalization, and even death (Anderson, 2001; Li & Conwell, 2007; Sherlock, 2005). Of added concern is the fact that depression is especially underrecognized and undertreated in older adults (Anderson, 2001; Brown, McAvay, Raue, Moses, & Bruce, 2003; Brown et al., 2004; Bruce et al., 2002; Ruchinskas, 2002).

The issue of depression gains urgency in the context of the rapidly aging population. According to census data, in 2010 40 million people aged 65 and older lived in the United States, accounting for 13% of the total population; however, by 2030, this group will grow to 72 million, or nearly 20% of the total population (U.S. Census Bureau, 2012). Experts have argued that new models will be needed to meet their health care needs. One recommendation from the influential Institute of Medicine (IOM) report Retooling for an Aging America (2008) is to expand the role of health care workers.

Occupational therapists are well positioned to expand their role in mental health with older adults. First, occupational therapy’s focus on meaningful occupation as essential to health and well-being is fully consonant with the scientific evidence showing that behavioral activation techniques, including the reintegration of pleasant events into the daily activities of adults with depression, are an effective means of depression treatment (Frederick et al., 2007; Lichtenberg, Kimbarow, Morris, & Vangel, 1996; Quijano et al., 2007; Steinman et al., 2007). Second, evidence has shown that occupational therapists can have a positive impact on mental health outcomes of older patients (Clark et al., 2012; Matuska, Giles-Heinz, Flinn, Neighbor, & Bass-Haugen, 2003), including those with mild to moderate cognitive impairment (Graff et al., 2007). These findings are important because the Centers for Disease Control and Prevention (2011) has reported that the prevalence of depression is now 10%. Taken together, these findings have suggested that mental health interventions based on cognitive–behavioral therapy should be provided by more occupational therapists in more settings to reach those with unmet needs.

In earlier research, our team developed a set of depression education materials in DVD format and evaluated three of the six DVDs using a pretest–posttest research design in a sample of 30 occupational therapists (Lysack et al., 2011). Study results showed that therapists’ knowledge improved significantly on 5 of 11 knowledge and attitude items. Results from the chart audit component of that study showed that therapists changed their practice behaviors after DVD training; most notably, they increased screening of patients for depression and communication of findings to the treatment team and incorporated more pleasant events into the daily routines of their patients with depression. As a result of this work, the American Occupational Therapy Association (AOTA) adopted the DVDs as an official AOTA continuing education (CE) product (http://myaota.aota.org/shop_aota/prodview.aspx?TYPE=D&SKU=4859).

The purpose of the study reported here was to build on this work and test the effectiveness of the complete set of six DVDs using a two-group randomized wait-list control study design. Responding to the 2008 IOM report, which urged an expansion of geriatric practice skills for all health professions, we chose to also use a train-the-trainer model in our approach, which in this study meant that the two lead investigators first identified and trained a small cohort of occupational therapists using the DVD materials, and they in turn trained the other occupational therapists at their respective work sites.

The specific goals of this study were (1) to assess occupational therapists’ mental health knowledge and attitudes before and after intervention using a set of professional education materials delivered in DVD format and (2) to assess change in occupational therapists’ clinical practices using a chart audit 3 mo before and 3 and 6 mo after DVD training.

Method

Research Design

The research study was conducted in partnership with two large subacute rehabilitation providers in Michigan. The provider organizations embraced patient-centered and goal-driven rehabilitative care. Both also had a large multidisciplinary staff (e.g., occupational therapy, physical therapy, speech therapy) and significant numbers of frail geriatric patients. The occupational therapists employed at these organizations provided therapy services at eight different nursing home locations.

Using a wait-list control study design meant that half of the sample received the DVD training (intervention) first, and the second half of the sample received it later. Thus, at baseline (Time 1 [T1]), neither the intervention nor the control group of therapists had received the DVD training; at Time 2 (T2) the intervention group was trained (but the control group was not); and at Time 3 (T3) the control group was trained. The institutional review board application was submitted and approved by Wayne State University. Informed consent was obtained from all study participants.

Participants

The participants in this study were 75 occupational therapy clinicians at the two rehabilitation provider organizations (ns = 36 and 39, respectively) plus 8 occupational therapy trainers. Included in the study were all occupational therapists employed full time providing rehabilitation services to patients ≥65 years old. The 8 individuals (4 from each organization) who volunteered to serve as the occupational therapist trainers received permission from their organizations to do so.

Procedures

The first 4 therapists identified by coin toss received 2 days of DVD-based depression training and then immediately returned to their workplace to train their occupational therapy coworkers. The two senior authors (Lysack and Lichtenberg) provided the train-the-trainer education at Wayne State University’s Institute of Gerontology. All of the therapists trained by these first 4 trainers became the intervention group. The second set of 4 was trained 5 mo later using the identical protocol, and the therapists they trained became the control group.

Intervention

The intervention was an educational intervention using a previously developed set of aging and mental health materials in DVD format (Lysack et al., 2011). The intervention was delivered first in the train-the-trainer workshops, in which each therapist trainer received (1) instruction on the content of the DVDs (including pretraining–posttraining knowledge testing); (2) a binder of educational resource materials; (3) clinical competency training in the depression and cognition screening and assessment tools; and (4) a set of strategies to implement their new knowledge and skills in the workplace. The main task was to ensure the therapist trainers were prepared to teach the new knowledge and skills they had learned, that is, using depression screening and assessment tools and incorporating pleasant events into the daily routines of their patients when depression was found.

All trainers were evaluated at the conclusion of their 2-day workshop to ensure they were comfortable with and accurate in administering and scoring the assessments, understanding the link between mood and depression, discussing their findings with patients and the rehabilitation team, and documenting findings in the patient chart. Practice continued until the trainers reported feeling 100% confident in returning to their workplace to share their new knowledge and skills. The final phase of training was a practical brainstorming session focused on overcoming barriers to implementing this new training in the settings in which they worked.

After the trainers were trained, the trainers returned to their workplaces and trained the occupational therapy clinicians there using the same DVD training they had received. This training was done, for the most part, in two 1-hr sessions each week over 3 wk. Midway through the training of the occupational therapy clinicians, the senior authors (Lysack and Lichtenberg) scheduled a brief phone call to the trainers to solidify their initial training and provide support. An additional “booster call” was made approximately 3 mo after the first call to problem solve any barriers to implementation in the study sites.

Outcome Measures

Knowledge Testing.

We designed a 32-item knowledge questionnaire to test knowledge change with input from our clinical partner organizations. The questionnaire was a refinement and expansion of the 15-item questionnaire developed during earlier testing of the first three DVDs (Lysack et al., 2011). Although no psychometric testing of the questionnaire was undertaken, several steps were taken to design a quality instrument. In the first step, individual experts on the study team submitted questions that reflected the primary knowledge for each DVD module they designed. Then, the study’s two principal investigators (PIs) reviewed and revised the questions for readability and to ensure each had four response options (apart from the two yes–no items). The items received further review by AOTA staff for their CE concurrence. Unclear questions were edited until AOTA was ready to endorse them. The final questionnaire included 4 true–false questions and 28 items with four response options in a multiple-choice format. The questionnaires were administered at three time points. Baseline testing was conducted for all therapists (N = 75) before any training was conducted to measure pretraining knowledge. Testing at T2 was conducted for the intervention group after training was complete (n = 37) and concurrently with the control group (n = 34) to measure that no change in knowledge had occurred. (The ns were smaller at T2 because of therapist vacation time or illness.) This test was conducted 1 mo after baseline. Knowledge testing at T3 was a posttraining test for the control group (n = 36) and was conducted concurrently with that for the intervention group (n = 39) to measure knowledge retention. This test was administered 5 mo after T2.

Chart Review.

The chart review was designed to evaluate whether therapists’ clinical practices improved after the DVD training. Patient charts were reviewed for a set of 21 performance indicators that the investigators and their rehabilitation provider organizations deemed most indicative of stronger mental health practices. These indicators had also been tested in earlier work (Lysack et al., 2011). Key items included the use of depression and cognitive screening and assessment tools, communication of assessment results to the rehabilitation team, and use of pleasant events and behavioral activation techniques when depression was found. The indicators were designed to have behavioral anchors to reduce the inferences chart reviewers had to make. Thus, indicators were either present or absent (e.g., therapists performed depression screening, yes or no). Patient charts were collected and reviewed for a 3-mo period before therapist training, 3 mo after the intervention group training, and 3 mo after the control group training.

The original goal was to review 960 charts in total (160 charts from the intervention group and 160 from the control group at three time points each). A power calculation suggested this number would be more than adequate to identify practice changes. However, the investigators were also interested to see whether the DVD training actually penetrated the geriatric rehabilitation practices of the therapists in the study. In short, we wanted the DVD training to have a therapeutic impact on as many older adults as possible, beyond the goals we had for the research study. With 160 charts in each study group and approximately 37 therapists in each group, each therapist had on average four to five of their patient charts reviewed at each time point. Because therapists in this study had a caseload of about 30 patients at any given time, the study reached 5 of 30 patients, or 17% of a typical therapist’s caseload.

The actual number of charts reviewed was 943, which was the result of misapplication of the chart inclusion criteria in four instances, 1 duplicate chart, 3 illegible chart notes, and 9 missing charts. In total, 313 charts were reviewed at baseline (156 intervention group, 157 control group), 311 charts were reviewed at T2 (152 intervention group, 159 control group), and 319 charts were reviewed at T3 (160 intervention group, 159 control group).

Educational DVDs

This study evaluated an educational intervention based on a previously developed DVD box set of six 1-hr learning modules. The primary focus of the DVDs was mood disorders and the identification and treatment of depression in geriatric patients. Three of six DVDs focus on depression identification and treatment. The other three focus on caregiver stress, falls and balance issues in older adults, and community mobility.

Each DVD contains lecture-style materials and short video clips of patient–therapist interactions so the viewer can learn from watching the dialogue and demonstrations of assessments and treatment techniques. The training package also includes one DVD with full-length patient video clips, one CD with all of the lecture slides in PowerPoint, and the seven clinical assessments in PDF format.

Data Collection

The knowledge questionnaires were anonymous, they were administered by the therapist trainers at each site, and they were returned to the research team for scoring.

The patient charts were collected at each of the eight sites by a single staff person supervised by the site’s occupational therapy manager. The charts were consecutively completed charts of patients seen by the occupational therapists in the study. All charts had to meet two inclusion criteria: The patient had to be age 60 or older and currently receiving services from a therapist in the study. The staff charged with collecting the charts and providing redacted chart notes for review were blinded to study group assignment. The therapists were blinded as well. They did not know there was a chart audit component to the study; they believed it consisted only of knowledge testing before and after DVD training.

The chart data were then passed to two independent occupational therapist raters completely unassociated with the clinical sites and blinded to which group (intervention or control) each patient chart belonged. The raters evaluated each chart for evidence of the performance indicators, the presence of which indicated positive practice change. The raters were trained by the PIs to be as accurate as possible in their identification of performance indicators. The PIs conducted a small interrater test with the raters on a subset of early charts. The raters coded 60 charts each (20 from each of the three chart review periods) and searched for the presence or absence of 23 items on the performance indicator checklist in every chart. This exercise yielded 1,379 opportunities for agreement (note: one variable in one file was excluded because of missing data). The raters coded 1,267 items similarly, translating to a κ of .838. Interrater reliability in this range was characterized as “almost perfect” by Landis and Koch (1977) and as “excellent” by Fleiss (1981) . With this assurance, the two raters proceeded to review every patient chart against the performance indicator checklist. Each rater reviewed an approximately equal number of charts and completed the task over 10 mo.

Note that the original performance indicators measure consisted of 21 dichotomous items scored “yes—present” or “not present” when reviewed by the two experienced therapists. Three items (“Referral to team members (including physician) for depression,” “Referral to team members (including physician) for cognitive problems,” and “Notation about shared findings in team conference”) were summed and included as a single item, resulting in a total of 19 items. It did not affect the overall score and was done for conceptual reasons, that is, combining the team communication variables into one. A total score was calculated by summing the number of correct responses. Because each occupational therapist in the study was trained by one of four occupational therapist trainers, a prospective one-way analysis of variance (ANOVA) was used to verify that scores on the primary measure did not significantly vary among the four trainers. A two-way ANOVA was used to investigate the effect of group assignment (intervention vs. control) and time of training (baseline [T1], T2, and T3) on this measure of clinical behaviors. Post hoc t tests were completed to better describe differences suggested by the two-way ANOVA. Chi-square tests of independence were completed for participants in the intervention group, comparing rates of clinical behaviors by time of training. Because of inconsistency in the distribution of participants between response options, and because the one question scored 0–3 (described earlier) resulted in a 4 × 2 matrix rather than a 2 × 2 matrix, Cramer’s V was selected as the effect size measure for these comparisons.

Analysis

Data were analyzed using SPSS Version 19.0 (IBM, Armonk, NY). Knowledge change and practice change were both assessed using two-way ANOVA. We calculated effect sizes (i.e., the effect of the DVD training on knowledge change and practice change using the formula for Cohen’s d and the population correlation coefficient r (Cohen, 1988). Interpretation of effect size was based on Cohen’s (1988) recommendations.

Results

Eight occupational therapist trainers and 75 additional therapists participated in the study; no participants dropped out. Nonetheless, a small amount of data was missing. At baseline, there were 39 intervention group questionnaires and 36 control group questionnaires. At T2, after DVD training for the intervention group, there were 37 intervention group questionnaires and 34 control group questionnaires as a result of therapists’ vacation time or illness. At T3, these numbers had returned to baseline levels.

The interpretation of study results is aided by knowing with what type of patients the occupational therapists worked. The 943 patients were typical of the geriatric rehabilitation population: Their mean age was 80, 66% (n = 622) were women, and the mean length of patient stay in rehabilitation was 32 days (range = 5–112 days). The most common primary and secondary diagnoses, made by physicians in the charts before patient referral to occupational therapy and before the charts were accessible for the study were musculoskeletal (other) conditions (61%), hypertension (59%), generalized weakness (48%), heart disease (47%), diabetes (27%), dementia or Alzheimer’s disease (25%), and fractures (24%); 16% of the patients had a diagnosis of depression.

Knowledge Change

Complete knowledge questionnaire data were available for 59 clinicians at baseline (n = 30 intervention group, n = 29 control group), 63 at the second test (n = 35 intervention group, n = 28 control group), and 69 participants at the third test (n = 37 intervention group, n = 32 control group). The mean scores on the knowledge questionnaire for the intervention group were 19.83 (standard deviation [SD] = 2.10) at baseline, 25.4 (SD = 2.17) for the second test, and 24.19 (SD = 3.26) for the third test. Mean scores for the control group were 19.86 (SD = 3.00) at baseline, 19.71 (SD = 2.58) for the second test, and 23.25 (SD = 3.02) for the third test. We found no significant difference between the knowledge levels of the intervention and control groups at baseline. However, an independent-samples t test indicated significant improvement after the control group received training (i.e., comparing T2 and T3), t(58) = 4.89, p < .001, d = 1.25. Knowledge items that improved most from training were questions about mood, depression, mobility, fall assessment, alcohol abuse assessment, and caregiver information.

A 2-way ANOVA investigating the effects of training on knowledge questionnaire score revealed a significant Group × Time interaction, F(3, 118) = 40.53, p < .001, η²_p = .26; and significant main effects for time, F(3, 118) = 36.44, p < .001, η²_p = .24; and group, F(3, 118) = 39.72, p< .001, η²_p = .25. These η²_p values indicate large effect sizes (Cohen, 1988) for the interaction term and both main effects. The results show that knowledge increased after training in both groups: The higher knowledge scores of the intervention group compared with those of the control group at T2 was statistically significant, as was the increase in the control group’s knowledge after training at T3 (see Figure 1). Knowledge gain was largely retained by the intervention group over the 3-mo follow-up.

Figure 1.

Mean total knowledge questionnaire scores at Times 1–3 for participants in intervention and control groups.

Full Size | Slide (.ppt)

Chart Review

The chart review data are provided in Tables 1 and 2. The chart review was conducted to assess whether the DVD mental health training resulted in a change in therapists’ practice behaviors in the intervention group versus the control group—without behavior change, the mental health needs of the patients treated by the occupational therapist study participants would remain unaddressed. The results showed that trained therapists displayed significant improvements in practice.

Table 1.

Effect Size Calculations and Interpretations of Chart Review (Performance Indicator) Data

Comparison	T (df)^a	Effect Size (Cohen’s d)	R	Interpretation
Intervention T1, Intervention T2^b	8.54 (306)	0.98	.45	Large
Intervention T2, Intervention T3	0.79 (310)	0.09	.05	Small
Control T1, Control T2	4.07 (314)	0.46	.24	Medium
Control T2, Control T3	5.28 (316)	0.59	.28	Medium
Intervention T1, Control T1	5.01 (311)	0.57	.28	Medium
Intervention T2, Control T2	8.89 (309)	1.01	.45	Large
Intervention T3, Control T3	3.17 (317)	0.36	.17	Small

Note. T1 = Time 1; T2 = Time 2; T3 = Time 3.

aAll comparisons were statistically significant (p < .001). ^bComparison contrasts pre- and posttraining.

View Large

Table 1.

Effect Size Calculations and Interpretations of Chart Review (Performance Indicator) Data

Comparison	T (df)^a	Effect Size (Cohen’s d)	R	Interpretation
Intervention T1, Intervention T2^b	8.54 (306)	0.98	.45	Large
Intervention T2, Intervention T3	0.79 (310)	0.09	.05	Small
Control T1, Control T2	4.07 (314)	0.46	.24	Medium
Control T2, Control T3	5.28 (316)	0.59	.28	Medium
Intervention T1, Control T1	5.01 (311)	0.57	.28	Medium
Intervention T2, Control T2	8.89 (309)	1.01	.45	Large
Intervention T3, Control T3	3.17 (317)	0.36	.17	Small

Note. T1 = Time 1; T2 = Time 2; T3 = Time 3.

aAll comparisons were statistically significant (p < .001). ^bComparison contrasts pre- and posttraining.

Table 2.

Top Nine Performance Indicators in Patient Charts Before and After DVD Training (for Intervention Group Members Only)

	% Patient Charts With Performance Indicator Present
Item	Pretraining	Posttraining	χ²	Cramer’s V	Interpretation
Depression screen	1	22	35.3***	0.34	Medium
Use of MLDT GDS–3	0	35	65.77***	0.46	Large
Use of GDS–15	0	22	39.69***	0.36	Medium
Use of MLDT orientation to time and place and animal naming	0	35	65.77***	0.46	Large
Referring patients and sharing findingsi^a	1	4	8.17*	0.16	Small
Notation of alcohol use	0	6	10.40***	0.18	Small
Notation of patient’s balance confidence	8	18	7.46**	0.15	Small
Use of ABC Scale	0	3	4.08*	0.12	Small
Evidence of behavioral activation or pleasant events	0	5	7.21**	0.15	Small

Note. ABC Scale = Activities-specific Balance Confidence Scale; GDS = Geriatric Depression Scale; MLDT = MacNeill–Lichtenberg Decision Tree.

aThis item consisted of three test items: Referral to team members (including physician) for depression,” “Referral to team members (including physician) for cognitive problems,” and “Notation about shared findings in team conference.”

*p < .05. **p ≤ .01. ***p ≤ .001.

View Large

Table 2.

Top Nine Performance Indicators in Patient Charts Before and After DVD Training (for Intervention Group Members Only)

	% Patient Charts With Performance Indicator Present
Item	Pretraining	Posttraining	χ²	Cramer’s V	Interpretation
Depression screen	1	22	35.3***	0.34	Medium
Use of MLDT GDS–3	0	35	65.77***	0.46	Large
Use of GDS–15	0	22	39.69***	0.36	Medium
Use of MLDT orientation to time and place and animal naming	0	35	65.77***	0.46	Large
Referring patients and sharing findingsi^a	1	4	8.17*	0.16	Small
Notation of alcohol use	0	6	10.40***	0.18	Small
Notation of patient’s balance confidence	8	18	7.46**	0.15	Small
Use of ABC Scale	0	3	4.08*	0.12	Small
Evidence of behavioral activation or pleasant events	0	5	7.21**	0.15	Small

Note. ABC Scale = Activities-specific Balance Confidence Scale; GDS = Geriatric Depression Scale; MLDT = MacNeill–Lichtenberg Decision Tree.

*p < .05. **p ≤ .01. ***p ≤ .001.

The most important analysis was the two-way ANOVA used to identify the effect of group membership and training on clinical behavior change (assessed using the performance indicators checklist; see Figure 2). This analysis produced a statistically significant interaction between group and time, F(2, 937) = 15.25, p < .001, η²_p = .03, with a significant main effect for both group, F(1, 941) = 85.01, p < .001, η²_p = .08, and time, F(2, 940) = 57.66, p < .001, η²_p = .11.

Figure 2.

Mean number of performance indicators identified in the chart review data at Times 1–3 for participants in intervention and control groups.

Full Size | Slide (.ppt)

Post hoc independent-samples t tests (shown in Table 1) showed the effect of training within and between groups. The first two t tests examined change from T1 to T2 within groups. Results showed that the intervention group’s performance on this measure of clinical behaviors increased significantly from a mean score of 1.62 at baseline to a mean score of 3.23 after training, t(170) = 8.54, p < .001, yielding a large effect size of d = 0.98. In other words, the occupational therapists trained with the DVD modules in the intervention group showed significant evidence of positive clinical actions to identify and treat depression. The control group’s performance also improved modestly from a mean of 1.24 at baseline to a mean of 1.55 at T2, t(276) = 4.06, p < .001, characterized by a medium effect size of d = 0.46.

We then compared the scores of the intervention and control groups at all three time points. The small difference between group means (intervention group mean = 1.62, control group mean = 1.24) at baseline was statistically significant, t(274) = 5.01, p < .001, and had a medium effect size of d = 0.57. This difference grew in magnitude after training of the intervention group (intervention group mean score = 3.23, control group mean score = 1.55), producing a large effect size of d = 1.01, t(189) = 8.89, p < .001. At T3, after the control group received the DVD training, the magnitude of this group difference decreased (intervention group = 3.03, control group = 2.33), t(297) = 3.17, p = .002, to a small effect size of d = 0.36.

Next, we compared scores within groups at T2 and T3. For the control group, this comparison reflected the effect of training. The control group’s mean score on the performance indicators measure increased from 1.55 to 2.33. This statistically significant difference, t(226) = 5.28, p < .001, yielded a medium effect size of d = 0.59. Finally, we compared the intervention group’s performance at T2 and T3, reflecting maintenance of gains over time. These group means did not differ significantly, t(310) = 0.80, p = .43, d = 0.09, suggesting that the effects of training were robust throughout the course of this study.

Chi-square tests of independence were completed to identify those items that best distinguished between pretest and posttest scores for the intervention group. Of all the performance indicators, nine significantly distinguished between pretraining and posttraining. Results of these comparisons are displayed in Table 2. As can be seen in Table 2, large effect sizes were found for depression screening and cognitive screening. Smaller effect sizes were noted for notations in patient charts about alcohol use, balance confidence, and behavioral activation (pleasant events) interventions for depression.

Finally, we undertook several additional tests to test for differences between the study groups. The mean ages of patients treated by the occupational therapists in this study were compared using a one-way ANOVA. The mean ages did not differ significantly between the intervention and control groups, F(1, 941) = 1.15, p = .28, or between time points, F(2, 940) = 0.28, p = .76. We also completed a one-way ANOVA to test for differences between occupational therapy clinicians trained by different trainers. This analysis was completed using data for the intervention group from T2. Results suggested no significant differences by trainer, F(3, 31) = 0.32, p = .81.

Discussion

The major finding of this study was that the educational intervention delivered by DVD was effective: Occupational therapy clinicians increased their knowledge about mood disorders and depression and demonstrated positive behavior changes that included more screening and treatment of depression. Our findings are important because they provide an evidence base for this training product.

The results highlight the range of factors that must be present for educational interventions to succeed in real practice settings. Our first observation was that the training was put to use. Therapists used the new brief screening tests for depression and cognitive impairment introduced during training. Use of the evidence-based tools increased from none before training to 35% afterward. Team communication also improved, with therapists much more likely to share patient findings after this educational intervention than before. Given the varied factors that work against adoption of new clinical practices in occupational therapy (Rochette, Korner-Bitensky, & Thomas, 2009), including avoidance of complex and difficult patients (Lequerica, Donnell, & Tate, 2009) and poor interprofessional collaboration (Nugus, Greenfield, Travaglia, Westbrook, & Braithwaite, 2010), we are pleased with this achievement. Although we believe the effective DVD training package was the primary reason for the improved clinical practices by occupational therapists, the project’s duration (13-mo active phase) and commitment of key administrative leaders at the clinical sites were likely also positive factors.

Second, the results showed increased attention to systematic screening for balance confidence (using the Activities-specific Balance Confidence Scale; Peretz, Herman, Hausdorff, & Giladi, 2006) during rehabilitation sessions focused on mobility and falls. We believe that therapists valued this standardized tool as a reliable method of identifying older adults at risk of falls, just as they valued the Geriatric Depression Scale (Yesavage et al., 1983) and the MacNeill–Lichtenberg Decision Tree (MacNeill & Lichtenberg, 2000) tools as reliable methods for screening for depression and cognition. Moreover, and as a direct result of training, therapists were more systematic in their efforts to remind patients to observe how their mood often improved when actively engaged in therapy sessions. We believe these simple but regular reminders about the link between activity and mood, coupled with efforts to foster an encouraging therapeutic environment, contributed to our positive results. As therapists saw their patients respond to their efforts, their new behaviors were positively reinforced.

Third, and most important given our study aims, we found that 5% of the geriatric rehabilitation patients in the study (compared with none at baseline) received some behavioral intervention for depression. This result is encouraging. We acknowledge that the quality and intensity of the pleasant events and behavioral activation interventions were not measured in this study, and our data consisted of professional behaviors with patients gleaned from chart reviews. Still, we are confident that if therapists undertook behavioral and pleasant-events efforts, they did so with those patients exhibiting the signs and symptoms of depression, often identified by therapists’ own screening.

Finally, our results also show that the clinical indicators demonstrated in the intervention group at T2 were sustained at T3. We believe the most probable explanation for this finding is that the education materials were simple and doable and that by that point in the study, therapists were convinced of their utility. At the outset of the project, we were committed to a study in which clinicians would immediately recognize the value of using evidence-based assessments. We thought new case identification would be paramount to motivating occupational therapists to (re)introduce pleasant events into the routines of their geriatric patients when they are found to have symptoms of depression. We believe that the intervention group’s sustained level of clinical practice change at T3 means the therapists were convinced their training made a meaningful difference (Russo, Leach, Lysack, Paulson, & Lichtenberg, in press).

Outside of the study design and therefore not systematically measured was the anecdotal finding that occupational therapists reported more professional confidence as a result of study participation; they felt they had become stronger members of the rehabilitation team.

Study Limitations

Despite the positive findings, several limitations must be acknowledged. First, the study measures—both the knowledge questionnaire and the chart review checklist—were designed specifically for this study. No validity or reliability testing was undertaken with either; thus, it is possible that these measures did not fully capture the effects of the DVD training. Still, the measures had face validity, and they were developed by an expert team and received extensive vetting in the AOTA CE process. If the questionnaire and behavioral indicators measures were inadequate, we expect they would have underestimated, not overestimated, the positive impact of the DVD training.

A second limitation is related to the chart review and the fact that performance indicators were based on groups of patients under therapists’ care and not the same individual patients under care by the same therapists over time. Unfortunately, matching therapists with their patients and tracking them over time was beyond the resources of the study. We also acknowledge that no independent confirmation of depression cases was undertaken. Nonetheless, the fact that the performance indicators in the control group increased significantly after the DVD intervention, just as those of the intervention group did, increases our confidence that the DVD training was responsible for the improved clinical practices observed.

The third limitation is related to the wait-list control study design, which by definition necessitates a delay in one group’s receiving the intervention. As Figure 2 shows, therapists in the control group had slightly weaker clinical performance as measured by the behavioral indicators at baseline and T3. They also responded slightly less strongly to the intervention. This result may have been due to unrecognized differences between the two groups that were related to the limitations of randomizing across only eight clinical sites. It is also possible that some events transpired in the control group setting that made implementation of the intervention problematic. It is also possible that the differences are not meaningful and are merely regression to the mean.

Future Research

Future research is needed to build on the results offered here. Participation and randomization of an even larger number of clinical sites is advised because it helps to ensure the equivalence of the intervention and control groups. Future studies must also carefully measure the delivery of the educational intervention (i.e., treatment fidelity) to ensure equivalence of groups before introduction of the intervention and to ensure the intervention is delivered in the identical fashion to both study groups. Differences in therapist training and background, clinical setting, and organizational support can all influence study findings.

Implications for Occupational Therapy Practice

After the DVD education training, occupational therapists in both the intervention group and the control group demonstrated increased knowledge and more evidence of positive clinical behaviors, including screening for depression and cognitive impairment and behavioral activation and facilitation of more pleasant events in their patients’ lives.
Our findings suggest that when clinicians are trained with proven evidence-based tools and treatments for mental health problems such as depression that are simple to learn and incorporate in practice, these tools and treatments will be used.

Acknowledgments

This work was supported by two Retirement Research Foundation grants (2010-017 and 2006-024), the generous partnership of HCR ManorCare and Integrity Rehabilitation Services in Michigan, and the participation of their dedicated occupational therapists.

References

Alexopoulos, G. S., & Bruce, M. L. (2009). A model for intervention research in late-life depression. International Journal of Geriatric Psychiatry, 24, 1325–1334. http://dx.doi.org/10.1002/gps.2287 [Article] [PubMed]

Anderson, D. N. (2001). Treating depression in old age: The reasons to be positive. Age and Ageing, 30, 13–17. http://dx.doi.org/10.1093/ageing/30.1.13 [Article] [PubMed]

Brown, E. L., Bruce, M. L., McAvay, G. J., Raue, P. J., Lachs, M. S., & Nassisi, P. (2004). Recognition of late-life depression in home care: Accuracy of the outcome and assessment information set. Journal of the American Geriatrics Society, 52, 995–999. http://dx.doi.org/10.1111/j.1532-5415.2004.52271.x [Article] [PubMed]

Brown, E. L., McAvay, G., Raue, P. J., Moses, S., & Bruce, M. L. (2003). Recognition of depression among elderly recipients of home care services. Psychiatric Services, 54, 208–213. http://dx.doi.org/10.1176/appi.ps.54.2.208 [PubMed]

Bruce, M. L., McAvay, G. J., Raue, P. J., Brown, E. L., Meyers, B. S., Keohane, D. J., … Weber, C. (2002). Major depression in elderly home health care patients. American Journal of Psychiatry, 159, 1367–1374. http://dx.doi.org/10.1176/appi.ajp.159.8.1367 [Article] [PubMed]

Centers for Disease Control and Prevention. (2011). An estimated 1 in 10 U.S. adults report depression. Retrieved from www.cdc.gov/features/dsdepression/

Clark, F., Jackson, J., Carlson, M., Chou, C. P., Cherry, B. J., Jordan-Marsh, M., … Azen, S. P. (2012). Effectiveness of a lifestyle intervention in promoting the well-being of independently living older people: Results of the Well Elderly 2 Randomised Controlled Trial. Journal of Epidemiology and Community Health, 66, 782–790. http://dx.doi.org/10.1136/jech.2009.099754 [Article] [PubMed]

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Fleiss, J. L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.

Frederick, J. T., Steinman, L. E., Prohaska, T., Satariano, W. A., Bruce, M., Bryant, L., … Snowden, M.; Late Life Depression Special Interest Project Panelists., (2007). Community-based treatment of late life depression: An expert panel-informed literature review. American Journal of Preventive Medicine, 33, 222–249. http://dx.doi.org/10.1016/j.amepre.2007.04.035 [Article] [PubMed]

Graff, M. J., Vernooij-Dassen, M. J., Thijssen, M., Dekker, J., Hoefnagels, W. H., & Olderikkert, M. G. (2007). Effects of community occupational therapy on quality of life, mood, and health status in dementia patients and their caregivers: A randomized controlled trial. Journals of Gerontology, Series A: Biological Sciences and Medical Sciences, 62, 1002–1009. http://dx.doi.org/10.1093/gerona/62.9.1002 [Article]

Institute of Medicine. (2008). Retooling for an aging America: Building the health care workforce. Washington, DC: National Academies Press.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. http://dx.doi.org/10.2307/2529310 [Article] [PubMed]

Lequerica, A. H., Donnell, C. S., & Tate, D. G. (2009). Patient engagement in rehabilitation therapy: Physical and occupational therapist impressions. Disability and Rehabilitation, 31, 753–760. http://dx.doi.org/10.1080/09638280802309095 [Article] [PubMed]

Li, L. W., & Conwell, Y. (2007). Mental health status of home care elders in Michigan. Gerontologist, 47, 528–534. http://dx.doi.org/10.1093/geront/47.4.528 [Article] [PubMed]

Lichtenberg, P. A., Kimbarow, M. L., Morris, P., & Vangel, S. J. (1996). Behavioral treatment of depression in predominantly African American medical patients. Clinical Gerontologist, 17, 15–33. http://dx.doi.org/10.1300/J018v17n02_03 [Article]

Lysack, C., Lichtenberg, P. A., & Schneider, B. (2011). Effect of a DVD intervention on therapists’ mental health practices with older adults. American Journal of Occupational Therapy, 65, 297–305. http://dx.doi.org/10.5014/ajot.2011.001354 [Article] [PubMed]

MacNeill, S. E., & Lichtenberg, P. A. (2000). The MacNeill-Lichtenberg Decision Tree: A unique method of triaging mental health problems in older medical rehabilitation patients. Archives of Physical Medicine and Rehabilitation, 81, 618–622. [PubMed]

Matuska, K., Giles-Heinz, A., Flinn, N., Neighbor, M., & Bass-Haugen, J. (2003). Outcomes of a pilot occupational therapy wellness program for older adults. American Journal of Occupational Therapy, 57, 220–224. http://dx.doi.org/10.5014/ajot.57.2.220 [Article] [PubMed]

Nugus, P., Greenfield, D., Travaglia, J., Westbrook, J., & Braithwaite, J. (2010). How and where clinicians exercise power: Interprofessional relations in health care. Social Science and Medicine, 71, 898–909. http://dx.doi.org/10.1016/j.socscimed.2010.05.029 [Article] [PubMed]

Peretz, C., Herman, T., Hausdorff, J. M., & Giladi, N. (2006). Assessing fear of falling: Can a short version of the Activities-specific Balance Confidence Scale be useful?. Movement Disorders, 21, 2101–2105. http://dx.doi.org/10.1002/mds.21113 [Article] [PubMed]

Quijano, L. M., Stanley, M. A., Petersen, N. J., Casado, B. L., Steinberg, E. H., Cully, J. A., & Wilson, N. L. (2007). Healthy IDEAS: A depression intervention delivered by community based case managers serving older adults. Journal of Applied Gerontology, 26, 139–156. http://dx.doi.org/10.1177/0733464807299354 [Article]

Rochette, A., Korner-Bitensky, N., & Thomas, A. (2009). Changing clinicians’ habits: Is this the hidden challenge to increasing best practices. Disability and Rehabilitation, 31, 1790–1794. http://dx.doi.org/10.1080/09638280902803773 [Article] [PubMed]

Ruchinskas, R. (2002). Rehabilitation therapists’ recognition of cognitive and mood disorders in geriatric patients. Archives of Physical Medicine and Rehabilitation, 83, 609–612. http://dx.doi.org/10.1053/apmr.2002.32496 [Article] [PubMed]

Russo, T., Leach, C., Lysack, C., Paulson, D., & Lichtenberg, P. (in press). Efficacy of a train-the-trainer curriculum for increasing occupational therapists’ mental health knowledge. Occupational Therapy in Mental Health.

Sherlock, M. (2005). Screening for depression in 21st-century home care. Home Health Care Management and Practice, 17, 183–187. http://dx.doi.org/10.1177/1084822304271791 [Article]

Steinman, L. E., Frederick, J. T., Prohaska, T., Satariano, W. A., Dornberg-Lee, S., Fisher, R., … Snowden, M.; Late Life Depression Special Interest Project Panelists., (2007). Recommendations for treating depression in community-based older adults. American Journal of Preventive Medicine, 33, 175–181. http://dx.doi.org/10.1016/j.amepre.2007.04.034 [Article] [PubMed]

U.S. Department of Health and Human Services. (1999). Mental health: A report of the Surgeon General. Rockville, MD: Author.

U.S. Census Bureau. (2012). Older Americans 2012: Key indicators of well-being. Retrieved from ww.agingstats.gov/Main_Site/Data/2012_Documents/docs/EntireChartbook.pdf

Yesavage, J. A., Brink, T. L., Rose, T. L., Lum, O., Huang, V., Adey, M., & Leirer, V. (1983). Development and validation of a geriatric depression screening scale: A preliminary report. Journal of Psychiatric Research, 17, 37–49. http://dx.doi.org/10.1016/0022-3956(82)90033-4, [Article]

Figure 1.

Mean total knowledge questionnaire scores at Times 1–3 for participants in intervention and control groups.

Full Size | Slide (.ppt)

Figure 2.

Mean number of performance indicators identified in the chart review data at Times 1–3 for participants in intervention and control groups.

Full Size | Slide (.ppt)

Table 1.

Effect Size Calculations and Interpretations of Chart Review (Performance Indicator) Data

Comparison	T (df)^a	Effect Size (Cohen’s d)	R	Interpretation
Intervention T1, Intervention T2^b	8.54 (306)	0.98	.45	Large
Intervention T2, Intervention T3	0.79 (310)	0.09	.05	Small
Control T1, Control T2	4.07 (314)	0.46	.24	Medium
Control T2, Control T3	5.28 (316)	0.59	.28	Medium
Intervention T1, Control T1	5.01 (311)	0.57	.28	Medium
Intervention T2, Control T2	8.89 (309)	1.01	.45	Large
Intervention T3, Control T3	3.17 (317)	0.36	.17	Small

Note. T1 = Time 1; T2 = Time 2; T3 = Time 3.

aAll comparisons were statistically significant (p < .001). ^bComparison contrasts pre- and posttraining.

View Large

Table 1.

Effect Size Calculations and Interpretations of Chart Review (Performance Indicator) Data

Comparison	T (df)^a	Effect Size (Cohen’s d)	R	Interpretation
Intervention T1, Intervention T2^b	8.54 (306)	0.98	.45	Large
Intervention T2, Intervention T3	0.79 (310)	0.09	.05	Small
Control T1, Control T2	4.07 (314)	0.46	.24	Medium
Control T2, Control T3	5.28 (316)	0.59	.28	Medium
Intervention T1, Control T1	5.01 (311)	0.57	.28	Medium
Intervention T2, Control T2	8.89 (309)	1.01	.45	Large
Intervention T3, Control T3	3.17 (317)	0.36	.17	Small

Note. T1 = Time 1; T2 = Time 2; T3 = Time 3.

aAll comparisons were statistically significant (p < .001). ^bComparison contrasts pre- and posttraining.

Table 2.

Top Nine Performance Indicators in Patient Charts Before and After DVD Training (for Intervention Group Members Only)

	% Patient Charts With Performance Indicator Present
Item	Pretraining	Posttraining	χ²	Cramer’s V	Interpretation
Depression screen	1	22	35.3***	0.34	Medium
Use of MLDT GDS–3	0	35	65.77***	0.46	Large
Use of GDS–15	0	22	39.69***	0.36	Medium
Use of MLDT orientation to time and place and animal naming	0	35	65.77***	0.46	Large
Referring patients and sharing findingsi^a	1	4	8.17*	0.16	Small
Notation of alcohol use	0	6	10.40***	0.18	Small
Notation of patient’s balance confidence	8	18	7.46**	0.15	Small
Use of ABC Scale	0	3	4.08*	0.12	Small
Evidence of behavioral activation or pleasant events	0	5	7.21**	0.15	Small

Note. ABC Scale = Activities-specific Balance Confidence Scale; GDS = Geriatric Depression Scale; MLDT = MacNeill–Lichtenberg Decision Tree.

*p < .05. **p ≤ .01. ***p ≤ .001.

View Large

Table 2.

Top Nine Performance Indicators in Patient Charts Before and After DVD Training (for Intervention Group Members Only)

	% Patient Charts With Performance Indicator Present
Item	Pretraining	Posttraining	χ²	Cramer’s V	Interpretation
Depression screen	1	22	35.3***	0.34	Medium
Use of MLDT GDS–3	0	35	65.77***	0.46	Large
Use of GDS–15	0	22	39.69***	0.36	Medium
Use of MLDT orientation to time and place and animal naming	0	35	65.77***	0.46	Large
Referring patients and sharing findingsi^a	1	4	8.17*	0.16	Small
Notation of alcohol use	0	6	10.40***	0.18	Small
Notation of patient’s balance confidence	8	18	7.46**	0.15	Small
Use of ABC Scale	0	3	4.08*	0.12	Small
Evidence of behavioral activation or pleasant events	0	5	7.21**	0.15	Small

Note. ABC Scale = Activities-specific Balance Confidence Scale; GDS = Geriatric Depression Scale; MLDT = MacNeill–Lichtenberg Decision Tree.

*p < .05. **p ≤ .01. ***p ≤ .001.