Facilitated Learning Model to Teach Habits of Evidence-Based Reasoning Across an Integrated Master of Science in Occupational Therapy Curriculum

Ellen S. Cohn; Wendy J. Coster; Jessica M. Kramer

doi:10.5014/ajot.2014.685S05

Free

Research Article | September 2014

Facilitated Learning Model to Teach Habits of Evidence-Based Reasoning Across an Integrated Master of Science in Occupational Therapy Curriculum

Ellen S. Cohn; Wendy J. Coster; Jessica M. Kramer

Author Affiliations

Article Information

American Journal of Occupational Therapy, September/October 2014, Vol. 68, S73-S82. doi:10.5014/ajot.2014.685S05

Abstract

We describe an integrated master of science in occupational therapy curriculum and a coordinated sequence of evidence-based practice (EBP) courses that incorporate systematic, pragmatic teaching strategies to develop students’ EBP skills and habits of reasoning. The EBP courses focus sequentially on the occupational lives of clients and methods for gaining information about occupational performance and needs; appraising the internal, external, and statistical validity of intervention evidence; and generating evidence from one’s own practice to answer questions about individual or group client outcomes. All EBP courses use facilitated learning processes that encourage graduate students to take responsibility for their own learning, guided by a carefully structured series of assignments. The integrated curriculum scaffolds the translation and application of previously learned knowledge and skills, including EBP knowledge, into different contexts. Student survey data suggest that graduating students view EBP as an integral part of the clinical process and begin to internalize the habits necessary to be evidence-based practitioners.

To design client-centered interventions, occupational therapy practitioners must determine what type and amount of intervention will achieve the best possible outcomes. Determining the best possible intervention and collaboratively engaging clients in the intervention process require evidence-based practice (EBP), the “conscientious, explicit and judicious use of current evidence in making decisions” about the intervention recommendation (Sackett, Rosenberg, Muir Gray, Haynes, & Richardson, 1996, p. 71). Client-centered EBP also includes reasoning that integrates the practitioners’ clinical expertise and understanding of the intervention environment with the clients’ values and circumstances (Egan, Dubouloz, & Vallerand, 1998). In this model of evidence-based occupational therapy, the practitioner communicates with clients about research evidence to engage them in a collaborative evidence-based decision-making process. Thus, the practitioner must understand and be prepared to effectively communicate the evidence in ways that facilitate collaboration with clients.

Evidence-based reasoning skills are first developed within the context of the professional academic program. Ideally, this initial experience provides a firm foundation for application of evidence-based reasoning throughout the occupational therapy practitioner’s career (Thomas, Saroyan, & Dauphinee, 2011). Sackett, Rosenberg, Richardson, and Haynes (1997) noted that implementation of EBP required not only mastery of specific skills (e.g., appraising research evidence) but also the development of self-directed habits to help sustain routine use of current evidence throughout the professional’s career. Both of these elements—skills and habits—need to be developed during entry-level professional education. It is well known that practice matters when learning a new set of skills or routines. Therefore, to develop the habits of mind that will support implementation of evidence-based reasoning as an everyday norm, it is recommended that EBP be incorporated into every aspect of health practitioners’ academic curriculum (Thomas et al., 2011).

Lin, Murphy, and Robinson (2010) described six steps to evidence-based occupational therapy: (1) formulating a clinical question, (2) searching efficiently for the best available evidence, (3) critically analyzing the evidence for its validity and usefulness, (4) integrating the appraisal with personal clinical expertise and the client’s preferences, (5) evaluating one’s performance or outcomes of actions, and (6) disseminating and communicating knowledge. Although the steps in the EBP process are well described, many studies have documented that occupational therapy educators must continue to seek effective teaching methods to help students develop the skills and habits for integrating research evidence into practice (DeAngelis, DiMarco, & Toth-Cohen, 2013; Thomas et al., 2011; Thomas, Saroyan, & Snider, 2012).

In one recent survey about teaching EBP (De Angelis et al., 2013), more than 90% of the responding master of science in occupational therapy (MSOT) programs reported teaching the first four steps in the evidence-based process described by Lin et al. (2010) . The number of programs that reported teaching Steps 5 and 6, related to evaluation and communicating evidence-based knowledge, was slightly lower (79.3% and 86.2%, respectively). Eighty-two percent of the programs reported incorporating EBP concepts into at least seven courses, and 90% or more of the programs reported incorporating EBP concepts into clinical skills and assessment courses. Although EBP is a priority, the specific methods to prepare students to develop sustainable skills and habits have not been described. Moreover, little is known about students’ ability to integrate EBP into fieldwork settings and future practice.

This article describes an integrated MSOT curriculum and a coordinated sequence of EBP courses that incorporate systematic, pragmatic teaching strategies to develop students’ evidence-based skills and habits of mind. Specifically, we describe “facilitated learning” activities that meet the Accreditation Council for Occupational Therapy Education standards for EBP and research (American Occupational Therapy Association [AOTA], 2012; see Table 1) in a way that is relevant to students and feasible for faculty to implement. We present student-reported outcomes regarding satisfaction with this learning process as it relates to EBP courses and the application of knowledge and skills during Level II fieldwork.

Table 1.

Accreditation Council for Occupational Therapy Education Standards (American Occupational Therapy Association, 2012)

Number	Standard
B.8.2.	Effectively locate, understand, critique and evaluate information, including the quality of research evidence.
B.8.3.	Use scholarly literature to make evidence-based decisions.
B.8.4.	Understand and use basic descriptive, correlational, and inferential quantitative statistics and code, analyze, and synthesize qualitative data.
B.8.5.	Understand and critique the validity of research studies, including designs (both quantitative and qualitative) and methodology.
B.8.6.	Demonstrate the skills necessary to design a scholarly proposal that includes the research question, relevant literature, sample, design, measurement, and data analysis.
B.8.7.	Participate in scholarly activities that evaluate professional practice, service delivery, and/or professional issues (e.g., Scholarship of Integration, Scholarship of Application, Scholarship of Teaching and Learning).
B.8.8.	Demonstrate basic skills necessary to write a scholarly report in a format for presentation or publication.

View Large

Table 1.

Accreditation Council for Occupational Therapy Education Standards (American Occupational Therapy Association, 2012)

Number	Standard
B.8.2.	Effectively locate, understand, critique and evaluate information, including the quality of research evidence.
B.8.3.	Use scholarly literature to make evidence-based decisions.
B.8.4.	Understand and use basic descriptive, correlational, and inferential quantitative statistics and code, analyze, and synthesize qualitative data.
B.8.5.	Understand and critique the validity of research studies, including designs (both quantitative and qualitative) and methodology.
B.8.6.	Demonstrate the skills necessary to design a scholarly proposal that includes the research question, relevant literature, sample, design, measurement, and data analysis.
B.8.7.	Participate in scholarly activities that evaluate professional practice, service delivery, and/or professional issues (e.g., Scholarship of Integration, Scholarship of Application, Scholarship of Teaching and Learning).
B.8.8.	Demonstrate basic skills necessary to write a scholarly report in a format for presentation or publication.

MSOT Curriculum

The Boston University (BU) MSOT curriculum is designed to prepare future practitioners to become agents of change for clients and the occupational therapy profession by advocating for a system and culture in which “best practice” is carried out every day and is the norm. To provide excellent intervention in an increasingly complex societal context that is constantly evolving in response to scientific advances and economic and political forces, occupational therapy practitioners must be able to make connections among seemingly disparate information and use a wide range of knowledge to make decisions (Association of American Colleges and Universities, 2002). Making these connections requires practitioners to integrate and adapt learning from one situation to problems encountered in other situations over time. The goal in occupational therapy education is to prepare students to develop the clinical reasoning habits that will guide them to make judgments informed by the best available evidence and to synthesize and translate their academic learning to fieldwork experiences and ultimately future practice. With these goals in mind, the BU occupational therapy curriculum was designed to provide integrated and recursive instruction in which knowledge and skills are intentionally connected and applied across courses.

The BU curriculum uses an integrated approach in which knowledge and skills are learned as they relate to clients’ functional and contextual factors rather than specific diagnoses, primary area of health concern (physical disabilities or mental health), or age groups (pediatrics or older adults). Courses focus on the processes of assessment and intervention as they dynamically shift according to the specific occupational needs of the client population. Total class size varies from 45 to 60. Courses, including EBP, may include all students together or in smaller sections, depending on the focus of the course. The processes of EBP are first introduced in a series of three 2-credit courses in which students learn to apply knowledge and skills needed to describe client populations, critically evaluate intervention and prediction research, and measure outcomes at the individual and group level.

Sequence of Evidence-Based Practice Courses

In writing about how to teach EBP, Sackett and colleagues (1997) and Straus, Richardson, Glasziou, and Haynes (2005) noted that in the real world health care practitioners are extremely busy. These practitioners often need new clinical information, both diagnostic and therapeutic, to implement the best intervention possible. Therefore, Sackett and colleagues argued, practitioners need an efficient process and structure and emphasized the need to teach students strategies to minimize the amount of time and burden required to implement evidence-based reasoning.

With Sackett and colleagues’ (1997) ideas as guiding principles, our integrated curriculum includes a sequence of three 2-credit EBP courses and structured learning experiences that are intentionally designed to teach skills and strategies to support lifelong habits for evidence-based reasoning (Taylor, 2011). Table 2 provides an overview of the sequence, including course focus, objectives, and assignments. We modified as needed many of the materials developed by Sackett et al. to better support the occupational therapy decision-making context. These materials include structured appraisal guides for different types of study designs and the format of the Critically Appraised Topic (CAT). The appraisal guides and CAT template lead the user through the critical questions that need to be addressed in the appraisal process and then help the user organize the important information with the least effort into a concise report. Once they have learned these EBP skills and strategies, students are expected to apply them in other course assignments throughout the curriculum. This approach reflects the important curriculum goal of teaching students to use specifically designed resources that can realistically be applied in a busy practice setting.

Table 2.

Evidence-Based Practice: Course, Focus, Objectives, and Assignments

Course	Focus	Objectives	Assignments
EBP I	Framing focused clinical questions; finding, appraising, applying, and communicating about descriptive and assessment evidence	Understand how evidence informs clinical reasoning Plan an effective strategy for getting to know a client and his or her needs by writing relevant, answerable questions related to the client’s needs Effectively search research literature and judiciously select relevant evidence Accurately interpret basic qualitative and quantitative research Communicate findings from research evidence with colleagues and clients	Read case studies with related research articles and weekly guiding questions to guide application of research findings to the case Search for, select, and justify relevant research related to a clinical scenario Communicate evidence from one article to colleagues Synthesize findings from two articles and describe how to communicate findings to parents described in case study Conduct in-class analysis of qualitative data Complete final exam based on application of two research articles to case study
EBP II	Appraising the internal, external, and statistical validity of evidence related to intervention effectiveness and predicting client outcomes	Develop PICO (P: patient, problem, or population; I: intervention; C: comparison, control, or comparator; O: outcomes) questions related to intervention and prediction Conduct efficient and effective searches of Internet, library, and other resources Critically examine procedures, measures, and statistics used in intervention studies, and discuss the impact on study internal, external, and statistical validity Synthesize results from research studies on the basis of analysis of research quality and client and contextual factors Communicate the results of a research appraisal using the CAT format Interpret syntheses of research to answer a clinical question Interpret predictive research evidence to evaluate factors that predict client outcomes Prepare a research proposal for an intervention study	Write two CATs to address a clinical question using assigned intervention articles Complete three search assignments identifying intervention, predictive, and systematic review evidence Critically appraise and synthesize a small body of literature (four articles) to address a clinical question Develop a research proposal to address a major limitation identified in a critical appraisal that affects the ability to fully answer a clinical question Complete final exam involving critical appraisal of a research article and application of findings
EBP III	Systematically generating evidence from one’s own practice to answer clinical questions about individual or group client outcomes using knowledge and skills about SSDs and program evaluation	Describe and apply program evaluation methodology to evaluate the outcomes of a community-based program Evaluate the rigor of single-subject research and use SSD research evidence to answer clinical questions Describe and apply single-subject research methodology to an evaluation of individual outcomes after occupational therapy intervention Prepare manuscripts and other materials to disseminate individual or program outcomes to occupational therapy practitioners, other professionals, agencies, and clients	Complete a CAT to address a clinical question using evidence from a SSD research study Apply understanding of internal, external, and statistical validity of SSD to investigate and report on a theory-driven intervention designed for a student Complete a unit exam based on application of critically appraising a SSD research article Prepare evidence-based recommendations for a community agency’s program evaluation process aligned with the program’s logic model

Note. CAT = critically appraised topic; EBP = evidence-based practice; SSD = single-subject design.

View Large

Table 2.

Evidence-Based Practice: Course, Focus, Objectives, and Assignments

Course	Focus	Objectives	Assignments
EBP I	Framing focused clinical questions; finding, appraising, applying, and communicating about descriptive and assessment evidence	Understand how evidence informs clinical reasoning Plan an effective strategy for getting to know a client and his or her needs by writing relevant, answerable questions related to the client’s needs Effectively search research literature and judiciously select relevant evidence Accurately interpret basic qualitative and quantitative research Communicate findings from research evidence with colleagues and clients	Read case studies with related research articles and weekly guiding questions to guide application of research findings to the case Search for, select, and justify relevant research related to a clinical scenario Communicate evidence from one article to colleagues Synthesize findings from two articles and describe how to communicate findings to parents described in case study Conduct in-class analysis of qualitative data Complete final exam based on application of two research articles to case study
EBP II	Appraising the internal, external, and statistical validity of evidence related to intervention effectiveness and predicting client outcomes	Develop PICO (P: patient, problem, or population; I: intervention; C: comparison, control, or comparator; O: outcomes) questions related to intervention and prediction Conduct efficient and effective searches of Internet, library, and other resources Critically examine procedures, measures, and statistics used in intervention studies, and discuss the impact on study internal, external, and statistical validity Synthesize results from research studies on the basis of analysis of research quality and client and contextual factors Communicate the results of a research appraisal using the CAT format Interpret syntheses of research to answer a clinical question Interpret predictive research evidence to evaluate factors that predict client outcomes Prepare a research proposal for an intervention study	Write two CATs to address a clinical question using assigned intervention articles Complete three search assignments identifying intervention, predictive, and systematic review evidence Critically appraise and synthesize a small body of literature (four articles) to address a clinical question Develop a research proposal to address a major limitation identified in a critical appraisal that affects the ability to fully answer a clinical question Complete final exam involving critical appraisal of a research article and application of findings
EBP III	Systematically generating evidence from one’s own practice to answer clinical questions about individual or group client outcomes using knowledge and skills about SSDs and program evaluation	Describe and apply program evaluation methodology to evaluate the outcomes of a community-based program Evaluate the rigor of single-subject research and use SSD research evidence to answer clinical questions Describe and apply single-subject research methodology to an evaluation of individual outcomes after occupational therapy intervention Prepare manuscripts and other materials to disseminate individual or program outcomes to occupational therapy practitioners, other professionals, agencies, and clients	Complete a CAT to address a clinical question using evidence from a SSD research study Apply understanding of internal, external, and statistical validity of SSD to investigate and report on a theory-driven intervention designed for a student Complete a unit exam based on application of critically appraising a SSD research article Prepare evidence-based recommendations for a community agency’s program evaluation process aligned with the program’s logic model

Note. CAT = critically appraised topic; EBP = evidence-based practice; SSD = single-subject design.

Facilitated Learning.

All EBP courses use facilitated learning processes, which encourage students to take control and responsibility for their own learning, guided by a carefully structured series of assignments (Jiang & Elen, 2011). Facilitated learning is based on the premise that students learn more when they take responsibility for their own learning. Knowles (1984), a well-known theorist on adult learning, recommended that adult educators serve as facilitators of learning rather than as content transmitters. The professor’s role becomes that of a facilitator and organizer, providing resources and support to learners. Facilitated learning is also based on the premise that analysis and synthesis are more likely to be achieved when students have opportunities for interactive group discussions with guidance to elaborate, build on, question, and construct arguments for and against the contributions of their classmates. Students learn with and from each other as they identify and implement solutions to challenges and problems (Regmi, 2012).

Because students entering an occupational therapy master’s degree program vary in their knowledge of how to conduct systematic and efficient literature searches and in their understanding of research methods and statistics, each student takes responsibility for shaping his or her own learning goals and achievements. Students must assess their current knowledge and determine whether they need additional knowledge to complete weekly assignments and, if so, to seek and use supplemental resources. For example, rather than assign chapters in the research text on a weekly basis, the syllabus identifies which chapters contain information relevant to each week’s topic. In addition, supplemental learning resources are provided on the university’s electronic learning management platform (Blackboard Learn™) to support the range of learning needs in a cohort of students.

Flipped Classroom.

In EBP courses, and throughout the integrated curriculum, the traditional structure of a “sage on a stage” who lectures with companion slides to transmit knowledge is “flipped” so that most of the basic content knowledge acquisition is moved out of the classroom. Course content in the form of videos of lectures or interactive lessons are posted online for students to review and learn before class. Class time is then used to apply concepts and engage in facilitated learning activities. Student attendance, preparation before class, and active involvement in classroom activities are essential to this learning process. Congruent with facilitated learning, the role of the instructor is to guide and facilitate students in understanding and applying newly acquired knowledge (Wilson, 2013).

First Semester: Evidence-Based Practice I.

In the first-semester EBP course, students begin to learn how to analyze research evidence to inform their reasoning about occupational therapy practice. The course focuses on finding, using, and communicating about two types of published research reports that support the practitioner’s first task of getting to know a client: reports about the occupational lives and needs of people like the client (e.g., similar health conditions, gender, cultural group) and reports about the quality of different assessment methods for gaining information about a client’s occupational performance and needs.

Students learn how to frame appropriate clinical questions to address specific client concerns and then to search for and appraise descriptive and assessment studies to obtain the best scientific evidence to inform their clinical decisions. Before class each week, students read a client case story and a research article related to the client story. The client story might be about someone living with multiple sclerosis and the need to assess her challenges living with a chronic condition. The research article might be about a new assessment designed to measure the impact of fatigue on functional performance.

Students complete guiding questions that require them to apply concepts of EBP to interpret and use the information in the article to reason about the client case story. For example, basic information questions direct students to specific content knowledge they need, such as psychometrics, to inform their reasoning. A basic information question might be “The article refers to the internal consistency of the fatigue scale. What is the definition of this term?” Another type of basic information question might ask students to extract specific information from the article, for example, “In Table 1 of the article, which of the three subscales has the lowest internal consistency? What is the name of the statistic that was used to report this information?”

These content-focused questions are followed by higher-order questions that require students to “mentally manipulate bits of information previously learned to create an answer or support an answer with logical reasoning” (Jiang & Elen, 2011, p. 280). A higher order question might be “The article reports results of studies on two different fatigue scales. Identify a strength and limitation of each scale based on the information in the article. Which would you recommend for assessing a client with multiple sclerosis and why?” Students complete the readings and guiding questions before class and use class time for discussion to clarify understanding of relevant concepts and apply learning to the client case and other potential clinical situations. This structure requires students’ active involvement in the acquisition and application of knowledge.

Students in the first EBP course learn to synthesize their understanding of research studies to communicate relevant findings to various stakeholders (client, family members, other practitioners, managers, or payers) described in the case story. Before preparing a communication, students read textbook chapters about educating clients, health literacy, and effective client-centered communication (Berger, 2014, Tickle-Degnen, 2014). For example, students read a case story about a family with a school-age child who is returning to school after a traumatic brain injury. The communication task involves reading a research article focused on social participation among children with the same condition and explaining the relevant findings to the child’s parents in a manner that invites and facilitates collaboration about the child’s occupational therapy intervention plan. These communication assignments prepare students to begin developing the communication and dissemination skills described by Lin et al. (2010) .

Second Semester: Evidence-Based Practice II.

The second EBP course continues to develop students' skills in analyzing and using evidence to inform their clinical reasoning. The course focuses on appraising the internal, external, and statistical validity of evidence related to intervention effectiveness and predicting client outcomes. Students continue to refine literature search skills to efficiently and effectively retrieve intervention research to inform the EBP process. Because no currently available text matches the design of this course, a series of text lessons was created that focus on the key content covered each week. These lessons are posted on Blackboard Learn. Students are responsible for studying the lessons before class; a brief (5–10 min) “mini-lecture” is given at the beginning of class to ensure key points are covered.

All remaining time in the 2-hr class is spent in collaborative, small-group structured learning activities, which are typically organized around a short case example (Neistadt, Wight, & Mulligan, 1998; Nolinske & Millis, 1999). For example, small groups may work together to answer questions about a journal article by extracting relevant information from the article and drawing conclusions regarding the appropriate application of findings to a particular case. The questions are structured and sequenced to guide the students through the reasoning process so that they attend to the most important information and develop confidence in understanding key elements of research design. Students are expected to discuss questions or points of uncertainty in their small groups before consulting with the instructor. The instructor’s time is therefore spent discussing only areas in which students need additional instruction or guidance. Over the course of the semester, some of the same articles, which have varying methodological strengths, are revisited and compared and contrasted as students develop increasingly complex understanding of internal, external, and statistical validity.

One of the important reasons for having students work in small groups is to demonstrate the value of pooling their knowledge so that they will carry this collaborative model forward into practice. The benefits of this approach are emphasized during a multiweek assignment in which students work in small groups to critique and synthesize a body of evidence related to one clinical question and to derive an appropriate “clinical bottom line” (Sackett et al., 1997) for intervention in a particular practice context. Groups work in parallel on the different stages of this assignment, with two groups examining the same articles. At each stage, individuals or groups working on the same article or case meet to compare and discuss their findings and work together to resolve questions or discrepancies in interpretation. They then prepare a “consensus” critical appraisal of the article or synthesis of the findings. The articles for this assignment are purposefully selected by the instructor; articles selected for each topic may vary in rigor but have enough similarities to enable students to draw some conclusions about an intervention for a particular context or client group.

In the final part of the assignment, each student prepares a draft research proposal of a study designed to address a limitation identified in the synthesis of evidence. Specifically, the proposal must address one shortcoming in the body of evidence that limits the student’s ability to make confident intervention decisions for the client group represented in the clinical case. The task of writing a first research proposal is scaffolded for students by encouraging them to build on existing research and focus on resolving only one limitation.

Third Semester: Evidence-Based Practice III.

The third EBP course focuses on the systematic generation of evidence from one’s own practice to answer clinical questions about individual or group client outcomes using knowledge and skills about single-subject designs and program evaluation. This course emphasizes practical approaches to defining and collecting relevant data on outcomes for clients and occupation-focused interventions or programs. We emphasize to the students that such data are necessary to document the benefits of occupational therapy services in terms that are meaningful to clients, families, and payers, the important sixth step of evidence-based reasoning as described by Lin and colleagues (2010) . As with the first two EBP courses, students collaborate in small groups to pool knowledge and use the course instructor as a resource during class meeting times to complete course assignments.

At the level of individual client outcomes, students acquire the knowledge and skills needed to identify and measure discrete outcomes, regularly track outcomes over time, and analyze outcomes to determine intervention success. Students first learn how to appraise the internal, external, and statistical validity of single-subject design studies. They then apply that knowledge in small groups to conduct such a study investigating a theory-driven intervention. Teams made up of the small groups provide intervention for one student while simultaneously learning how to collect and analyze single-subject intervention outcome data. The parameters for outcomes and basic data collection procedures are provided in the assignment to ensure the time frame for data collection remains feasible and to facilitate a positive learning experience. However, student teams must problem-solve the remaining aspects of the process to conduct and evaluate a theory-driven and client-centered intervention. The group format again reinforces peer-supported habits of evidence-based clinical reasoning. The culminating assignment is to prepare a manuscript following guidelines for a peer-reviewed publication.

At the level of group outcomes, students collaborate with a community program to identify EBP evaluation procedures. Students learn about a community program and then work in small teams to design logic models, identify evidence-supported measures congruent with program outcomes, and make recommendations for future program evaluation. Teams then communicate and disseminate their recommendations to the community program in thoughtfully designed one-page executive summaries. Students’ efforts have benefited a range of community programs providing services such as mentoring to youth with disabilities and empowerment and self-management training for people recovering from mental illness. Inclusion of this final step demonstrates to students the potential impact of effectively communicating evidence to various stakeholders.

Incorporating and Applying Evidence-Based Practice Skills Across the Curriculum

Assignments throughout the curriculum are designed to scaffold the translation of previously learned knowledge and skills, including EBP knowledge, into different contexts. The assignments become progressively more integrative across the curriculum, reflecting the developmental and recursive structure of the curriculum. Students are expected to integrate and routinely practice their evidence-based reasoning skills and strategies to design and communicate about interventions reflecting best practice, clients’ needs and concerns, and current available research evidence. For example, students hone their new knowledge to analyze, synthesize, and apply this information to real-world occupational therapy practice scenarios by designing evidence-based intervention plans for a client observed during one of their Level I fieldwork experiences. Case-based assignments in the third semester require application of skills such as occupational analysis and knowledge of learning and behavior change theories and evidence-based reasoning, all learned in the first and second semesters of the program (Halliwell, 2008).

Before class, students watch videos of a faculty member interviewing a client and administering occupation-focused assessments. Students also view videos of the client engaged in various occupations. Students are required to observe and analyze the client’s occupational performance and generate clinical questions to design an intervention for the client. Class time is used for problem solving together to generate relevant clinical questions to guide their search for evidence. Each student then conducts a review of the evidence in the literature and prepares a synthesis with a focused clinical bottom line to share with classmates. This task directly requires application of knowledge and skills acquired in the EBP core courses. Because students rely on these syntheses to prepare intervention plans, they hold each other accountable for high-quality work and ask each other challenging questions as they begin to analyze the available evidence. In this learning context, students are constantly looking for examples of best practice by comparing and contrasting intervention effectiveness studies to determine the best recommendation for a clinical situation.

In a culminating fourth-semester capstone course, students integrate and apply knowledge and practice EBP skills and habits acquired in the first three semesters by designing and coleading an evidence-based occupation-focused group in the community for 10–12 weeks. Students use their EBP skills to design the intervention and their skills for measuring client outcomes and program evaluation to assess the effectiveness of the intervention. Students disseminate and communicate their evidence-based interventions and program evaluation through professional research posters presented to each other at a program-sponsored capstone symposium. Many students then submit proposals to present their posters at state or national professional conferences. The real-world application of knowledge and skills enables students to strengthen habits of evidence-based reasoning, understand how these skills can be applied to community-based practice settings, and practice and refine the evaluation and communication steps of EBP described by Lin et al. (2010) . This continuous process is critical to ensure that application of evidence-based reasoning habits and skills is implemented in routine clinical practice.

Curriculum Evaluation

An important goal of the BU MSOT curriculum is to prepare students to synthesize and translate learning into new situations (first in fieldwork and then in practice) and to use specifically designed EBP resources that can be integrated into routine practice. We evaluate the extent to which this goal has been achieved using two sources of information. First, at the end of the didactic component of the curriculum, before Level II fieldwork, all students complete an exit survey. The data in Table 3 indicate that more than 80% of students were extremely or very satisfied with the EBP courses. Student comments reflect their appreciation for the value of EBP. One student wrote,

EBP skills are taught from the first semester and how to go about analyzing articles and studies is incorporated into everything we do. We are provided with a great analysis of how to consider the evidence in practice. I feel very confident in my ability to use research to guide practice.

Table 3.

Student Satisfaction With Evidence-Based Practice Courses

Year	Class Size	Students Extremely or Very Satisfied With Courses, n (%)
2010	47	40 (85.1)
2011	42	40 (95.2)
2012	49	42 (85.7)
2013	48	40 (83.3)

View Large

Table 3.

Student Satisfaction With Evidence-Based Practice Courses

Year	Class Size	Students Extremely or Very Satisfied With Courses, n (%)
2010	47	40 (85.1)
2011	42	40 (95.2)
2012	49	42 (85.7)
2013	48	40 (83.3)

Table 4 shows self-report data from the Student Evaluation of the Fieldwork Experience (SEFWE) completed by three cohorts (academic years 2010, 2011, and 2012) of BU MSOT students at the end of their Level II fieldwork experiences. Occupational therapy programs have the opportunity to tailor curriculum evaluation questions to meet the needs of their particular curriculum. Therefore, we asked students to identify how they used various types of evidence (descriptive, assessment, intervention) during their fieldwork placement (see Table 4). Students could select more than one option. For example, for in-service presentations to staff, 45% of students reported using descriptive evidence; 16%, assessment evidence; and 47%, intervention evidence. Some students used more than one type of evidence or combined three types of evidence in their presentations. The high percentage of students using all three types of evidence for self-study during fieldwork (86% for descriptive studies, 56% for assessment studies, and 80% for intervention studies) suggests that the curriculum has met the goal of instilling in students the habit of using evidence to inform their reasoning. Fewer students reported using evidence to communicate with payers, insurance companies, or case managers (i.e., 12% descriptive, 8% assessment, and 6% intervention); this finding is most likely a reflection of limited opportunities to communicate with these stakeholders during fieldwork experiences.

Table 4.

Student Use of Evidence Type During Fieldwork Placement, 2010–2012 (N = 253 Placements)

	Students Using Evidence Type, n (%)
Fieldwork Placement	Descriptive Evidence	Assessment Evidence	Intervention Evidence
Self-study	218 (86)	142 (56)	202 (80)
Collaboration with team	185 (73)	121 (48)	121 (48)
Collaboration with client, family, or caregivers	96 (38)	63 (25)	121 (48)
Share with payers, insurance companies, or case managers	30 (12)	20 (8)	15 (6)
In-service presentation to staff	114 (45)	40 (16)	119 (47)
Educational materials or protocol development	83 (33)	18 (7)	89 (35)

View Large

Table 4.

Student Use of Evidence Type During Fieldwork Placement, 2010–2012 (N = 253 Placements)

	Students Using Evidence Type, n (%)
Fieldwork Placement	Descriptive Evidence	Assessment Evidence	Intervention Evidence
Self-study	218 (86)	142 (56)	202 (80)
Collaboration with team	185 (73)	121 (48)	121 (48)
Collaboration with client, family, or caregivers	96 (38)	63 (25)	121 (48)
Share with payers, insurance companies, or case managers	30 (12)	20 (8)	15 (6)
In-service presentation to staff	114 (45)	40 (16)	119 (47)
Educational materials or protocol development	83 (33)	18 (7)	89 (35)

On the SEFWE report, students also have the opportunity to share examples of leadership activities completed during fieldwork. Many of their examples described application of their EBP skills to practice. Students reported developing a monthly journal club, conducting an in-service to teach staff how to review evidence, providing an evidence-based Web conference for occupational therapy practitioners in multiple sites, using a review of evidence to revise a hospital policy for precaution protocols, and using evidence to make recommendations to an occupational therapy department for revision to an assessment protocol. In addition, BU MSOT students have used their EBP skills to develop evidence-based resources on diagnostic conditions for an occupational therapy textbook (Boyt Schell, Gillen, & Scaffa, 2014), publish a systematic review in the American Journal of Occupational Therapy (Berger, McAteer, Schreier, & Kaldenberg, 2013), complete CATs for AOTA, and write articles for OT Practice.

Discussion

Lin et al. (2010) identified six important steps to evidence-based occupational therapy. In this article, we described a sequence of courses, embedded in an integrated curriculum, designed to ensure students are well prepared in each of these steps: (1) formulating a clinical question, (2) searching efficiently for the best available evidence, (3) critically analyzing the evidence for its validity and usefulness, (4) integrating the appraisal with personal clinical expertise and a client’s preferences, (5) evaluating one’s performance or outcomes of actions, and (6) disseminating and communicating knowledge. The skills and knowledge of EBP are taught as an integral part of sound clinical reasoning, and this perspective is reinforced throughout all the courses and learning experiences in the curriculum. Through repeated meaningful practice, students develop habits of reasoning and confidence in their skills that enable them to carry this approach forward into clinical practice.

This curriculum design assumes students will have the capacity to take ownership of their own learning and feel comfortable taking an active role not only in their own learning, but also the learning of their peers. Some students are not comfortable initially with this shift in power and push back when instructors do not “teach” in a traditional didactic format. This integrated approach also requires all instructors in the curriculum to communicate and be willing to adopt strategies and assignments that are vetted by the full faculty; some faculty may be unaccustomed to relinquishing sole control over their syllabi. However, within our department, the learning benefits for students and the opportunity to work toward a shared educational vision have garnered full faculty support for this collaborative approach.

This approach to teaching EBP may sound labor intensive. However, our experience has been that this carefully structured “facilitated” model is more enjoyable and less demanding than other traditional approaches to teaching research in the entry-level curriculum such as having students work on group research projects and teaching research methods in lecture courses. Although the design of the assignments, lessons, and in-class learning experiences required upfront work by faculty, once these elements were in place, the amount of work was substantially reduced. Each year, we “refine” the materials or methods to implement a new idea or new resource; however, the fundamental structure does not need to be replanned each year.

Similarly, because the assignments are very structured, grading them is also less intensive than more open-ended projects. Once the students have mastered the fundamental methods and skills through these structured experiences, they learn to apply them flexibly through the assignments in other courses. Rather than preparation time, the major demand of the courses is that instructors must be ready to “think on their feet” to respond to the unique questions and learning needs that a particular student group brings forward.

Limitations and Future Research

This curriculum description has several limitations. We describe one MSOT program with a particular curriculum design and do not have a control group for comparison or pre- and postmeasures of student performance to evaluate the effectiveness of the curricula. Although we present student survey data on satisfaction and perceived competence, these data do not necessarily equate with effectiveness. In addition, we do not know whether students’ view of EBP would differ if taught using another curriculum. Moreover, because students are not yet providing occupational therapy intervention on a routine basis, we have limited ability to address the fifth step to EBP, evaluating one’s performance or outcomes of actions (Lin et al., 2010). Further follow-up research is needed to examine graduates’ implementation of all six EBP steps once they are working as occupational therapy practitioners.

Implications for Occupational Therapy Education and Practice

The findings from this description of an integrated MSOT curriculum and a coordinated sequence of EBP courses suggest the following implications for occupational therapy education and practice:

EBP skills and habits are essential for designing effective interventions and collaboratively engaging clients in the intervention process. These skills and habits can be developed within the context of the professional academic program.
Once new graduates are working as occupational therapy practitioners, it is recommended that their supervisors encourage them to reflect on and evaluate their performance and the outcomes of their efforts to provide EBP.

Conclusion

This article has described how the BU occupational therapy department developed an integrated curriculum and facilitated learning approach to prepare students with the knowledge and habits necessary to provide client-centered and evidence-based intervention. Evaluation data suggest that students who graduate from the program view EBP as an integral part of the clinical process and have begun to adopt the habit of seeking, appraising, and applying descriptive, assessment, and intervention evidence to inform their practice decisions. To achieve AOTA’s Centennial Vision to become a “powerful, widely recognized, science-driven, and evidence-based profession” (AOTA, 2007, p. 613), it is crucial that we educate future practitioners to understand and communicate the evidence in ways that facilitate collaboration with clients. The curriculum described here provides one model to support our quest to prepare students to develop necessary reasoning habits to become evidence-based practitioners.

Acknowledgments

The BU faculty appreciates the contributions of former BU faculty members Linda Tickle-Degnen and Sharon Cermak to earlier versions of the EBP courses presented in this article.

References

American Occupational Therapy Association. (2007). AOTA’s Centennial Vision and executive summary. American Journal of Occupational Therapy, 61, 613–614. http://dx.doi.org/10.5014/ajot.61.6.613 [Article]

American Occupational Therapy Association. (2012). 2011 Accreditation Council for Occupational Therapy Education (ACOTE^®) standards. American Journal of Occupational Therapy, 66, S6–S74. http://dx.doi.org/10.5014/ajot.2012.66S6 [Article]

Association of American Colleges and Universities. (2002). Greater expectations: A new vision for learning as a nation goes to college. Washington, DC: Author.

Berger, S. (2014). Educating clients. In B. A. Boyt Schell, G. Gillen, & M. E. Scaffa (Eds.), Willard and Spackman’s occupational therapy (12th ed., pp. 353–363). Philadelphia: Lippincott Williams & Wilkins.

Berger, S., McAteer, J., Schreier, K., & Kaldenberg, J. (2013). Occupational therapy interventions to improve leisure and social participation for older adults with low vision: A systematic review. American Journal of Occupational Therapy, 67, 303–311. http://dx.doi.org/10.5014/ajot.2013.005447 [Article] [PubMed]

Boyt Schell, B. A., Gillen, G., & Scaffa, M. E. (Eds.). (2014). Willard and Spackman’s occupational therapy (12th ed.). Philadelphia: Lippincott Williams & Wilkins.

DeAngelis, T. M., DiMarco, T. G., & Toth-Cohen, S. (2013). Evidence-based practice in occupational therapy curricula. Occupational Therapy in Health Care, 27, 323–332. http://dx.doi.org/10.3109/07380577.2013.843115 [PubMed]

Egan, M., Dubouloz, C. J., & Vallerand, J. (1998). The client-centred evidence-based practice of occupational therapy. Canadian Journal of Occupational Therapy, 65, 136–143. http://dx.doi.org/10.1177/000841749806500303 [Article]

Halliwell, V. (2008). Challenging knowledge reproduction: Problem-based learning for evidence-based practice. British Journal of Occupational Therapy, 71, 257–259.

Jiang, L., & Elen, J. (2011). Instructional effectiveness of higher-order questions: The devil is in the detail of students’ use of questions. Learning Environments Research, 14, 279–298. http://dx.doi.org/10.1007/s10984-011-9095-x [Article]

Knowles, M. (1984). Andragogy in action: Applying modern principles of adult education. San Francisco: Jossey-Bass.

Lin, S. H., Murphy, S. L., & Robinson, J. C. (2010). Facilitating evidence-based practice: Process, strategies, and resources. American Journal of Occupational Therapy, 64, 164–171. http://dx.doi.org/10.5014/ajot.64.1.164 [Article] [PubMed]

Neistadt, M. E., Wight, J., & Mulligan, S. E. (1998). Clinical reasoning case studies as teaching tools. American Journal of Occupational Therapy, 52, 125–132. http://dx.doi.org/10.5014/ajot.52.2.125 [Article] [PubMed]

Nolinske, T., & Millis, B. (1999). Cooperative learning as an approach to pedagogy. American Journal of Occupational Therapy, 53, 31–40. http://dx.doi.org/10.5014/ajot.53.1.31 [Article] [PubMed]

Regmi, K. (2012). A review of teaching methods—Lecturing and facilitation in higher education (HE): A summary of the published evidence. Journal of Effective Teaching, 12, 61–76.

Sackett, D. L., Rosenberg, W. M., Muir Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. British Medical Journal, 312, 71–72. http://dx.doi.org/10.1136/bmj.312.7023.71 [Article] [PubMed]

Sackett, D. L., Rosenberg, W., Richardson, W. S., & Haynes, R. B. (1997). Evidence-based medicine: How to practice and teach EBM. Edinburgh, Scotland: Churchill Livingstone.

Straus, S. E., Richardson, W. S., Glasziou, P., & Haynes, R. B. (2005). Evidence-based medicine: How to practice and teach EBM (3rd ed.). Edinburgh, Scotland: Churchill Livingstone.

Taylor, S. H. (2011). Engendering habits of mind and heart through integrative learning. About Campus, 16, 13–20. http://dx.doi.org/10.1002/abc.20076 [Article]

Thomas, A., Saroyan, A., & Dauphinee, W. D. (2011). Evidence-based practice: A review of theoretical assumptions and effectiveness of teaching and assessment interventions in health professions. Advances in Health Science Education, 16, 253–276. http://dx.doi.org/10.1007/s10459-010-9251-6 [Article]

Thomas, A., Saroyan, A., & Snider, L. M. (2012). Evidence-based practice behaviours: A comparison amongst occupational therapy students and clinicians. Canadian Journal of Occupational Therapy, 79, 96–107. http://dx.doi.org/10.2182/cjot.2012.79.2.5 [Article]

Tickle-Degnen, L. (2014). Communicating evidence to clients, managers, and funders. In M. Law & J. MacDermid (Eds.), Evidence-based practice: A guide to practice (3rd ed., pp. 275–304). Thorofare, NJ: Slack.

Wilson, S. G. (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. Teaching of Psychology, 40, 193–199. http://dx.doi.org/10.1177/0098628313487461 [Article]

Table 1.

Accreditation Council for Occupational Therapy Education Standards (American Occupational Therapy Association, 2012)

Number	Standard
B.8.2.	Effectively locate, understand, critique and evaluate information, including the quality of research evidence.
B.8.3.	Use scholarly literature to make evidence-based decisions.
B.8.4.	Understand and use basic descriptive, correlational, and inferential quantitative statistics and code, analyze, and synthesize qualitative data.
B.8.5.	Understand and critique the validity of research studies, including designs (both quantitative and qualitative) and methodology.
B.8.6.	Demonstrate the skills necessary to design a scholarly proposal that includes the research question, relevant literature, sample, design, measurement, and data analysis.
B.8.7.	Participate in scholarly activities that evaluate professional practice, service delivery, and/or professional issues (e.g., Scholarship of Integration, Scholarship of Application, Scholarship of Teaching and Learning).
B.8.8.	Demonstrate basic skills necessary to write a scholarly report in a format for presentation or publication.

View Large

Table 1.

Accreditation Council for Occupational Therapy Education Standards (American Occupational Therapy Association, 2012)

Number	Standard
B.8.2.	Effectively locate, understand, critique and evaluate information, including the quality of research evidence.
B.8.3.	Use scholarly literature to make evidence-based decisions.
B.8.4.	Understand and use basic descriptive, correlational, and inferential quantitative statistics and code, analyze, and synthesize qualitative data.
B.8.5.	Understand and critique the validity of research studies, including designs (both quantitative and qualitative) and methodology.
B.8.6.	Demonstrate the skills necessary to design a scholarly proposal that includes the research question, relevant literature, sample, design, measurement, and data analysis.
B.8.7.	Participate in scholarly activities that evaluate professional practice, service delivery, and/or professional issues (e.g., Scholarship of Integration, Scholarship of Application, Scholarship of Teaching and Learning).
B.8.8.	Demonstrate basic skills necessary to write a scholarly report in a format for presentation or publication.

Table 2.

Evidence-Based Practice: Course, Focus, Objectives, and Assignments

Course	Focus	Objectives	Assignments
EBP I	Framing focused clinical questions; finding, appraising, applying, and communicating about descriptive and assessment evidence	Understand how evidence informs clinical reasoning Plan an effective strategy for getting to know a client and his or her needs by writing relevant, answerable questions related to the client’s needs Effectively search research literature and judiciously select relevant evidence Accurately interpret basic qualitative and quantitative research Communicate findings from research evidence with colleagues and clients	Read case studies with related research articles and weekly guiding questions to guide application of research findings to the case Search for, select, and justify relevant research related to a clinical scenario Communicate evidence from one article to colleagues Synthesize findings from two articles and describe how to communicate findings to parents described in case study Conduct in-class analysis of qualitative data Complete final exam based on application of two research articles to case study
EBP II	Appraising the internal, external, and statistical validity of evidence related to intervention effectiveness and predicting client outcomes	Develop PICO (P: patient, problem, or population; I: intervention; C: comparison, control, or comparator; O: outcomes) questions related to intervention and prediction Conduct efficient and effective searches of Internet, library, and other resources Critically examine procedures, measures, and statistics used in intervention studies, and discuss the impact on study internal, external, and statistical validity Synthesize results from research studies on the basis of analysis of research quality and client and contextual factors Communicate the results of a research appraisal using the CAT format Interpret syntheses of research to answer a clinical question Interpret predictive research evidence to evaluate factors that predict client outcomes Prepare a research proposal for an intervention study	Write two CATs to address a clinical question using assigned intervention articles Complete three search assignments identifying intervention, predictive, and systematic review evidence Critically appraise and synthesize a small body of literature (four articles) to address a clinical question Develop a research proposal to address a major limitation identified in a critical appraisal that affects the ability to fully answer a clinical question Complete final exam involving critical appraisal of a research article and application of findings
EBP III	Systematically generating evidence from one’s own practice to answer clinical questions about individual or group client outcomes using knowledge and skills about SSDs and program evaluation	Describe and apply program evaluation methodology to evaluate the outcomes of a community-based program Evaluate the rigor of single-subject research and use SSD research evidence to answer clinical questions Describe and apply single-subject research methodology to an evaluation of individual outcomes after occupational therapy intervention Prepare manuscripts and other materials to disseminate individual or program outcomes to occupational therapy practitioners, other professionals, agencies, and clients	Complete a CAT to address a clinical question using evidence from a SSD research study Apply understanding of internal, external, and statistical validity of SSD to investigate and report on a theory-driven intervention designed for a student Complete a unit exam based on application of critically appraising a SSD research article Prepare evidence-based recommendations for a community agency’s program evaluation process aligned with the program’s logic model

Note. CAT = critically appraised topic; EBP = evidence-based practice; SSD = single-subject design.

View Large

Table 2.

Evidence-Based Practice: Course, Focus, Objectives, and Assignments

Course	Focus	Objectives	Assignments
EBP I	Framing focused clinical questions; finding, appraising, applying, and communicating about descriptive and assessment evidence	Understand how evidence informs clinical reasoning Plan an effective strategy for getting to know a client and his or her needs by writing relevant, answerable questions related to the client’s needs Effectively search research literature and judiciously select relevant evidence Accurately interpret basic qualitative and quantitative research Communicate findings from research evidence with colleagues and clients	Read case studies with related research articles and weekly guiding questions to guide application of research findings to the case Search for, select, and justify relevant research related to a clinical scenario Communicate evidence from one article to colleagues Synthesize findings from two articles and describe how to communicate findings to parents described in case study Conduct in-class analysis of qualitative data Complete final exam based on application of two research articles to case study
EBP II	Appraising the internal, external, and statistical validity of evidence related to intervention effectiveness and predicting client outcomes	Develop PICO (P: patient, problem, or population; I: intervention; C: comparison, control, or comparator; O: outcomes) questions related to intervention and prediction Conduct efficient and effective searches of Internet, library, and other resources Critically examine procedures, measures, and statistics used in intervention studies, and discuss the impact on study internal, external, and statistical validity Synthesize results from research studies on the basis of analysis of research quality and client and contextual factors Communicate the results of a research appraisal using the CAT format Interpret syntheses of research to answer a clinical question Interpret predictive research evidence to evaluate factors that predict client outcomes Prepare a research proposal for an intervention study	Write two CATs to address a clinical question using assigned intervention articles Complete three search assignments identifying intervention, predictive, and systematic review evidence Critically appraise and synthesize a small body of literature (four articles) to address a clinical question Develop a research proposal to address a major limitation identified in a critical appraisal that affects the ability to fully answer a clinical question Complete final exam involving critical appraisal of a research article and application of findings
EBP III	Systematically generating evidence from one’s own practice to answer clinical questions about individual or group client outcomes using knowledge and skills about SSDs and program evaluation	Describe and apply program evaluation methodology to evaluate the outcomes of a community-based program Evaluate the rigor of single-subject research and use SSD research evidence to answer clinical questions Describe and apply single-subject research methodology to an evaluation of individual outcomes after occupational therapy intervention Prepare manuscripts and other materials to disseminate individual or program outcomes to occupational therapy practitioners, other professionals, agencies, and clients	Complete a CAT to address a clinical question using evidence from a SSD research study Apply understanding of internal, external, and statistical validity of SSD to investigate and report on a theory-driven intervention designed for a student Complete a unit exam based on application of critically appraising a SSD research article Prepare evidence-based recommendations for a community agency’s program evaluation process aligned with the program’s logic model

Note. CAT = critically appraised topic; EBP = evidence-based practice; SSD = single-subject design.

Table 3.

Student Satisfaction With Evidence-Based Practice Courses

Year	Class Size	Students Extremely or Very Satisfied With Courses, n (%)
2010	47	40 (85.1)
2011	42	40 (95.2)
2012	49	42 (85.7)
2013	48	40 (83.3)

View Large

Table 3.

Student Satisfaction With Evidence-Based Practice Courses

Year	Class Size	Students Extremely or Very Satisfied With Courses, n (%)
2010	47	40 (85.1)
2011	42	40 (95.2)
2012	49	42 (85.7)
2013	48	40 (83.3)

Table 4.

Student Use of Evidence Type During Fieldwork Placement, 2010–2012 (N = 253 Placements)

	Students Using Evidence Type, n (%)
Fieldwork Placement	Descriptive Evidence	Assessment Evidence	Intervention Evidence
Self-study	218 (86)	142 (56)	202 (80)
Collaboration with team	185 (73)	121 (48)	121 (48)
Collaboration with client, family, or caregivers	96 (38)	63 (25)	121 (48)
Share with payers, insurance companies, or case managers	30 (12)	20 (8)	15 (6)
In-service presentation to staff	114 (45)	40 (16)	119 (47)
Educational materials or protocol development	83 (33)	18 (7)	89 (35)

View Large

Table 4.

Student Use of Evidence Type During Fieldwork Placement, 2010–2012 (N = 253 Placements)

	Students Using Evidence Type, n (%)
Fieldwork Placement	Descriptive Evidence	Assessment Evidence	Intervention Evidence
Self-study	218 (86)	142 (56)	202 (80)
Collaboration with team	185 (73)	121 (48)	121 (48)
Collaboration with client, family, or caregivers	96 (38)	63 (25)	121 (48)
Share with payers, insurance companies, or case managers	30 (12)	20 (8)	15 (6)
In-service presentation to staff	114 (45)	40 (16)	119 (47)
Educational materials or protocol development	83 (33)	18 (7)	89 (35)