The ability to drive an automobile is often an important factor associated with reintegration into the community and the workplace for people with a disability (Legh-Smith, Wade, & Hewer, 1986; Noreau, Dion, Vachon, Gervais, & Laramee, 1999). Although the freedom to drive increases a person’s independence and autonomy, driving is also associated with an increased risk of accidents that result in both property damage and bodily injury. Driving is a complex activity that requires the integration of many cognitive and perceptual–motor behaviors that can increase the impact on task performance, especially under varying environmental conditions.

Michon (1985)  described the driving activity using a hierarchical structure of three levels:

1. The strategic level occurs before getting behind the wheel and includes such tasks as planning the route of a trip in advance.

2. The tactical level occurs during driving and includes behaviors and decisions related to different situations that occur on the road, such as adapting the speed of the vehicle to road conditions or speed limits or deciding whether to pass a vehicle.

3. The operational level involves the basic abilities to drive a car, such as braking, steering, and shifting.

Although behaviors such as poor judgment and impulsivity influence strategic and tactical decisions, the operational level is mainly affected by inadequate visual scanning of traffic and environment, problems in spatial perception and orientation, poor tracking, slowness in acting, and confusion when more complex actions have to be carried out (van Zomeren, Brouwer, & Minderhoud, 1987). These problems are most frequently found in people who have sustained brain damage. Because perceptual and cognitive problems can interfere with safe driving habits (Sivak, Olson, Kewman, Won, & Henson, 1981), it is likely that people in this group, when driving, might be at higher risk for their own safety as well as the safety of pedestrians and other drivers, particularly when confronted with heavy traffic (Antrim & Engum, 1989; Dubinsky et al., 1991; LeBlang, 1979; Quigley & DeLisa, 1983; Shore, Gurgold, & Robbins, 1980; Sivak et al., 1981). Moreover, these people are often unable to correctly evaluate their driving capabilities and tend to overestimate them (Dubinsky et al., 1991; Golper, Rau, & Marshall, 1980).

Physicians, when examining clients with perceptual or cognitive deficits, evaluate the skills known to affect driving performance. If the physician is uncertain about a client’s driving capabilities, he or she then refer these people to a specialist in driving evaluation, usually an occupational therapist (Korner-Bitensky, Bitensky, Sofer, Man-Son-Hing, & Gélinas, 2006; Korner-Bitensky, Coopersmith, Mayo, & Leblanc, 1990; Sprigle, Morris, Nowached, & Karg, 1995; Unsworth, Lovell, Terrington, & Thomas, 2005).

Occupational therapists administer both off-road and on-road tests to determine whether their clients are able to safely drive an automobile. Off-road assessments are used to obtain information on the client’s strengths and weaknesses to identify people who are not suitable for an on-road assessment and to identify areas that should be investigated in more depth during the on-road assessment (Unsworth et al., 2005).

According to one study (Korner-Bitensky et al., 2006), off-road assessments are most often conducted with clients presenting impairments resulting from either a cerebrovascular accident (CVA) or a traumatic brain injury (TBI). The Cognitive Behavioral Driver’s Inventory (CBDI) is an off-road evaluation that was developed specifically to evaluate the prerequisite skills for driving in clients with a neurological condition. It consists of a battery of standardized tests that evaluate the cognitive abilities deemed necessary for the safe operation of a motor vehicle. This information can then be used to help predict whether a person will pass or fail an on-road test (Engum & Lambert, 1990; Engum, Lambert, & Scott, 1990; Engum, Pendergrass, Cron, Lambert, & Hulse, 1988; Engum et al., 1989).

The CBDI assesses basic operational cognitive–behavioral skills (e.g., attention, cognitive control, perceptual quickness) and is therefore used frequently by professionals in driving evaluation services when evaluating people with cognitive and perceptual problems. It includes the administration of four computerized tasks as well as standardized tasks borrowed from other test batteries. The CBDI is often partially administered because evaluation tools from other batteries may be unavailable or because the occupational therapist is not sufficiently experienced or permitted to administer some of the required tests. For example, two subtests of the Wechsler Adult Intelligence Scale–Revised (WAIS–R; Wechsler, 1982) that are part of the CBDI (Picture Completion and Digit Symbol), are tests that can be purchased only by psychologists. Moreover, some of the equipment specified for use with the CBDI may not be available in the clinical setting. For example, the CBDI requires that the horizontal peripheral visual field be evaluated using the Keystone Periometer Field of Vision (an attachment to the Keystone Driver Vision Telebinocular [Keystone View, Reno, NV]). Currently, the Canadian province of Quebec licensing board requires the use of a different measure for the evaluation of peripheral visual field. Consequently, occupational therapists most often do not use the perimetry results when scoring the CBDI.

It is not known whether the omission of some tasks on the CBDI affects the concurrent validity of the test. Each task generates at least 1 of the 28 item scores on which the global CBDI score (General Driver’s Index [GDI28]) is based. Because the CBDI software automatically assigns a value of 50 (standardized average) to a missing value, the global score can be substantially increased or decreased by partial administration of the CBDI battery. Therefore, the primary objective of this study was to determine whether the partial administration of the CBDI, compared with administration of the entire CBDI, has a significant effect on its concurrent validity (sensitivity and specificity) and index of validity.

To be admitted to the programs, all clients had to meet the medical eligibility requirements for driving set by the provincial licensing authorities. All clients with either a CVA or a TBI, who were evaluated using both the CBDI (Version 2.0) and the on-road evaluation between September 1997 and March 2003 in one of the three public driving evaluation programs in the Montreal area, were included. The three driving programs are located in different rehabilitation establishments within the Montreal metropolitan region. However, all three establishments have staff with recognized expertise in driving evaluation and belong to a communication network that ensures that the programs are comparable in referral criteria, structure, and procedures. Participants were excluded if the interval between the administration of the CBDI and the on-road test exceeded 76 days.

This was a retrospective study, and scores from the CBDI evaluation (GDI28) and the on-road evaluation (pass, fail, recommendation of lessons) as well as clinical and demographic information were extracted from the medical charts. Both the CBDI and on-road evaluations were conducted by experienced occupational therapists. Because the administration of the CBDI varied across institution, the charts were retained from each institution if they fit the usual administration mode for that center. For example, the complete CBDI was typically administered at Center 1 (CBDIc); therefore, any files that were incomplete at this institution were not included in the sample. At Center 2, the two subtests of the WAIS–R (Picture Completion and Digit Symbol) were never administered (CBDI-w), whereas at Center 3, neither the WAIS–R nor the results of the right–left peripheral vision test were administered (CBDI-w-p). For the latter two centers, files were rejected if any other tests were missing from the CBDI data sheet.

The CBDI includes four computerized tasks and six manually administered psychometric tasks (Engum, Pendergrass, et al., 1988). Together, these tests measure a combination of cognitive skills necessary for driving and include attention, concentration, reaction time, rapid decision making, visual scanning, visual alertness, attention to detail, visual–motor coordination, and sequencing. The four computerized tasks are adapted from Bracy’s (1985)  Cognitive Rehabilitation Programs for people with CVA or TBI, and together they generate 20 of the 28 scores. The six manually administered tasks are borrowed from other tests WAIS–R Digit Symbol and Picture Completion [Weschler, 1982 ]; Halstead–Reitan Trail Making Tests A and B [Reitan, 1986 ]; brake reaction time; horizontal left and right peripheral field of vision or perimetry, as measured with the Keystone Periometer field of vision), and they generate seven scores. These scores are manually entered into the CBDI software. On the basis of the intraindividual variance of the 27 scores, the software calculates a supplementary score (scatter variance), a measure of within-subject variability. These 28 raw scores are then converted by the software into standard scores with a mean of 50.0 and a standard deviation of 10.0 (Engum & Lambert, 1990). If a manual test is not administered, a standard score of 50 is automatically given to the missing item. The mean standard score of these 28 items constitutes the GDI28. On the basis of studies by the CBDI’s creators, a predictive result is obtained for the on-road test: pass (GDI28 ≤ 47), borderline (GDI28 = 48–51), or fail (GDI28 ≥ 52).

Two studies have indicated that the CBDI was able to accurately predict on-road performance (Engum et al., 1989, 1990). Evaluation of criterion-related validity (n = 81) demonstrated that the passing and failing scores of the CBDI were able to predict the result of the on-road test; however, borderline scores (35%) were frequent (Engum et al., 1989). Other psychometric testing indicated a high internal consistency (Cronbach’s α = .956) and a strong correlation (r = .851, p < .001) between the results of the on-road test and the GDI28; a strong correlation was also found between the on-road test results and the number of items passed (Engum & Lambert, 1990; Engum et al., 1990; Engum, Lambert, Womac, & Pendergrass, 1988). Finally, norms have been established for participants who had principally a CVA or a TBI (Engum & Lambert, 1990; Engum, Lambert, et al., 1988; Lambert & Engum, 1990).

In the current study, the results of the GDI28 were classified into pass, fail, and borderline, as has been done in similar studies (Bouillon, Mazer, & Gélinas, 2006; Klavora, Heslegrave, & Young, 2000). To measure sensitivity, specificity, and the index of validity, data must be expressed as dichotomous entities (Feinstein, 1977). Therefore, the results were combined to form two groups, Pass and Fail, by combining the borderline results with the failures. This approach provides a conservative bias toward prediction of false-fail rather than toward false-pass for the on-road test but favors, in cases of doubt, the security of the driver and others. This decision was made to optimize the sensitivity of the test; although this method can lead to a decrease in the test’s specificity, preventing on-road tests that could be potentially dangerous was deemed a priority.

The on-road tests were performed by an experienced occupational therapist and directed by a driving evaluator. The driving circuit consisted of both city and highway driving (as a standard procedure) and took approximately 60 min to complete for people with a CVA and between 90 and 120 min for people with a TBI. For participants with physical impairments, the vehicle was equipped with adaptations such as a spinner knob and a left accelerator. An assessment form was completed during the on-road evaluation, enabling the occupational therapist to document the client’s strengths and weaknesses. The therapist rated the performance on the road using a checklist of maneuvers (left turns, right turns, lane changes) and behaviors (visual exploration, planning, awareness, decision making). Once the driving evaluation was completed, the occupational therapist and driving instructor reviewed the client’s driving behaviors, knowledge, applications of driving regulations, and ability to maneuver the vehicle safely. They then determined whether the client had passed or failed or required driving lessons. For the purposes of this study, participants who received a recommendation of lessons were considered a fail because at the time of evaluation they did not meet the criteria of being sufficiently safe to drive.

Additional information, including age, gender, diagnosis, date of diagnosis, and date of test administration, was ascertained from the medical chart.

Descriptive statistics were used to illustrate participant demographic factors as well as CBDI and on-road test results. A screening test of driving ability is designed to provide an accurate indication of which people would be classified as safe or unsafe drivers. To determine the ability of the CBDI to identify those who are safe or unsafe drivers, we used sensitivity, specificity, and the index of validity to validate this screening tool; these were calculated for each institution (Table 1).

The sensitivity of the CBDI is its ability to correctly detect people who fail the on-road test; it indicates the proportion of all true driving failures detected by the screening test. Sensitivity was calculated using the following formula (Feinstein, 1977):

The specificity, or the ability to accurately predict who passes the on-road test, provides the proportion of all true on-road passes identified by the CBDI. It was calculated as follows (Feinstein, 1977):

The index of validity is the percentage of those who pass and fail the on-road evaluation who were correctly predicted by the CBDI and is calculated using the following formula (Feinstein, 1977):

To determine whether partial administration of the CBDI has an effect on its concurrent validity, the data from Center 1 were transformed to simulate results of the partial administration of the CBDI at Centers 2 and 3, and the data from Center 2 were transformed to simulate the data of Center 3. For example, to simulate partial administration of the CBDI-w, the data from Center 1 were recalculated, substituting the standard scores for the two WAIS–R tasks with values of 50, and the GDI28 score was modified accordingly.

These three parameters—sensitivity, specificity, and index of validity—were calculated for each of the CBDI scenarios that were either administered or simulated (Table 2).

κ statistics were used to evaluate whether the index of validity or the sensitivity or specificity were different, depending on the various degrees of administration of the CBDI (CBDIc, CBDI-w, CBDI-w-p). To calculate the κ test, the categorized results (Table 1) were regrouped and dichotomized where necessary. For example, to compare the index of validity for the four scenarios of Center 1 (CBDIc, CBDI-w, CBDI-p, CBDI-w-p), the categories TP and TN from the CBDIc were collapsed to form one single true category, whereas those categorized into FP and FN were collapsed into one single false category. This dichotomization was repeated for the other three scenarios of the CBDI administration. κ tests were then calculated to compare the index of validity of the four scenarios. Dichotomization of data was also performed for comparisons of the sensitivity indexes (TP, FN) and for specificity (FP, TN).

Of 344 charts examined, 245 met the inclusion criteria. However, from Center 3, we noted that only 5 of 58 (9%) of the participants failed the on-road test. On further investigation, it was revealed that on the basis of the performance on other tests, the occupational therapists did not administer the CBDI to participants who demonstrated a poor potential for passing the on-road test, therefore inducing a positive bias in the results. Unfortunately, this situation was not known to the principal investigator (Josée Duquette) before the onset of the study and the development of the research protocol. However, on the basis of this discovery, the information from Center 3 was not included in the study.

Table 3 lists the demographic information for participants by institution, after eliminating the participants from Center 3. Three of four participants were men. The participants were evenly distributed between those with a TBI and those with a CVA. The average age of participants from Center 1 was older than those from Center 2, and the postdiagnostic testing interval was significantly shorter, although the time between the administration of the CBDI and the on-road test was similar.

The results of the CBDI and those from the on-road test are presented in Table 4. There was no significant difference in scores on both of these measures between the two centers. Those participants who were classified as borderline by the CBDI represent 20% at Center 1 and 25% at Center 2.

We also compared the results of the CBDI with the results of the on-road test (Table 5) for the three simulations of partial administration for Center 1 and for the partial administration (CBDI-w) and simulation (CBDI-w-p) for Center 2. For Center 1, substitution of default scores in place of the actual scores for only the WAIS–R almost never affected the CBDI classification. By contrast, when default scores were substituted in place of the actual perimetry scores (CBDI-p and CBDI-w-p), the number of people who were classified as borderline on the CBDI increased and the number of people classified as passing decreased, regardless of whether the two WAIS–R default or actual scores were included. For Center 2, the comparison could be made only between two scenarios: the CBDI-w and CBDI-w-p. For this center, the use of default scores for actual perimetry scores mainly decreased the number classified as pass.

In addition, the modifications in Table 5 reflect global changes rather than the modifications that occurred on an individual basis, which are more numerous. For example, the simulated CBDI-w for Center 1 resulted in the alteration of eight (7%) results, not four, as it would appear in Table 5. For 4 of the participants, correspondence between the CBDI and the on-road test got worse: for 3 of the participants who failed the on-road test, the simulated GDI28 scores were lowered, resulting in two fails becoming borderline and one borderline becoming pass; for another person who passed the on-road test, a pass on the CBDI became a borderline when simulated. By contrast, correspondence between the CBDI and the on-road test for the other four cases was improved: The simulated CBDI for 1 participant who failed the road test changed from pass to borderline, whereas the simulated GDI28 scores for the other 3 participants who passed the road test changed from borderline to pass. The relations between the two scores were modified for the two other simulations as follows: For the CBDI-p, 16 individual scores were modified (14%; 7 less and 9 better correlated); for the CBDI-w-p, 17 scores were modified (15%; 10 less and 7 better correlated). For Center 2, the substitution of the default perimetry scores (CBDI-w-p) resulted in a modification of results in 11 cases (15%; 9 less well correlated and 2 better correlated).

Table 6 presents the index of validity of the CBDI (passes and fails on road correctly predicted by the CBDI) and its sensitivity (fails correctly predicted) and its specificity (passes correctly predicted) for each administration simulation of the two centers. In Center 1, the index of validity is 72.1%; the sensitivity, 51.2%; and the specificity, 85.3%, for the complete administration of the CBDI; thus, most on-road passes are correctly predicted, but the fails are not identified with as much accuracy. In effect, only half of the participants who failed the on-road test were correctly predicted by the CBDI, even though the borderline classifications were also included in this group. The simulated scenarios CBDI-w and CBDI-p demonstrate a tendency toward a decrease in specificity and an increase in sensitivity with respect to the CBDIc, although these changes did not significantly modify the three indicators (p < .001). For Center 2 (CBDI-w), the index of validity was 65.8%; the sensitivity, 54.2%; and the specificity, 71.2%. Indeed, the on-road test result of pass was correctly predicted 7 of 10 times, whereas approximately half of the fails were not predicted by the CBDI. The simulation of the partial CBDI-w-p by substituting 50 for the perimetry scores worsened the index of both validity (59.2%) and specificity (59.6%) but not of sensitivity, which was increased, as for Center 1. However, none of these changes was significant (p < .001).

To compare our results with Engum and Lambert’s (1990)  original CBDI validation on 180 participants, we placed their borderline participants into the Fail group and calculated sensitivity and specificity. Because the CBDI borderline results in our study (21%) are in agreement with those found in the final standardization (26%) for the CBDI (Engum & Lambert, 1990), we felt comfortable with this adjustment.

The index of validity for the Engum and Lambert (1990)  data was calculated at 80%. By contrast, our results were lower (72%) and more similar to those obtained in studies of people with CVA (Klavora et al., 2000) and with central nervous system problems (Bouillon et al., 2006), which reported values of 66% and 72%, respectively. We calculated a sensitivity of 93% for the Engum and Lambert data, whereas for our study, the sensitivity (i.e., the number of the road test failures that were predicted correctly by the CBDIc for Center 1) was only 51%. By contrast, our results are comparable to those of Bouillon et al. (2006), in which 62% of the road test failures were predicted correctly. We calculated a specificity of 80% for the Engum and Lambert data, a result that is equal to that of Bouillon et al. (2006)  and lower than that calculated in our study for the CBDIc (85%).

These diverse outcomes could be caused by several factors, which include interstudy variability in road-test criteria for pass–fail, or by a difference in the composition of the participant sample, such as proportion of participants with a specific diagnosis, time since the diagnosis, laterality of brain injury, age, and gender.

These results also demonstrate the importance of establishing concurrent validity of a measurement tool when using it in an environment that differs from the one in which it was tested, especially when considering errors in prediction or in interpretation of CBDI results that could be crucial. Because these non-U.S. results show that the CBDI does not predict 28%–34% of the on-road tests accurately, it is our opinion that one should proceed with caution and not assume that the CBDI can accurately predict the results of an on-road driving test, even when administered in its complete form. Indeed, it could be potentially dangerous for both the driver and others, especially if the evaluator bases a conclusion about issuing a driver’s license to a person solely on passing the CBDI. Note that Engum and Lambert (1990)  stated, “[I]t is strongly recommended that all patients complete both the Driver Performance (Weaver, 1989) and an independently administered road test” (p. 41).

The lack of administration of the WAIS–R Picture Completion and the WAIS–R Digit Symbol rarely modified the CBDI results or those of concurrent validity. Therefore, when an evaluator does not have the ability to procure or use those tests, their exclusion should not greatly affect the index of validity or concurrent validity of the CBDI. The exclusion of the two tests not greatly affecting the index of validity or concurrent validity of the CBDI may be caused by the strong corrected item–total correlations (the item’s correlation with the GDI28 total score excluding that particular item) of the WAIS–R Picture Completion (.68) and WAIS–R Digit Symbol (.73).

By contrast, corrected item–total correlations for the right (.39) and left (.39) perimetry are low. The omission of the perimetry scores tended to increase the sensitivity and decrease the specificity; although not significant, this feature might reach significance with a larger sample size. Because sensitivity is of primary concern when evaluating clients for driving safety, the importance of administering the perimetry subtest is substantiated by the current data. The fact that the concurrent validity of the CBDI was influenced by the omission of the perimetry score might be explained by the fact that the software automatically attributes a standard score of 50 for scores that are missing in the CBDI test matrix, a process that could lead to an increase in the final summary score (GDI28). In fact, the participants who are permitted to perform the road test typically have a peripheral field of vision that is normal or slightly reduced; their standardized scores are typically between 47 and 34, corresponding to a visual field between 77° and 90° on each side. Consequently, the fact that a default score of 50 would be attributed when the left and right perimetry are missing could increase the GDI28 and modify the predictive score.

For example, a passing score that is close to the borderline zone (GDI28 = 47) could fall into the borderline zone, which in the current study was included with the fail classification; subsequently, if the person passed the road test, he or she would have been classified into the FP category, thereby lowering the specificity of the CBDI. By contrast, a borderline result on the CBDI could become a fail; when the person failed the road test, the result would become a TP and increase the sensitivity of the CBDI. Given that the fields of vision are consistently evaluated during the off-road test, it is critical that these measures be entered into the CBDI software matrix. Even if the field of vision tests are administered with an apparatus that is not as precise as the Keystone Periometer Field of Vision apparatus, the outcome will be more correct than substituting the default score of 50 that is used by the CBDI software.

This study has certain limitations. The results are limited by the small sample size. The small number of failures on the road test from Center 3 necessitated the retraction of participants from that site, thereby reducing the size of the global sample and the possibility of making comparisons between institutions. In addition, the interinstitution differences with respect to age, postdiagnostic elapsed time, failure rate for road tests, and mode of administration of the CBDI may have affected the indicators of concurrent validity with respect to the different scenarios of partial administration of the CBDI.

In the current study, the results of the CBDI were compared with outcomes from the on-road test. The road tests used in the two centers were comparable, and the procedures were standardized; however, the routes were different and not validated scientifically, and the environmental conditions (temperature, road conditions, high-risk conditions) were not controlled. These parameters could have influenced the index of validity and the indicators of concurrent validity, although they reflect the reality of the clinical milieu. The protocol of this study was not blind, because the CBDI and road tests were administered by the same occupational therapist. The therapist’s knowledge of results of the CBDI could have engendered a bias of interpretation for the road test to confirm the results of the CBDI. Because this study was retrospective, however, the evaluators were not aware of the study’s goals when they were administering the tests. Thus, the results cannot be biased toward the goal of the study, and the index of validity and concurrent validity measures were therefore not artificially inflated by the therapists’ desire to please the researchers. In addition, this study was initiated at the request of the therapists at the clinical sites, who were not convinced of the reliability of the CBDI in predicting road tests. Indeed, they were not using it as a predictive tool, but rather used it to assess a participant’s behavior and disabilities.

In this retrospective study, we observed that the CBDI, even when administered completely, was unable to predict >50% of the road test failures for people with a CVA or TBI. Thus, it is our opinion that the CBDI should be used as a complement and not a substitution for a road test. With respect to his caveat, it should be noted that the authors of the CBDI similarly recommended that all patients complete an independently administered on-road test because it allows the examiner to observe the patients’ ability to integrate operational and tactical decision-making skills while driving a motor vehicle (Engum, Pendergrass, et al., 1988).

The partial administration of the CBDI can affect the predictive results. The concurrent validity of the CBDI, in particular, can be affected by the omission of the peripheral vision tests but only minimally by omitting the WAIS–R Picture Completion or WAIS–R Digit Symbol tests. Other studies are needed to evaluate whether the CBDI differs according to the diagnosis, the side of the brain lesion in the case of people with CVAs, and the time since diagnosis.

We thank the Quebec Rehabilitation Research Network for financing the summer student bursary and the Centre de recherche interdisciplinaire en readaptation du Montréal metropolitain for financial support. We also thank François Lemieux, Jacques Audet, Helene Bergeron, Johanne Lavallee (Centre de readaptation Lucie-Bruneau), Louise Bouillon (Jewish Rehabilitation Hospital), and Julie Lamoureux. The results of this study were partially presented during the 6th National Workshop for Driver Rehabilitation Specialists, Edmonton (Canada), May 2004.