Establishing Criterion Validity of the Useful Field of View Assessment and Stroke Drivers’ Screening Assessment: Comparison to the Result of On-Road Assessment

Stacey George; Maria Crotty

doi:10.5014/ajot.64.1.114

Free

Research Article | January 2010

Establishing Criterion Validity of the Useful Field of View Assessment and Stroke Drivers’ Screening Assessment: Comparison to the Result of On-Road Assessment

Stacey George; Maria Crotty

Author Affiliations

Article Information

American Journal of Occupational Therapy, January/February 2010, Vol. 64, 114-122. doi:10.5014/ajot.64.1.114

Abstract

OBJECTIVES. We sought to determine the criterion validity of the Useful Field of View (UFOV) assessment and Stroke Drivers’ Screening Assessment (SDSA) through comparison to the results of on-road assessment.

METHOD. This was a prospective study with people with stroke. Outcome measures used were UFOV, SDSA, and the results of on-road assessment.

RESULTS. Both the results on UFOV (Divided Attention subtest, p < .01; Selective Attention subtest, p < .05) and SDSA (p < .05) were significantly related to the recommendation from on-road assessment. The Divided Attention subtest of the UFOV had the highest sensitivity value (88.9%).

CONCLUSIONS. UFOV and SDSA are valid assessments of driving ability for stroke. The Divided Attention subtest of the UFOV can guide decision making of occupational therapists in stroke driver rehabilitation and in determining those who require further assessment on road because they pose a safety risk. Screening assists people with stroke to decide whether they are ready to have an on-road assessment.

Stroke is a significant cause of disability in the older population (Australian Institute of Health and Welfare, 2004) and leads to an increased relative risk of crash while driving (Sagberg, 2006). Recently, there has been an increased survival rate and longevity after stroke, which has resulted in increasing numbers of people with perceptual and cognitive impairments who wish to resume driving (Korner-Bitensky, Bitensky, Sofer, Man-Son-Sing, & Gelinas, 2006). People with stroke have a range of deficits that may influence their driving ability, including reduced visual scanning, attention, information-processing speed, and visuospatial skills (Fisk, Owsley, & Mennemeier, 2002; Lundqvist, Gerdle, & Ronnberg, 2000; Simms, 1985).

In Australia, medical practitioners, including rehabilitation physicians, are required to make recommendations regarding the licensing of people after stroke. These possible recommendations include (1) the person is not medically fit and therefore is not recommended to return to driving, (2) the person is medically fit and requires an assessment of on-road ability to determine his or her safety, and (3) the person is medically fit and is recommended to return to driving (Austroads, 2003).

An assessment of on-road driving ability is used to determine an individual's functional ability to drive. Medical practitioners generally refer to occupational therapy driving rehabilitation programs, which use an array of predriver screening tools (Korner-Bitensky et al., 2006; Unsworth, Lovell, Terrington, & Thomas, 2005). To determine evidence-based guidelines on which occupational therapy clinicians could base decision making regarding driving after stroke, the research related to the validation of these predriver screening tools needs examination. Several different outcome measures have been used as the criteria for validating predriving assessment tools, including on-road assessment (Bouillon, Mazer, & Gelinas, 2006), team recommendation (Akinwuntan et al., 2006), and driving cessation (Fisk et al., 2002). On-road evaluation has high face validity (Lundberg, Caneman, Samuelsson, Hakamies-Blomqvist, & Almkvist, 2003) and remains the criterion standard on which the predictive value of tests can be measured (Fox, Bowden, & Smith, 1998).

A systematic review of assessments predicting driving ability after stroke recommended that when investigating the validity of predriver screening tools, cutoff scores with validated psychometric properties including sensitivity, specificity, and positive and negative predictive values (Marshall et al., 2007) are required. In the context of driving, the following explanations apply: (1) sensitivity is the ability of a test to predict those who fail an on-road evaluation, (2) specificity is the ability of the test to predict those who pass the on-road assessment, (3) positive predictive value is the proportion of those who fail the screening test and on-road assessment of all people who failed the screening assessment, and (4) negative predictive value is the proportion who passed the screening and on-road assessment of all people who passed the screening assessment.

The primary clinical interest regarding driving screening tools is the sensitivity (Bouillon et al., 2006) because of the safety risk these drivers potentially create on the road. Those who fail a screening test, which has a high sensitivity, should undergo further assessment in the form of an on-road evaluation. Only a small number of studies investigating the validity of predriving assessment tools in stroke have reported sensitivity. The highest values is 84.2% (Nouri & Lincoln, 1993) with the Stroke Drivers’ Screening Assessment (SDSA), which consists of a combination of table top tests, and 87.5% (Myers, Ball, Kalina, Roth & Goode, 2000) with the Useful Field of View (UFOV) assessment, a computer-based test of attention.

The SDSA (Lincoln, Radford, & Nouri, 2004) consists of three tests that contribute to the final pass–fail classification. First, the Dot Cancellation Test consists of a sheet of A4-size paper with rows of groups of three, four, or five dots. The participant is instructed to cross out all groups of four dots, and results are expressed as time taken, number of misses, and number incorrect or false alarms.

Second, the Compass Test requires the placement of cards on a matrix of 4 × 4 squares, as illustrated in Figure 1. Four directions, one for each row, are aligned along the side of the matrix, and four other directions, one for each column, are aligned along the top of the matrix. The stimulus cards depict a roundabout joining eight roads on which two cars are traveling on two roads. The participant is required to place the stimulus card in the square corresponding (1) to the intersection between the row corresponding to the direction of travel of one car and (2) to the column corresponding to the direction of travel of the second car. Twenty-seven cards are provided, of which only 15 can be correctly placed. Maximum time for completion is 5 min. One point is given for each correctly placed car; scoring on each card ranges from 0 to 2 and up to 32 for the test as a whole.

Figure 1.

The Compass Test from the Stroke Drivers Screening Assessment.

From Lincoln, N. B., Radford, K. A., & Nouri, F. M. (2004). Stroke Drivers Screening Assessment: Revised Manual (p. 9). Nottingham, England: University of Nottingham. Copyright © 2004 by the University of Nottingham. Reproduced with permission.

Full Size | Slide (.ppt)

Third, the Road Sign Recognition Test consists of 12 cards with pictures of different traffic situations, such as children crossing and a railway crossing. The participant is given 19 cards with traffic signs and is required to match the appropriate signs with the pictures. The time limit is 3 min, and 1 point is given for each correct answer.

The results from the three tests are recorded into a discriminant equation (Lincoln et al., 2004), which provides a predictive value for whether people would pass the road test and a predictive value for whether people will fail the road test. A higher value is taken to specify the likely outcome of the road test. The authors of the SDSA developed the discriminant equations through validation with the result of an on-road test (Nouri & Lincoln, 1992, 1993), which was chosen because it produced the highest predictive values. The discriminant equations are as follows: The classification function for passes is (Dot cancellation, time × 0.012) + (Dot cancellation, false alarms × 0.216) + (Compass × 0.409) + (Road sign recognition × 1.168) – 13.79 (constant). The corresponding function for fails is (Dot cancellation, time × 0.017) + (Dot cancellation, false alarms × 0.035) + (Compass × 0.185) + (Road sign recognition × 0.813) – 10.042 (constant). The psychometric properties of the discriminant equations have been further examined and have been found to (1) have criterion validity by comparison with a general practitioners’ recommendation related to driving (Nouri & Lincoln, 1993), (2) have content validity (Radford & Lincoln, 2004) when compared with measures of attention and executive abilities, and (3) have test–retest reliability (Lincoln & Fanthome, 1994).

Further validation of the discriminant equations of the SDSA were performed in Scandinavia (Lundberg et al., 2003), and <70% of people were correctly classified compared with the outcome of an on-road test. Modified discriminant equations were developed that correctly classified 78% of people (Lundberg et al., 2003). The modified discriminant equations are as follows for passes: (Dot cancellation, time × 0.0298) + (Dot cancellation, misses × 0.1017) + (Compass × 0.3666) + (Road sign recognition × 1.0415) – 16.7757 (constant); for fails: (Dot cancellation, time × 0.0294) + (Dot cancellation, misses × 0.1563) + (Compass × 0.2607) + (Road sign recognition × 0.88780) – 15.1846 (constant). Similarly, as mentioned previously, the function producing the highest result indicates whether a pass or fail was recommended. The Nordic version of the SDSA (Lundberg et al., 2003) when validated with the result of an on-road assessment reported a low sensitivity value, 36.1%, used in stroke; however, this study had a higher specificity value as the main concern.

A more recent study (Akinwuntan et al., 2006) evaluating the SDSA found an association between the component tests but was unable to replicate the findings related to sensitivity of Nouri and Lincoln (1993) . However, methodological factors could have influenced these findings because Akinwuntan et al. (2006) used only the component tests of the SDSA and not the original discriminant equations recommended by the developers (Lincoln et al., 2004). The SDSA, using its original discriminant equations, requires further research to determine whether the results reporting high sensitivity (Nouri & Lincoln, 1993) can be replicated.

The second predriving screening test reporting high sensitivity values for stroke is the UFOV. This is a specially designed software program that presents visual stimuli on a large computer screen (Visual Awareness Inc., 2002). The useful field of view is defined as the area from which one can extract visual information in a brief glance without head or eye movement. The UFOV is a visual and cognitive assessment tool that analyzes three aspects of visual attention: processing speed, divided attention, and selective attention. The assessment is performed in a darkened room free of distraction.

Processing speed, the first task, requires the participant to identify a centrally located object, either a car or a truck. Participants must indicate that they saw a car or a truck by touching the appropriate image on the computer screen after each trial. Duration of object presentation is gradually decreased until the participant can no longer identify which of the two objects is presented; it ranges from 250 ms to 12.5 ms.

Divided attention, the second task, requires participants to identify the centrally located target and to locate a simultaneously presented peripheral target. The peripheral target appears randomly at any of 24 locations, representing all combinations of eccentricity and directions. Divided attention is tested at varying exposure durations, ranging from 240 ms to 40 ms. Time of response is the duration at which participants achieve 75% accuracy.

Selective attention, the third task, provides a measure of distractibility by having participants repeat the divided attention task with the addition of distracters in the field. The participant is presented with white triangles throughout the screen to evaluate his or her ability to differentiate the peripheral target from the distracters. Results of each subtest are assigned to a risk category and an overall risk category determined by the software through the addition of the results on the three subtests.

The UFOV has demonstrated criterion validity when compared with crash risk (Ball, Owsley, Sloane, Roenker, & Bruni, 1993) and the result of an on-road test (Myers et al., 2000) and test–retest reliability (Edwards et al., 2005) with older people. People with stroke have diminished UFOV abilities (Fisk & Mennemeier, 2006; Fisk et al., 2002; Mazer, Sofer, Korner-Bitensky, & Gelinas, 2001). Results on the UFOV have been found to be related to on-road driving ability in stroke in a retrospective study (Akinwuntan et al., 2002) but not in a prospective study (Akinwuntan et al., 2006). These studies (Akinwuntan et al., 2002, 2006) did not report on individual UFOV test performance that has been found to be discriminative in predicting driving ability after brain injury (Novack et al., 2006). Individual subtests of the UFOV have been demonstrated to respond to retraining after stroke (Mazer et al., 2003).

Both the UFOV and the SDSA require further investigation to determine which is the most useful to guide clinical practice in driving after stroke. Our aim in this prospective study was to determine the criterion validity, including the sensitivity, specificity, and positive and negative predictive values, of the UFOV and SDSA, using a comparison with the result of on-road assessment.

Method

Research Design

This was a prospective correlational study of a cohort of drivers with stroke. Eligible participants were administered the UFOV and the SDSA. Descriptors, including gender, date of birth, diagnosis, onset of diagnosis, time since injury, Barthel index score (Shah, Vanclay, & Cooper, 1989), and driving experience, were recorded.

Participants

Participants were recruited at three rehabilitation services in Adelaide, South Australia. All participants met the following inclusion criteria: had a stroke, drove before onset, older than age 18, able to provide written informed consent, and assessed by the research therapist (Stacey George) as having adequate cognition to follow instructions to complete the assessments. In addition, all participants were recommended to have an on-road assessment by the rehabilitation physician if they were thought to have the potential to return to driving and there were concerns regarding their motor, perceptual, or cognitive abilities. Those referred for on-road assessment were required to have 120° horizontal vision (Austroads, 2003), be medically stable, and not require the use of complex modifications, such as a left-foot accelerator, to be eligible to participate in an on-road assessment.

Instruments

Once written informed consent had been obtained, the following assessments were performed by the researcher: (1) UFOV (Visual Awareness Inc., 2002) as described previously, (2) SDSA (Lincoln et al., 2004) as described previously, and (3) a standardized on-road driving assessment (Lister, 1998) performed within 6 weeks of completion of the UFOV and SDSA. The delay of performance between the on-road assessment and predriving assessments followed the procedures in place at the clinical institutions where the research was performed, which separated the predriving screen and on-road assessment.

The on-road assessment, which has been used in other validity studies (George, Clark, & Crotty, 2007), has a standardized route and scoring protocol used to record driving behaviors. Once the assessment was completed, the driving assessors, a driving instructor, and a driver-trained occupational therapist determined recommendation, and participants were classified as either a pass (consisting of those who passed and those recommended for lessons) or a fail. Before the commencement of the study, we decided to classify those who were recommended for lessons in the pass group, because our experience in our driving clinic is that the majority of people who are recommended for lessons after stroke proceed to pass in future on-road assessments. The on-road driver assessors were blinded to the results of the UFOV and SDSA.

Statistical Analysis

Data were analyzed using the Statistical Package for Social Sciences Version 12.0 (SPSS Inc., Chicago). Analyses were descriptive, with the objective of detailing the range of scores on the assessments. Results are presented as tables of scores with percentages rounded to one decimal point and 95% confidence intervals.

Relationships between categorical variables for the stroke participants were tested using the chi-square test for categorical variables and independent samples t test and Mann–Whitney U test for continuous variables; p was set at <.05.

UFOV scoring categories in each subtest and overall risk category were combined on the basis of the numbers assigned to each category to allow analysis of the association with the outcome of the assessment of on-road driving ability. For analyses, UFOV categories were combined as follows:

Processing Speed (Subtest 1):
- Normal category—Normal central vision and processing speed and normal central vision but somewhat slowed processing speed (scores >0 but ≤60 ms)
- Difficulty category—Central vision loss or slowed processing speed/severe central vision loss or very slowed processing speed (scores of >60 but <500 ms)
Divided Attention (Subtest 2):
- Normal category—Normal (scores of >0 but <100 ms)
- Difficulty category—Some difficulty and severe difficulty (scores of ≥100 but <500 ms)
Selective Attention (Subtest 3):
- Normal category—Normal ability (scores of >0 but <350 ms)
- Difficulty category—Difficulty and severe difficulty (scores of ≥350 but <500 ms).

In Subtests 1, 2, and 3, the Normal categories were assigned a Pass outcome and the Difficulty categories a Fail outcome.

Overall Risk Category (as assigned by the UFOV software):
- Low and Very Low, which was assigned a Pass outcome
- Moderate, High, and Very High, which was assigned a Fail outcome.

Pass or fail on the UFOV and the SDSA were compared with the pass or fail recommendations on the on-road assessment. Specificity, sensitivity, positive and negative predictive values, and confidence intervals were calculated for those screening tests that were statistically significantly associated with the outcome of the on-road assessment.

Results

Participants recruited for the study (n = 66 [78.8% men]) included the diagnoses of 24 right-hemisphere stroke (36.4%), 37 left-hemisphere stroke (56.1%), and 5 other stroke (7.6%). The mean age was 65.9 ± 8.4 years, median days since stroke was 56 (range = 10–2,190), and mean years of driving experience was 47.3 (range = 5–76). The mean Barthel index score (Shah et al., 1989) was 92.6 (range = 42–100), indicating a high level of functional independence.

Scores on the SDSA were significantly associated with processing scores (χ²[2] = 25.85, p < .01), divided attention (χ²[2] = 23.22, p < .01), selective attention (χ²[2] = 30.30, p < .01), and overall risk category (χ²[2] = 29.56, p < .01) of the UFOV. Participants who passed the SDSA were more likely to be rated in the lower risk categories on the UFOV subtests.

Forty-three of the 66 participants who were recommended to participate in an on-road assessment did so. For the 23 (34.8%) who did not complete the on-road assessment, the reasons cited were as follows: geographic location (n = 9), modifications required (n = 2), and client did not consent (n = 12). A significant difference in age (t[52] = 2.13, p = .04), Barthel index score (t[52] = 2.06, p = .05), time since drove (Mann–Whitney U = 138, p = .03), time since stroke (Mann–Whitney U = 128, p = .02), and driving experience (Mann–Whitney U = 135, p = .03) was found between those who performed the on-road assessment and those who chose not to perform the on-road assessment. The participants who performed the on-road assessment were younger, had less driving experience, had a higher level of function as scored on the Barthel index, and had a greater period of time since their stroke and since they had driven than those who did not consent to participate in the on-road assessment.

The on-road assessment was performed a mean of 38 days (range = 12–41 days) after the performance of the SDSA and UFOV assessments. Of those participants who had an on-road assessment, 36 (83.7%) passed and 7 (16.3%) failed. No significant differences were found between the outcome groups (i.e., those who passed or failed the on-road assessment) in terms of demographics and personal factors, as summarized in Table 1.

Table 1.

Comparison Between Demographic and Personal Factors and the Outcome on the On-Road Assessment

Demographic/Personal Factors	On-Road Outcome
Age	t(39) = 0.15, p =.14
Gender	χ²(1) = 0.05, p =.83
Diagnoses	χ²(2) = 1.23, p =.54
Barthel Index Score	T(39) = 1.90, p = .18
Time since last drove	Mann–Whitney U = 108, p = .70
Time since stroke	Mann–Whitney U = 101, p =.53
Driving experience	Mann–Whitney U = 75.5, p =. 13

View Large

Table 1.

Comparison Between Demographic and Personal Factors and the Outcome on the On-Road Assessment

Demographic/Personal Factors	On-Road Outcome
Age	t(39) = 0.15, p =.14
Gender	χ²(1) = 0.05, p =.83
Diagnoses	χ²(2) = 1.23, p =.54
Barthel Index Score	T(39) = 1.90, p = .18
Time since last drove	Mann–Whitney U = 108, p = .70
Time since stroke	Mann–Whitney U = 101, p =.53
Driving experience	Mann–Whitney U = 75.5, p =. 13

Results on the SDSA in relation to the on-road driving assessment are reported in Table 2. Recommendations on the SDSA, using the original discriminant equation (Lincoln et al., 2004), were significantly associated with the evaluation of on-road ability, indicating it is a valid assessment of driving ability. Lundberg et al. (2003) also recommended an alternate discriminant equation. The results if calculated according to the Lundberg et al. (2003) method, as reported in Table 2, were not significantly associated with the outcome of the on-road assessment.

Table 2.

On-Road Result by Results on Stroke Drivers’ Screening Assessment (SDSA) for Participants (Valid Percentages)

	On-Road Assessment Result
SDSA Equation	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
SDSA original equation (Lincoln, Radford & Nouri, 2004)
Fail (n = 13)	5 (71.4)	8 (22.2)	6.73*
Pass (n = 30)	2 (28.6)	28 (77.8)
SDSA Modified equation (Lundberg et al., 2003)
Fail (n = 10)	3 (42.9)	7 (19.4)	1.8
Pass (n = 33)	4 (57.1)	29 (80.6)

*p < .01

View Large

Table 2.

On-Road Result by Results on Stroke Drivers’ Screening Assessment (SDSA) for Participants (Valid Percentages)

	On-Road Assessment Result
SDSA Equation	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
SDSA original equation (Lincoln, Radford & Nouri, 2004)
Fail (n = 13)	5 (71.4)	8 (22.2)	6.73*
Pass (n = 30)	2 (28.6)	28 (77.8)
SDSA Modified equation (Lundberg et al., 2003)
Fail (n = 10)	3 (42.9)	7 (19.4)	1.8
Pass (n = 33)	4 (57.1)	29 (80.6)

*p < .01

Results of the analyses of UFOV and outcome of the on-road assessment listed in Table 3 indicate that the subtests of Divided Attention (Subtest 2) and Selective Attention (Subtest 3) were significantly associated with the evaluation of on-road ability.

Table 3.

On-Road Result by Results on Useful Field of View Assessment for Participants (Valid Percentages)

	On-Road Assessment Result
UFOV Test	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
Processing Speed (Subtest 1)
Fail (n = 5)	2 (28.6)	3 (8.3)	2.34
Pass (n = 38)	5 (71.4)	33 (91.7)
Divided Attention (Subtest 2)
Fail (n = 17)	6(85.7)	11(30.6)	7.46*
Pass (n = 26)	1 (14.3)	25 (69.4)
Selective Attention (Subtest 3)
Fail (n =7)	3(42.9)	4(11.1)	4.33**
Pass (n = 36)	4(57.1)	32(88.9)
Overall risk category
Fail (n =9)	3 (42.9)	6 (16.7)	2.43
Pass (n = 34)	4(57.1)	30(83.3)

*p < .05.

**p <.01.

View Large

Table 3.

On-Road Result by Results on Useful Field of View Assessment for Participants (Valid Percentages)

	On-Road Assessment Result
UFOV Test	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
Processing Speed (Subtest 1)
Fail (n = 5)	2 (28.6)	3 (8.3)	2.34
Pass (n = 38)	5 (71.4)	33 (91.7)
Divided Attention (Subtest 2)
Fail (n = 17)	6(85.7)	11(30.6)	7.46*
Pass (n = 26)	1 (14.3)	25 (69.4)
Selective Attention (Subtest 3)
Fail (n =7)	3(42.9)	4(11.1)	4.33**
Pass (n = 36)	4(57.1)	32(88.9)
Overall risk category
Fail (n =9)	3 (42.9)	6 (16.7)	2.43
Pass (n = 34)	4(57.1)	30(83.3)

*p < .05.

**p <.01.

An examination of the sensitivity and specificity values of those tests that were significantly associated with the on-road outcome are reported in Table 4. Divided Attention (Subtest 2) of the UFOV had the highest sensitivity (85.7%) and Selective Attention (Subtest 3) of the UFOV had the highest specificity (88.9%). The Divided Attention task correctly classified 77.5% of participants when compared with the pass/fail classifications from the on-road driving assessment

Table 4.

Results of the On-Road Assessment and Useful Field of View (UFOV) Assessment and Stroke Drivers’ Screening Assessment (SDSA) Test Results

	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value	Predictive Ability
UFOV assessment
Divided Attention (Subtest 2; %)	85.7	69.4	35.3	96.2	77.5
95% CI	59.8–100	54.3–84.5	21.0–49.6	90.5–100
Selective Attention (Subtest 3; %)	42.9	88.9	42.9	88.9	65.9
95% CI	6.2–79.6	78.7–99.1	28.2–57.6	79.5–98.3
SDSA test original equation	71.4	77.8	38.5	93.3	74.6
95% CI	37.9–100.0	64.3–91.3	24.0–53.0	85.8–100.0

View Large

The sensitivity of the SDSA was found to be 71.4%, which is lower than that found for the divided attention (Subtest 2) of the UFOV. On the SDSA, a borderline category is recommended for marginal scores (i.e., those in which the difference between pass and fail scores is ± 0.49) to indicate that more assessment is required, including another on-road assessment (Lincoln et al., 2004). This was the case for two participants who failed the SDSA but passed the on-road assessment. If we adjust their scores to a pass, then the SDSA specificity value of 77.8%, reported in Table 4, would increase to 83.3%.

Discussion

The results of the study indicate that the UFOV (Subtests 1 and 2 and overall risk category) and the SDSA are valid tests of driving ability for stroke. Both the SDSA and UFOV measure related abilities; however, the Divided Attention task (Subtest 2) of the UFOV had the highest sensitivity, or identified those likely to fail (85.7%), and correctly classified 77.5% of the participants compared with the pass–fail classifications from the on-road driving assessment. This high sensitivity value is important for practice because it can guide clinicians in correctly predicting those who require further assessment on road (Bouillon et al., 2006). Interestingly, Subtest 2 of the UFOV has also been demonstrated to be the most strongly related to on-road performance, compared with the other UFOV subtests, in traumatic brain injury (Novack et al., 2006) and has been found to respond to retraining after stroke (Mazer et al., 2003).

Moreover, Subtest 3 of the UFOV had the highest specificity value. In addition, 11 of 17 participants who failed Subtest 2 of the UFOV passed the on-road assessment, resulting in a positive predictive value of 35.3%. Therefore, the results indicate that the probability of passing the on-road assessment after failing Subtest 2 is 64.7%, indicating that a higher number of patients would be forwarded for on-road assessment and then would pass. However, this is the lesser requirement of a driving screening tool because these drivers will pass the on-road test without disqualification. This low positive predictive value for Subtest 2 of the UFOV is a consequence of the low fail rate in the on-road assessment.

Even though the SDSA was also found to be a valid test of driving ability after stroke, the sensitivity value (71.4%) was not high enough to guide clinical decisions on the likelihood of failure in the on-road evaluation. In comparison to other studies, this sensitivity value was lower than that reported in one study (84.2%; Nouri & Lincoln, 1993) but higher than reported in another (36%; Lundberg et al., 2003) evaluating the validity of SDSA. The study (Lundberg et al., 2003) reporting this very low sensitivity value altered the original discriminant equations used to classify results into pass or fail and had a priority of achieving a higher specificity value. In all studies concerning the SDSA, as with the results of this study, the specificity values reported were higher than the sensitivity values.

The sensitivity value reported for the UFOV Subtest 2 (85.7%) is higher than the majority of other sensitivity values reported for other predriving assessment tools for stroke in the literature (Bouillon et al., 2006; Korner-Bitensky et al., 2000; Korner-Bitensky & Sofer, 1998; Lundberg et al., 2003; Mazer, Korner-Bitensky, & Sofer, 1998; Nouri & Lincoln, 1993; Schanke & Sundet, 2000). Two other studies evaluating a cognitive test battery achieved higher sensitivities (90%, Galski, Bruno, & Ehle, 1993; 92%, McKenna, 1998); however, both studies involved a large number of tests and included diagnoses other than stroke.

The differences in psychometric properties of the UFOV and SDSA reported in this study from other studies may be because of differences in methodology, including the classification of pass or fail from the on-road evaluation. In this study, those who were recommended lessons as a result of their on-road assessment were classified as pass, because we decided that it was clinically meaningful to rate as pass those suitable for driving rehabilitation because they had the potential to return to safe driving, which has precedence in a study evaluating older drivers (Kay, Bundy, Clemson, & Jolly, 2008). Other studies (Lundberg et al., 2003) have a borderline pass group and a borderline fail group, indicating those who passed or failed a second driving test. The outcome of a second driving test for those who were recommended lessons was not included in our study. Requiring lessons as a recommendation from the on-road assessment is sometimes classified into the fail group (Bouillon et al., 2006).

Inclusion criteria for published studies also varied in comparison to this study, with some studies including participants requiring modifications to the vehicle for the assessment of on-road ability (Akinwuntan et al., 2002, 2006; Mazer et al., 1998; Mazer et al., 2003; Nouri & Lincoln, 1992; Schanke & Sundet, 2000). This inclusion criterion would lead to a higher number of people recommended to fail the on-road assessment, with a greater proportion being caused by physical limitations, which could potentially lower the sensitivity value.

Limitations

A limitation of this study is the small sample sizes, particularly the subset of those recommended as a fail on the on-road assessment, which is a similar limitation of other studies (Akinwuntan et al., 2002). In Australia, on-road assessments can only be performed with those people who are recommended to have one by their medical practitioner (Austroads, 2003). It could be assumed that medical practitioners generally recommend people for on-road assessments when they are likely to pass. Moreover, wide confidence intervals around sensitivity and specificity scores result from small sample sizes. Despite these small sample sizes, they were sufficient to achieve significant findings.

On clinical application of these findings, other limitations need to be considered. First, the delay between the performance of prescreening assessments and on-road assessment demonstrates ecological validity because it reflects the clinical procedures in the real world at the institution where the research was conducted; however, this may not be relevant in other places.

Future Research

This study found that after undergoing an occupational therapy predriving assessment, a large number of people did not proceed with an on-road assessment (18%), even though this was recommended. Those who did not proceed were significantly older, had a shorter time since stroke, and had a lower level of function and therefore would be at a greater risk of failing the on-road assessment. This finding suggests that the predriving assessment assists people with stroke to determine whether they are ready to undergo the on-road assessment.

Determination of cutoff scores, and their psychometric properties, for the predriving assessment tools examined in this study can be used as a guide for clinical decision making related to driving intervention after stroke. The high sensitivity values of the UFOV (Subtest 2) point to their use in identifying those people requiring further assessment because they may pose a road safety risk. This further assessment could take the form of an on-road assessment by occupational therapists. The high specificity value related to the UFOV (Subtest 3) indicates that it may be clinically useful for determining the timing of participation in an on-road assessment to ensure the greatest likelihood of passing and preventing repeated assessments.

Validation of the findings reported in this study in larger prospective studies would confirm the applicability of the findings reported here in the clinical setting. Although the UFOV has established test–retest reliability (Edwards et al., 2005) with older people, this has not been examined with drivers after stroke. Also, the UFOV and SDSA need to be examined for sensitivity to clinical change. In stroke where recovery of function occurs, the validated predriving assessment tools may potentially be administered several times. This will allow a therapist to gauge the best time to conduct the on-road assessment. Moreover, as younger people with stroke age, they may require repeat assessments to determine whether further on-road assessments are necessary. This will ensure that they are driving safely as age-related changes affect driving abilities.

Research is required to determine whether performing the UFOV subtests in isolation yields the same findings as the results presented in this study. This could lessen the time required for occupational therapists to perform predriving assessments in stroke while still gaining the most valuable information on which to guide practice.

Conclusion

Both the SDSA and UFOV are valid tests of driving ability after stroke. The high sensitivity score of Subtest 2, Divided Attention, indicates the inclusion of the UFOV in an occupational therapist's predriving assessment to determine drivers requiring an on-road assessment because they are more likely to have reduced safety in driving. Predriving assessment plays an important role in people with stroke making a decision to proceed with an on-road assessment and is therefore important clinically.

Acknowledgments

This research was supported by grants from the Australian Research Council Linkage Project LP0454144 and the Flinders Institute for Health and Medical Research Competitive Infrastructure and Strategic Programs. We acknowledge the Repatriation General Hospital Occupational Therapy Department for assisting with data collection.

Akinwuntan, A. E., Feys, H., De Weerdt, W., Baten, G., Arno, P., & Kiekens, C. (2006). Prediction of driving after stroke: A prospective study. Neurorehabilitation and Neural Repair, 20(3), 417–423. [Article] [PubMed]

Akinwuntan, E. A., Feys, H., DeWeerdt, W., Pauwels, J., Baten, G., & Strypstein, E. (2002). Determinants of driving after stroke. Archives of Physical Medicine and Rehabilitation, 83(3), 334–341. [Article] [PubMed]

Australian Institute of Health and Welfare. (2004). Heart, stroke and vascular diseases—Australian facts 2004 (Cardiovascular Disease Series No. 22. AIHW Cat. No. CVD 27). Canberra, Australian Capital Territory: AIHW and National Heart Foundation of Australia.

Austroads. (2003). Assessing fitness to drive—For commercial and private vehicle drivers. Sydney, New South Wales, Australia: Auhor.

Ball, K., Owsley, C., Sloane, M. E., Roenker, D. L., & Bruni, J. B. (1993). Visual attention problems as a predictor of vehicle crashes in older drivers. Investigative Ophthalmology and Visual Science, 34(11), 3110–3123. [PubMed]

Bouillon, L., Mazer, B., & Gelinas, I. (2006). Validity of the Cognitive Behavioural Driver's Inventory in predicting driving outcome. American Journal of Occupational Therapy, 60, 420–427. [Article] [PubMed]

Edwards, J. D., Vance, D. E., Wadley, V. G., Cissell, G. M., Roenker, D. L., & Ball, K. K. (2005). Reliability and validity of Useful Field of View test scores as administered by personal computer. Journal of Clinical and Experimental Neuropsychology, 27(5), 529–543. [Article] [PubMed]

Fisk, G. D., & Mennemeier, M. (2006). Common neuropsychological deficits associated with stroke survivors’ impaired performance on a useful field of view test. Perception and Motor Skills, 102(2), 387–394. [Article]

Fisk, G. D., Owsley, C., & Mennemeier, M. (2002). Vision, attention, and self-reported driving behaviours in community-dwelling stroke survivors. Archives of Physical Medicine and Rehabilitation, 83(4), 469–477. [Article] [PubMed]

Fox, G. K., Bowden, S. C., & Smith, D. S. (1998). On-road assessment of driving competence after brain impairment: Review of current practice and recommendations for a standardised examination. Archives of Physical Medicine and Rehabilitation, 79, 1288–1296. [Article] [PubMed]

Galski, T., Bruno, R. L., & Ehle, H. E. (1993). Prediction of behind-the-wheel driving performance in patients with cerebral damage: Discriminant function analysis. American Journal of Occupational Therapy, 47, 391–396. [Article] [PubMed]

George, S., Clark, M., & Crotty, M. (2007). Development of the Adelaide Driving Self-Efficacy Scale. Clinical Rehabilitation, 21, 56–61. [Article] [PubMed]

Kay, L., Bundy, A., Clemson, L., & Jolly, N. (2008). Validity and reliability of the on-road driving assessment with senior drivers. Accident Analysis and Prevention, 40, 751–759. [Article] [PubMed]

Korner-Bitensky, N., Bitensky, J., Sofer, S., Man-Son-Sing, M., & Gelinas, I. (2006). Driving evaluation practices of clinicians in the United States and Canada. American Journal of Occupational Therapy, 60, 428–434. [Article] [PubMed]

Korner-Bitensky, N., Mazer, B. L., Sofer, S., Gelinas, I., Meyer, M. B., Morrison, C., et al. (2000). Visual testing for readiness to drive after stroke. American Journal of Physical Medicine and Rehabilitation, 79(3), 253–259. [Article] [PubMed]

Lincoln, N. B., & Fanthome, Y. (1994). Reliability of the Stroke Drivers Screening Assessment. Clinical Rehabilitation, 8(2), 157–160. [Article]

Lincoln, N. B., Radford, K. A., & Nouri, F. M. (2004). Stroke Drivers Screening Assessment: Revised manual. Nottingham, England: University of Nottingham.

Lister, R. (Revised by Berndt, A., & George, S.). (1998). Driving assessment route and scoring key. In M. S. Clark, J. Hecker, E. Cleland, C. Field, A. Field, & M. Crotty (Eds.), Dementia and driving (pp. 33–46). Canberra, Australian Capital Territory: Australian Transport Safety Bureau.

Lundberg, C., Caneman, G., Samuelsson, S. M., Hakamies-Blomqvist, L., & Almkvist, O. (2003). The assessment of fitness to drive after stroke: The Nordic Stroke Driver Screening Assessment. Scandinavian Journal of Psychology, 44(1), 23–30. [Article] [PubMed]

Lundqvist, A., Gerdle, B., & Ronnberg, J. (2000). Neuropsychological aspects of driving after a stroke—In the simulator and on the road. Applied Cognitive Psychology, 14, 135–150. [Article]

Marshall, S. C., Molnar, F., Man-Son-Hing, M., Blair, R., Brosseau, L., & Finestone, H. M. (2007). Predictors of driving ability following stroke: A systematic review. Topics in Stroke Rehabilitation, 14(1), 98–114. [Article] [PubMed]

Mazer, B., Korner-Bitensky, N., & Sofer, S. (1998). Predicting ability to drive after stroke. Archives of Physical Medicine and Rehabilitation, 79, 743–749. [Article] [PubMed]

Mazer, B., Sofer, S., Korner-Bitensky, N., & Gelinas, I. (2001). Use of the UFOV to evaluate and retrain visual attention skills in clients with stroke: A pilot study. American Journal of Occupational Therapy, 55(5), 552–557. [Article] [PubMed]

Mazer, B., Sofer, S., Korner-Bitensky, N., Gelinas, I., Hanley, J., & Wood-Dauphinee, S. (2003). Effectiveness of a visual attention retraining program on driving performance of clients with stroke. Archives of Physical Medicine and Rehabilitation, 84, 541–550. [Article] [PubMed]

McKenna, P. (1998). Fitness to drive: A neuropsychological perspective. Journal of Mental Health, 7, 9–18. [Article]

Myers, R. S., Ball, K., Kalina, T., Roth, D. L., & Goode, K. T. (2000). Relation of useful field of view and other screening tests to on-road driving performance. Perceptual and Motor Skills, 91, 279–290. [Article] [PubMed]

Nouri, F. M., & Lincoln, N. B. (1992). Validation of a cognitive assessment: Predicting driving performance after stroke. Clinical Rehabilitation, 6, 275–281. [Article]

Nouri, F. M., & Lincoln, N. B. (1993). Predicting driving performance after stroke. British Medical Journal, 307, 482–483. [Article] [PubMed]

Novack, T. A., Banos, J. H., Alderson, A. L., Schneider, J. J., Weed, W., Blakenship, J., et al. (2006). UFOV performance and driving ability following traumatic brain injury. Brain Injury, 20(5), 455–461. [Article] [PubMed]

Radford, K. A., & Lincoln, N. B. (2004). Concurrent validity of the Stroke Drivers Screening Assessment. Archives of Physical Medicine and Rehabilitation, 85, 324–328. [Article] [PubMed]

Sagberg, F. (2006). Driver health and crash involvement: A case-control study. Accident Analysis and Prevention, 38, 28–34. [Article] [PubMed]

Schanke, A. K., & Sundet, K. (2000). Comprehensive driving assessment: Neuropsychological testing and on-road evaluation of brain injured patients. Scandinavian Journal of Psychology, 41, 113–121. [Article] [PubMed]

Shah, S., Vanclay, F., & Cooper, B. (1989). Improving the sensitivity of the Barthel Index for stroke rehabilitation. Journal of Clinical Epidemiology, 42, 703–709. [Article] [PubMed]

Simms, B. (1985). Perception and driving: Theory and practice. British Journal of Occupational Therapy, 48, 363–366.

Unsworth, C. A., Lovell, R. K., Terrington, N. S., & Thomas, S. A. (2005). Review of tests contributing to the occupational therapy off-road driver assessment. Australian Occupational Therapy Journal, 52, 57–74. [Article]

Visual Awareness, Inc. (2002). UFOV user's guide. Birmingham, AL: Author. Retrieved from http://crag.uab.edu/VAI/PDF%20Pubs/UFOV_Manual_V6.0.6.pdf

Figure 1.

The Compass Test from the Stroke Drivers Screening Assessment.

Full Size | Slide (.ppt)

Table 1.

Comparison Between Demographic and Personal Factors and the Outcome on the On-Road Assessment

Demographic/Personal Factors	On-Road Outcome
Age	t(39) = 0.15, p =.14
Gender	χ²(1) = 0.05, p =.83
Diagnoses	χ²(2) = 1.23, p =.54
Barthel Index Score	T(39) = 1.90, p = .18
Time since last drove	Mann–Whitney U = 108, p = .70
Time since stroke	Mann–Whitney U = 101, p =.53
Driving experience	Mann–Whitney U = 75.5, p =. 13

View Large

Table 1.

Comparison Between Demographic and Personal Factors and the Outcome on the On-Road Assessment

Demographic/Personal Factors	On-Road Outcome
Age	t(39) = 0.15, p =.14
Gender	χ²(1) = 0.05, p =.83
Diagnoses	χ²(2) = 1.23, p =.54
Barthel Index Score	T(39) = 1.90, p = .18
Time since last drove	Mann–Whitney U = 108, p = .70
Time since stroke	Mann–Whitney U = 101, p =.53
Driving experience	Mann–Whitney U = 75.5, p =. 13

Table 2.

On-Road Result by Results on Stroke Drivers’ Screening Assessment (SDSA) for Participants (Valid Percentages)

	On-Road Assessment Result
SDSA Equation	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
SDSA original equation (Lincoln, Radford & Nouri, 2004)
Fail (n = 13)	5 (71.4)	8 (22.2)	6.73*
Pass (n = 30)	2 (28.6)	28 (77.8)
SDSA Modified equation (Lundberg et al., 2003)
Fail (n = 10)	3 (42.9)	7 (19.4)	1.8
Pass (n = 33)	4 (57.1)	29 (80.6)

*p < .01

View Large

Table 2.

On-Road Result by Results on Stroke Drivers’ Screening Assessment (SDSA) for Participants (Valid Percentages)

	On-Road Assessment Result
SDSA Equation	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
SDSA original equation (Lincoln, Radford & Nouri, 2004)
Fail (n = 13)	5 (71.4)	8 (22.2)	6.73*
Pass (n = 30)	2 (28.6)	28 (77.8)
SDSA Modified equation (Lundberg et al., 2003)
Fail (n = 10)	3 (42.9)	7 (19.4)	1.8
Pass (n = 33)	4 (57.1)	29 (80.6)

*p < .01

Table 3.

On-Road Result by Results on Useful Field of View Assessment for Participants (Valid Percentages)

	On-Road Assessment Result
UFOV Test	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
Processing Speed (Subtest 1)
Fail (n = 5)	2 (28.6)	3 (8.3)	2.34
Pass (n = 38)	5 (71.4)	33 (91.7)
Divided Attention (Subtest 2)
Fail (n = 17)	6(85.7)	11(30.6)	7.46*
Pass (n = 26)	1 (14.3)	25 (69.4)
Selective Attention (Subtest 3)
Fail (n =7)	3(42.9)	4(11.1)	4.33**
Pass (n = 36)	4(57.1)	32(88.9)
Overall risk category
Fail (n =9)	3 (42.9)	6 (16.7)	2.43
Pass (n = 34)	4(57.1)	30(83.3)

*p < .05.

**p <.01.

View Large

Table 3.

On-Road Result by Results on Useful Field of View Assessment for Participants (Valid Percentages)

	On-Road Assessment Result
UFOV Test	Fail (n = 7)	Pass (n = 36)	χ² (df = 1)
Processing Speed (Subtest 1)
Fail (n = 5)	2 (28.6)	3 (8.3)	2.34
Pass (n = 38)	5 (71.4)	33 (91.7)
Divided Attention (Subtest 2)
Fail (n = 17)	6(85.7)	11(30.6)	7.46*
Pass (n = 26)	1 (14.3)	25 (69.4)
Selective Attention (Subtest 3)
Fail (n =7)	3(42.9)	4(11.1)	4.33**
Pass (n = 36)	4(57.1)	32(88.9)
Overall risk category
Fail (n =9)	3 (42.9)	6 (16.7)	2.43
Pass (n = 34)	4(57.1)	30(83.3)

*p < .05.

**p <.01.

Table 4.

Results of the On-Road Assessment and Useful Field of View (UFOV) Assessment and Stroke Drivers’ Screening Assessment (SDSA) Test Results

	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value	Predictive Ability
UFOV assessment
Divided Attention (Subtest 2; %)	85.7	69.4	35.3	96.2	77.5
95% CI	59.8–100	54.3–84.5	21.0–49.6	90.5–100
Selective Attention (Subtest 3; %)	42.9	88.9	42.9	88.9	65.9
95% CI	6.2–79.6	78.7–99.1	28.2–57.6	79.5–98.3
SDSA test original equation	71.4	77.8	38.5	93.3	74.6
95% CI	37.9–100.0	64.3–91.3	24.0–53.0	85.8–100.0

View Large