Determining the Internal Validity of the Inventory of Reading Occupations: An Assessment Tool of Children’s Reading Participation

Lenin C. Grajo; Catherine Candler; Patricia Bowyer; Sally Schultz; Jennifer Thomson; Karen Fong

doi:10.5014/ajot.2016.017582

Abstract

The Inventory of Reading Occupations (IRO) is an assessment tool of children’s reading participation. In this study, we used Rasch methods to determine the internal validity of the IRO. Participants included 192 typical and struggling readers from kindergarten to third grade from five different states in the United States. We analyzed the fit of each of the items in the 17 reading categories, the test items in the three dimensions of reading participation, and the contexts of reading in the IRO. Analysis indicated that the IRO items support the Rasch model of unidimensionality. Analysis also indicated that 1 of the 30 test items can be revised to strengthen test validity. Moreover, the analysis also suggested that the IRO is more useful for first- to third-grade students. This study provides evidence of internal validity of a useful tool to assess children’s reading participation.

Reading is a complex construct, and it is difficult to capture what exactly is involved when a reader decodes words and understands the meaning of text (Hosp & Suchey, 2014). Reading comprises multidimensional subprocesses that include the understanding that symbols have meaning and the ability to decode these symbols to form words. Primarily, reading is a language skill, and reading disorders are traditionally evaluated from a language processing perspective (Swanson & Hoskyn, 1998). The symbols used in the writing systems of the world are represented by language units, and decoding these language units is an important problem for poor readers (Catts & Kamhi, 2005). Reading interventionists, therefore, assess reading disorders using a language processing perspective.

Commonly used assessments and approaches to remediate reading typically include addressing component language skills, word reading efficiency, comprehension, and fluency. A meta-analysis of reading interventions revealed the need to provide more holistic assessments and interventions to support children with reading difficulties (National Reading Panel & National Institute of Child Health and Human Development [NRP–NICHD], 2000). The NRP–NICHD’s (2000) study suggested that language-based training alone should not constitute a complete reading program and that there is a need to include other aspects such as motivation, engagement, interest, and attention to reading (pp. 2–6). Follow-up longitudinal studies have supported the NRP–NICHD’s meta-analysis, citing the need to address reading from more than one perspective (Al Otaiba & Fuchs, 2006).

Several other studies in education and cognitive psychology have supported the relationship among reading participation, motivation, and reading ability. Reading motivation has been found to be directly and positively related to amount of and engagement in reading and reading comprehension (De Naeghel, Van Keer, Vansteenkiste, & Rosseel, 2012). Higher positive attitudes in reading also yield higher academic achievement (Mihandoost, 2011), and children’s ability to choose what they read and when they read is related to reading frequency and perceptions of reading self-efficacy (Wigfield, Guthrie, Tonks, & Perencevich, 2004). There have been several reading assessment tools published in the education field, such as nonstandardized reading inventories, to support the need for a holistic approach to reading. However, many of these inventories still focus on the language components of reading (Nilsson, 2008) or are limited to assessing motivations for reading academic texts (Wigfield, Guthrie, & McGough, 1996). Few assessments consider the different dimensions of participation in reading as an occupation, which includes other reading materials that are part of daily living activities.

Reading can be understood from the perspective of occupational engagement and participation. When a child reads, the reader engages with a task object within a context, and many variables within this context influence participation (Grajo & Candler, 2014). According to Law (2002), participation in occupations has several dimensions. These dimensions include the person’s preferences and interests in activities, what he or she does, where and with whom, and how much enjoyment and satisfaction the person finds in participating in these activities (Law, 2002, p. 642). When Law’s perspective on participation is applied to reading participation, new avenues are opened for consideration to support currently used reading intervention methods and to provide a more holistic approach to addressing reading as suggested by the NRP–NICHD’s (2000) meta-analysis. Assessment of and intervention with reading from the perspective of occupational participation could have a positive impact on the approaches currently used to assist struggling readers.

The purpose of this study was to provide preliminary evidence on the internal validity of an assessment that presents reading as an occupation and measures children’s reading participation. The assessment is called the Inventory of Reading Occupations (IRO; Grajo, Candler, & Bowyer, 2014).

Method

Instrument

The IRO is a two-part interview and self-report assessment that is used to identify (1) what materials the child reads on the basis of 17 listed categories, (2) level of preference in reading various materials, (3) the child’s perception of mastery of reading materials, (4) the frequency with which the child reads these materials, (5) the contexts in which the child reads, (6) whom the child reads with, (7) resources available for reading participation, and (8) goals identifying reading materials the child wants to master. The IRO can be administered by occupational therapists, speech–language pathologists, reading specialists, and classroom and special education teachers to provide insight into a child’s reading participation or to assess the impact of therapeutic or educational intervention in reading participation. It can be administered to typical or struggling readers.

The IRO focuses on participation in reading rather than evaluation of reading skills as traditionally defined in the literature. The contents of the IRO are based on the theoretical premise that with increased challenge in the occupational environment (e.g., school, home, community), a child with reading difficulties is pressed to show increased mastery in a very challenging task (Grajo & Candler, 2014). Because of the child’s awareness of his or her reading difficulties, the child may show a variety of responses to reading participation. These responses may include avoidance, dislike, low self-esteem, and decreased perception of competence, which may result in decreased engagement in meaningful reading tasks. By measuring a child’s frequency of reading participation, perception of mastery of reading a variety of materials, and how much a child likes reading the material, the IRO might be useful in providing insights to help investigate whether decreased reading participation may be related to an actual reading skill difficulty.

The contents of the IRO were developed after (1) interviews and classroom observations of patterns of reading participation of 14 children with reading difficulties, (2) pilot testing of a beta version with children with reading difficulties, and (3) consultation with five experts in children’s literacy. The consultants had graduate degrees in education (language and literacy) and a wide range of experience (5–14 yr) teaching reading in public schools. The experts were also consulted on the terminologies used in the different reading categories of the IRO to ensure that children understand these terms.

After pilot testing, the test items were further developed after a review of other assessments of children’s occupational participation in the occupational therapy literature. Some of the assessments reviewed include the Pediatric Interest Profiles (Henry, 2000), a measure of children’s play and leisure participation; the Children’s Assessment of Participation and Enjoyment and Preferences for Activities of Children (King et al., 2004), a tool that measures six dimensions of children’s occupational participation; and the Short Child Occupational Profile (Bowyer, Ross, Schwartz, Kielhofner, & Kramer, 2005), a tool that gives a broad overview of a child’s occupational participation and analyzes skills and environments affecting occupational participation.

The IRO has two parts. The first part contains 17 categories of reading materials. Under each reading category are six questions that define dimensions of reading participation: preference, mastery, frequency, contexts and environments, social supports, and availability of resources (Figure 1). The second part is a goal-setting portion that asks the child to list five reading categories that he or she wants to be able to read well. This goal-setting portion of the IRO can potentially provide information to reading interventionists and families on the kinds of reading materials that can be used for intervention or education. At the time this study was conducted, the IRO did not have a total score sheet or a reading profiles score form. The scores given for each test item under each reading category initially aimed to provide descriptive data on reading participation. A reading profiles scoring system is currently under development.

Figure 1.

Categories of reading and dimensions of reading participation in the Inventory of Reading Occupations.

Full Size | Slide (.ppt)

Participants

A total of 192 children completed the IRO. Participants were recruited mainly from one private school (n = 90) and one public charter school (n = 50) in St. Louis and from various cities in four other states (n = 52). The participants were composed of students from kindergarten to third grade (kindergarten, n = 38; Grade 1, n = 59; Grade 2, n = 49; Grade 3, n = 46), with more boys than girls (boys, n = 101; girls, n = 91) and with more typical readers than children with reading difficulties (typical readers, n = 133; children with reading difficulties, n = 59).

The children recruited by study liaisons were a combination of children attending private and public schools. To be included in the study, the children needed to be enrolled in kindergarten to third grade (ages 5–9 yr). The children were typical or struggling readers as indicated by standardized or academic educational assessments previously administered by the school district. We included children with diagnoses of developmental dyslexia, attention deficit disorders, learning disabilities, and motor coordination disabilities, identified through parent report during the consent process. To make sure that the decreased reading participation was secondary to true dyslexia and not a major impact of other conditions, we excluded students with pervasive developmental disorders and neurological and intellectual disabilities from the study. Data about the ethnicity and specific academic and medical diagnoses of student participants were not included in the analysis and are not reported.

Data Collection

The institutional review boards of Saint Louis University and Texas Woman’s University granted approval for the study; we also had letters of support from the two elementary schools that served as primary research sites. The first author and three graduate research assistants performed group administration of the IRO to 140 students in kindergarten to third grade from the two elementary schools in St. Louis. Occupational therapy practitioners and speech–language pathologists practicing in the field were invited to be study liaisons. The study liaisons were recruited from workshops conducted by the first author in different cities in the United States to help recruit children to complete the IRO. The liaisons were also recruited to participate in a separate qualitative study to determine the clinical utility of the IRO. Twenty-five study liaisons completed the requirements and recruited 52 children who were included in the study.

Data Analysis

Following a quantitative design, in this study we used the Rasch model of measurement to determine the internal validity of the IRO. We chose to use the Rasch methods versus the traditional Classical Test Theory (CTT) methods as a preliminary means to measure the psychometric properties of the tool. The Rasch model uses sample-invariant item parameter estimation and has additive properties that are reported as areas of weakness of the more commonly used CTT methods (Hambleton & Jones, 1993). In an analysis comparing the use of Rasch and CTT, Magno (2009) found that (1) Rasch estimates of item difficulties did not change across samples compared with inconsistencies found using CTT, (2) difficulty indexes of tests were also more stable across different forms of tests than the CTT approach, and (3) Rasch methods provided more stable internal consistencies and construct validity estimates across samples than CTT methods (pp. 9–10). Rasch methods have been shown to be a powerful tool to determine construct and internal validity of assessments and not merely a support to psychometric properties that use CTT.

Fit statistics that use Rasch methods have been established as indicators of construct-irrelevant variance and construct underrepresentation, which determines construct and internal validity of an assessment tool (Baghaei, 2008). Moreover, according to Rasch analysis, items that fit the analysis are likely to be measuring the single dimension intended by the construct theory (Baghaei, 2008). Baghaei (2008) explained that the advantage of the Rasch model is the creation of a hypothetical unidimensional line and that test items analyzed that fall close to this hypothetical line contribute to the measurement of the single dimension defined in the construct theory. Rasch analysis has been determined to have an advantage over CTT methods to abstract equal units of measurement from raw data that can be estimated and used with confidence in many clinical measurements (Bond & Fox, 2007; McAllister, 2008).

Rasch analysis follows the principle of unidimensionality. By converting ordinal data into interval data, Rasch analysis is able to define estimates of person ability and test item difficulty into a measure of a single attribute (Bond & Fox, 2007). The unidimensionality principle that Rasch analysis creates indicates internal validity of a tool. Unidimensionality can be confirmed with Rasch methods in several ways. In this study, we used goodness-of-fit analysis and analysis of standardized residuals (Bond & Fox, 2007; Linacre, 2014). Goodness of fit in the Rasch model is an indicator of how well each test item fits within an underlying construct and supports unidimensionality of a tool. Analysis of standardized residuals may indicate distortions in data and convergence problems that are threats to internal and construct validities (Linacre, 2014). The residual value (expressed as standardized residuals) is the difference between the Rasch model’s theoretical expectation of item performance and performance actually encountered for that item in the data matrix (Bond & Fox, 2007).

We investigated the measurement properties of the IRO using the Many-Facet Rasch Measurement (MFRM) model (Linacre, 2014). MFRM refers to a class of models suitable for simultaneous analysis of multiple variables potentially having an impact on assessment outcomes (Eckes, 2011). From a Rasch perspective, various elements in an assessment interact to produce an observed outcome. These definable elements in an assessment that exert influence on an assessment process can be classified into facets (Linacre, 2002).

The data were entered in a spreadsheet and exported to FACETS (Version 3.71.4; Winsteps, Beaverton, OR). The scores entered in FACETS were the raw scores for each child as he or she responded to each of the items of the IRO. At the time of analysis, the IRO did not have a score sheet or a process of totaling scores to identify specific reading profiles. The raw data were composed of more than 35,000 data points. After a series of consultations with Rasch experts from the University of Illinois at Chicago, the data files (i.e., student ability, reading categories, reading dimensions, social contexts, physical contexts) were entered as five different facets for analysis. The multiple facets analyzed determined the choice of FACETS and MFRM as the more suitable Rasch software and model to use. Because of the amount of data points in each facet, Rasch expert consultants suggested that the data were too complex to run as one continuous analysis. The data files were then processed as three separate analyses looking at the goodness-of-fit analysis and analysis of residual values of (1) student abilities (student’s level of reading participation); the 17 reading categories; and the mastery, preference, and frequency reading dimensions of the IRO; (2) student abilities, 17 reading categories, and the physical contexts of reading; and (3) student abilities, 17 reading categories, and the social contexts of reading. The logarithmic conversion of data in FACETS was expressed in logits (log-odds units) as units of measurement (Bond & Fox, 2007).

Using FACETS, we reported two forms of fit statistics as χ² ratios called infit and outfit mean-squares (MnSqs). Outfit MnSq values are sensitive to unexpected observations by people on items that are relatively easy or very hard (Linacre, 2014). Infit MnSq values are sensitive to unexpected patterns of observation by people on items that are roughly targeted on them (Linacre, 2014).

Using FACETS, we also generated an analysis of standardized residuals (equivalent to principal-components analysis in the Winsteps software) and an analysis of unexpected responses by the students in various items of the IRO that may indicate distortions in the data. As a measure of reading participation, the IRO is considered unidimensional and internally valid when no more than 5% of the items fail to fit the Rasch model (Smith, 2002) after analysis of residuals. After the analyses of the residual values, we investigated and diagnosed test items and person ability estimates potentially causing misfit or dimensionality issues in the IRO.

Results

Goodness of Fit

For rating scale type tests, reasonable infit and outfit MnSq values should be within the 0.6–1.4 logits range (Wright & Linacre, 1994). Additionally, for each MnSq value, FACETS reported standardized MnSq values as standardized z (ZStd) scores. Like MnSq values, ZStd scores greater than 2.0 indicate great distortion in the measurement system.

Logit values of MnSq < 0.6, MnSq >1.4, and ZStd > 2.0 were used throughout the analyses as primary criteria for fit of items of the IRO with the Rasch unidimensional model. Items that are >1.4 logits were considered an underfit with the Rasch model and items that are <0.6 logits were considered an overfit. Underfitting items degrade the quality of ensuing measures and prompt researchers to analyze what went wrong in the assessment measurement (Bond & Fox, 2007). Overfitting items can lead to misleading conclusions that the quality of the assessment measure is better than what it intends to measure.

Figure 2 illustrates the vertical ruler and item map of student reading participation with the 17 categories of reading and the three dimensions of reading participation (preference, mastery, frequency). The figure illustrates the placement of student abilities, reading categories, and reading dimensions in the Rasch model of measurement expressed in logits. A vertical ruler indicates that the closer the items are to a 0-logit value, the better the fit in the Rasch model. The map of the interaction between student abilities and the different test items of the IRO indicate a general good fit in the Rasch unidimensionality model.

Figure 2.

Vertical ruler of student ability, reading categories, and reading dimensions of the Inventory of Reading Occupations.

Full Size | Slide (.ppt)

The results of the goodness-of-fit analysis indicated that 15 of the 17 reading categories, the three reading dimensions (mastery, preference, frequency), the three items of physical contexts (home, school, community), and four of five items of the social contexts of reading (reading with parents, reading with friends and classmates, reading with teachers, reading with other family members) fit the unidimensional Rasch model. Two of the reading categories showed underfit with the Rasch model (story books, outfit MnSq = 1.52; game consoles, outfit MnSq = 1.56). One of the social contexts test items, Reading on My Own, also showed underfit with the Rasch model (outfit MnSq = 1.87). We conducted an analysis of unexpected responses that may have contributed to the underfitting of the two reading categories and the social context item. In the reading categories analyses, 28% of the unexpected responses were observed from kindergarten participants. We investigated the impact of removing data from kindergarten participants on the overall fit of the reading categories test items of the IRO. When data from all kindergarten participants were removed, all 17 reading categories indicated good fit of the items (within the 0.6–1.4 logit value criteria) with the Rasch model.

We also conducted an analysis of unexpected responses in the social context items. The analysis revealed that 80% of the unexpected responses came from the Reading on My Own test item. We investigated the impact of removing the Reading on My Own item on the overall fit of the social contexts dimension with the Rasch model. When all data from the Reading on My Own item were removed, the data indicated that the remaining four social context items fit the Rasch model. Table 1 provides a summary of the fit statistics of the revised test items of the IRO.

Table 1.

Fit Analysis of Revised Inventory of Reading Occupations (IRO) Test Items

IRO Item	Measure	SE	Infit MnSq	ZStd	Outfit MnSq	ZStd
Reading categories
Magazines	0.27	0.05	0.82	−2.5	0.87	−1.2
Labels	0.30	0.04	0.88	−1.8	0.85	−1.7
E-readers	−0.15	0.05	0.99	0.0	0.86	−1.2
Computers	−0.03	0.04	0.87	−1.9	0.88	−1.2
Notebooks	−0.17	0.04	1.01	0.1	0.90	−0.9
Comic books	0.08	0.05	1.02	0.2	0.91	−0.8
Television	0.09	0.05	1.01	0.1	0.94	−0.5
Bulletin boards	0.17	0.05	0.97	−0.4	0.95	−0.4
Game boards	0.03	0.04	1.00	0.0	0.96	−0.3
Posters	0.03	0.04	0.99	0.0	0.99	−0.1
Subject books	−0.05	0.05	1.03	0.4	1.02	0.2
Signs	−0.08	0.04	1.09	1.2	1.03	0.3
Story books	−0.20	0.04	1.11	1.4	1.12	1.1
Chalkboard	−0.12	0.04	1.14	1.9	1.11	1.0
Worksheets	−0.01	0.04	1.01	0.1	1.20	2.1
Game consoles	−0.07	0.05	1.14	1.7	1.20	1.7
Cellphone	0.07	0.05	1.16	1.9	1.32	2.5
Reading dimensions
Mastery	−0.12	0.02	0.90	−3.4	0.92	−1.7
Preference	−0.12	0.02	0.93	−1.2	0.98	−0.3
Frequency	0.24	0.02	1.13	5.2	1.23	5.7
Physical contexts
Community	1.09	0.05	0.95	−2.5	0.92	−2.7
School	0.15	0.04	0.92	−5.2	0.94	−2.5
Home	−1.24	0.05	1.13	4.5	1.33	5.2
Social contexts
Teachers	0.38	0.05	0.97	−1.3	0.89	−1.8
Other family members	0.10	0.05	1.01	0.3	1.01	0.3
Parents	−0.93	0.05	0.99	−0.2	1.03	0.8
Friends and classmates	0.46	0.05	1.04	1.4	1.04	0.6

Note. MnSq = mean square; SE = standard error; ZStd = standard z.

View Large

Table 1.

Fit Analysis of Revised Inventory of Reading Occupations (IRO) Test Items

IRO Item	Measure	SE	Infit MnSq	ZStd	Outfit MnSq	ZStd
Reading categories
Magazines	0.27	0.05	0.82	−2.5	0.87	−1.2
Labels	0.30	0.04	0.88	−1.8	0.85	−1.7
E-readers	−0.15	0.05	0.99	0.0	0.86	−1.2
Computers	−0.03	0.04	0.87	−1.9	0.88	−1.2
Notebooks	−0.17	0.04	1.01	0.1	0.90	−0.9
Comic books	0.08	0.05	1.02	0.2	0.91	−0.8
Television	0.09	0.05	1.01	0.1	0.94	−0.5
Bulletin boards	0.17	0.05	0.97	−0.4	0.95	−0.4
Game boards	0.03	0.04	1.00	0.0	0.96	−0.3
Posters	0.03	0.04	0.99	0.0	0.99	−0.1
Subject books	−0.05	0.05	1.03	0.4	1.02	0.2
Signs	−0.08	0.04	1.09	1.2	1.03	0.3
Story books	−0.20	0.04	1.11	1.4	1.12	1.1
Chalkboard	−0.12	0.04	1.14	1.9	1.11	1.0
Worksheets	−0.01	0.04	1.01	0.1	1.20	2.1
Game consoles	−0.07	0.05	1.14	1.7	1.20	1.7
Cellphone	0.07	0.05	1.16	1.9	1.32	2.5
Reading dimensions
Mastery	−0.12	0.02	0.90	−3.4	0.92	−1.7
Preference	−0.12	0.02	0.93	−1.2	0.98	−0.3
Frequency	0.24	0.02	1.13	5.2	1.23	5.7
Physical contexts
Community	1.09	0.05	0.95	−2.5	0.92	−2.7
School	0.15	0.04	0.92	−5.2	0.94	−2.5
Home	−1.24	0.05	1.13	4.5	1.33	5.2
Social contexts
Teachers	0.38	0.05	0.97	−1.3	0.89	−1.8
Other family members	0.10	0.05	1.01	0.3	1.01	0.3
Parents	−0.93	0.05	0.99	−0.2	1.03	0.8
Friends and classmates	0.46	0.05	1.04	1.4	1.04	0.6

Note. MnSq = mean square; SE = standard error; ZStd = standard z.

Analysis of Standardized Residuals

Table 2 provides a summary of the analysis of residual values of the different test items of the IRO after all kindergarten data have been removed from the reading categories and social contexts items (as previously done in the goodness-of-fit analysis). According to Linacre (2014), when the data parameters are successfully estimated during analysis of standardized residuals, the mean residual value is 0.0. When the data fit the Rasch model, the mean of the standardized residuals is expected to be near 0.0, and the sample standard deviation is expected to be near 1.0. The results of the analysis showed that the standardized residuals and standard deviations indicate minimal distortions in the data and no issues with convergence (mean of residuals near 0, and standard deviation near 1.0). Of the mean 7,085 item responses used in the estimation of fit to the Rasch model in the test items of the IRO, between 71 and 100 responses (1.0%–1.4%) indicated unexpected responses on the basis of analysis of standardized residual values. The amount of unexpected responses indicated minimal distortions and no convergence issues in the test items of the tool. Lack of convergence is an indication that the data do not fit the model well because there are too many poorly fitting observations (Linacre, 1987). When there are no convergence issues, the data fit the unidimensional Rasch model and support internal validity of the tool (Smith, 2002).

Table 2.

Measurable Data Summary of the Different Test Items of the Inventory of Reading Occupations

Category	Score	Expected	Residual Value	Standard Residual
Reading dimensions
M (n = 5,993)	3.86	3.86	0.00	0.00
Population SD	1.40	0.74	1.20	1.00
Sample SD	1.40	0.74	1.20	1.00
Physical contexts
M (n = 6,998)	0.55	0.55	0.00	−0.02
Population SD	0.50	0.25	0.43	1.03
Sample SD	0.50	0.25	0.43	1.03
Social contexts
M (n = 9,340)	0.36	0.36	0.00	0.00
Population SD	0.48	0.25	0.41	1.00
Sample SD	0.48	0.25	0.41	1.00

Note. M = mean; SD = standard deviation.

View Large

Table 2.

Measurable Data Summary of the Different Test Items of the Inventory of Reading Occupations

Category	Score	Expected	Residual Value	Standard Residual
Reading dimensions
M (n = 5,993)	3.86	3.86	0.00	0.00
Population SD	1.40	0.74	1.20	1.00
Sample SD	1.40	0.74	1.20	1.00
Physical contexts
M (n = 6,998)	0.55	0.55	0.00	−0.02
Population SD	0.50	0.25	0.43	1.03
Sample SD	0.50	0.25	0.43	1.03
Social contexts
M (n = 9,340)	0.36	0.36	0.00	0.00
Population SD	0.48	0.25	0.41	1.00
Sample SD	0.48	0.25	0.41	1.00

Note. M = mean; SD = standard deviation.

Discussion

This study is a preliminary investigation of the psychometric properties of the IRO. Because educational literature suggested the need to assess reading from a holistic perspective, in this study we explored the internal validity of an occupation- and participation-focused assessment of reading. With Rasch methods, the goodness-of-fit analyses of the different IRO items showed a good fit with the Rasch unidimensional model, suggesting strong internal validity. The analysis of standardized residuals indicates no convergence problems of the different IRO items and supports the fit analyses results to establish the internal validity of the tool. With MFRM, the results of the study indicate that, collectively, the reading dimensions, physical contexts, and social contexts of reading items of the IRO measure the level of a child’s reading participation on the basis of the different reading categories that the child identifies that he or she reads. Except for the Reading on My Own item, the study also indicates that the different test items of the IRO may be useful for clinicians in determining a profile of reading participation of a child who may be an average or a struggling reader.

A possible profile that may be gleaned from the IRO is a profile of a child with a limited repertoire of reading materials but who indicates high levels of mastery, preference, and frequency of reading. Another reading profile is that of a child who has a wide range of materials that he or she is interested in reading but who shows decreased levels of mastery, frequency, and limited contexts of reading participation. The vertical ruler and item map of student abilities (levels of reading participation) indicated not only the fit of the test items with the theoretical model but also the level by which the different test items of the IRO demonstrate a continuum of reading participation in both typical readers and children with reading difficulties.

The results of the Rasch analyses also provided insights on how to modify the tool to demonstrate better fit with the Rasch model. First, almost a third of unexpected responses in the reading categories were from kindergarten participants and caused some underfitting measures in the analyses, which might indicate that kindergarteners were overinflating, guessing, or just randomly responding to the IRO items. It might also indicate that the current version of the IRO is too structured and challenging for kindergarteners and therefore would be more useful for children in first to third grades. After the data from the kindergarten participants were removed, the reading categories items of the IRO showed better fit with the Rasch model. Second, data from one of the items from the social contexts test items (Reading on My Own) were removed. After the data were removed, the fit analysis indicated lesser distortions in the data and overall better fit with the Rasch model. This might indicate the need to further define or clarify the test item.

As a preliminary study, the results of this investigation were limited to the analysis of the internal validity of the tool and the production of recommended revisions on the validation version of the IRO. We did not include analysis of rating scale functioning and test reliability studies conducted as part of a bigger research project. The impact of suggested revisions on the IRO’s measurement properties cannot be determined or assumed in this current study. Additional revisions and retesting of the IRO are needed to develop a tool that can provide a perspective on children’s participation in reading occupations. In this study, we also used a limited sample size with the majority of the students attending private school. Caution must be taken in generalizing the results of this study, and sampling needs to be expanded to have more robust analyses. Moreover, in this study, we were limited to using the Rasch methods to analyze the measurement properties of the tool. Classical test methods can be used to further support and confirm findings from this study.

Implications for Occupational Therapy Practice

In this study, we established the preliminary measurement properties of an occupation-based and participation-focused assessment of children’s reading. The results of this study may have several implications for occupational therapy practice:

Occupational therapists can support the assessment of children’s reading from the perspective of participation, which may include identifying contexts of reading; availability of social supports and resources; and the frequency, amount, and preferences for reading of children.
The IRO appears to be a valid tool on the basis of the results of this study. The tool can be used by occupational therapists, speech–language pathologists, reading specialists, and classroom teachers for children from first to third grades to gather information about the reading participation of typical readers and children with reading difficulties. Reading participation is essential in performance of many daily activities and fulfillment of important life roles.
The IRO may be able to provide a continuum of reading participation on the basis of a child’s preference, mastery, and frequency of reading various materials and supports available in different contexts of reading. This profile of reading participation may provide insights into how occupational therapists, reading interventionists, classroom teachers, and parents can support children with or without reading difficulties. This reading profile from the IRO may also provide a holistic perspective on reading that can potentially respond to a gap in the reading assessment and intervention literature.

Implications for Occupational Therapy Research

The IRO supports the American Occupational Therapy Association and American Occupational Therapy Foundation (2011) Research Agenda that promotes the development of assessments that contribute to the body of evidence of the profession. In this study, we provided insights on future directions for the development of research related to the IRO. Implications for occupational therapy research are as follows:

Data gathered from this study can be used and analyzed with CTT methods to support the preliminary psychometric properties identified in the Rasch analysis.
To support the clinical utility of the IRO and its ability to measure changes in children’s reading participation, clinicians can administer the IRO in a group study of typical readers and children with reading difficulties receiving traditional classroom literacy instruction or reading intervention. The IRO can be administered at the beginning and end of a semester, school year, or intervention period to measure changes in reading participation as a result of reading instruction or intervention.
Because the validation version of the IRO appears to be most useful for first to third graders, developing a preschool and kindergarten version and a version for children in later elementary levels of schooling can be explored.

Acknowledgments

We thank Everett Smith (University of Illinois at Chicago) for his guidance and support in the data analysis. We also thank Meagan Paulsen, Sarah Friedman, and Kathleen Goldman (graduate research students from Saint Louis University) for their assistance in the study. Finally, we thank all the occupational therapists, speech–language pathologists, reading specialists, literacy experts, teachers, parents, and students who shared their valuable time to participate in this study.

References

Al Otaiba, S., & Fuchs, D. (2006). Who are the young children for whom best practices in reading are ineffective? An experimental and longitudinal study. Journal of Learning Disabilities, 39, 414–431. http://dx.doi.org/10.1177/00222194060390050401 [Article] [PubMed]

American Occupational Therapy Association & American Occupational Therapy Foundation. (2011). Occupational therapy research agenda. American Journal of Occupational Therapy, 65(Suppl.), S4–S7. http://dx.doi.org/10.5014/ajot.2011.65S4 [Article]

Baghaei, P. (2008). The Rasch model as a construct validation tool. Rasch Measurement Transactions, 22, 1145–1146.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Erlbaum.

Bowyer, P., Ross, M., Schwartz, O., Kielhofner, H., & Kramer, J. (2005). The Short Child Occupational Profile (SCOPE) (Version 2.0). Chicago: University of Illinois at Chicago.

Catts, H. W., & Kamhi, A. G. (2005). Defining reading disabilities. In H. W. Catts & A. G. Kamhi (Eds.), Language and reading disabilities (pp. 50–71). Boston: Allyn & Bacon.

De Naeghel, J., Van Keer, H., Vansteenkiste, M., & Rosseel, Y. (2012). The relation between elementary students’ recreational and academic reading motivation, reading frequency, engagement, and comprehension: A self-determination theory perspective. Journal of Educational Psychology, 104, 1006–1021. http://dx.doi.org/10.1037/a0027800 [Article]

Eckes, T. (2011). Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments. Frankfurt: Peter Lang.

Grajo, L., & Candler, C. (2014). Children with reading difficulties: How occupational therapy can help. OT Practice, 19(13), 16–17.

Grajo, L., Candler, C., & Bowyer, P. (2014). The Inventory of Reading Occupations. Unpublished assessment.

Hambleton, R. K., & Jones, R. W. (1993). Comparison of Classical Test Theory and Item Response Theory and their applications to test development. Educational Measurement: Issues and Practice, 12, 38–47. http://dx.doi.org/10.1111/j.1745-3992.1993.tb00543.x [Article]

Henry, A. (2000). Pediatric Interest Profiles: Surveys of play for children and adolescents. San Antonio: Therapy Skill Builders.

Hosp, J., & Suchey, N. (2014). Reading assessment: Reading fluency, reading fluently, and comprehension—Commentary on the special topic. School Psychology Review, 43, 59–68.

King, G., Law, M., King, S., Hurley, P., Rosenbaum, P., Hanna, S., . . . Young, N. (2004). Children’s Assessment of Participation and Enjoyment and Preferences for Activities of Children. San Antonio: Pearson.

Law, M. (2002). Participation in the occupations of everyday life. American Journal of Occupational Therapy, 56, 640–649. http://dx.doi.org/10.5014/ajot.56.6.640 [Article] [PubMed]

Linacre, J. (1987). Rasch estimation: Iteration and convergence. Rasch Measurement Transactions, 1, 7–8.

Linacre, J. M. (2002). Facets, factors, elements and levels. Rasch Measurement Transactions, 16, 880.

Linacre, J. M. (2014). FACETS Rasch measurement computer program user’s guide. Beaverton, OR: Winsteps.

Magno, C. (2009). Demonstrating the difference between Classical Test Theory and Item Response Theory using derived test data. International Journal of Educational and Psychological Assessment, 1, 1–11.

McAllister, S. (2008). Introduction to the use of Rasch analysis to assess patient performance. International Journal of Therapy and Rehabilitation, 15, 482–490. http://dx.doi.org/10.12968/ijtr.2008.15.11.31544 [Article]

Mihandoost, Z. (2011). Quantitative study on reading attitude: A meta-analysis of quantitative result. Nature and Science of Sleep, 10, 75–82.

National Reading Panel & National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Washington, DC: National Institute of Child Health and Human Development.

Nilsson, N. L. (2008). A critical analysis of eight informal reading inventories. Reading Teacher, 61, 526–536. http://dx.doi.org/10.1598/RT.61.7.2 [Article]

Smith, E. V., Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231. [PubMed]

Swanson, H. L., & Hoskyn, M. (1998). Experimental intervention research on students with learning disabilities: A meta-analysis of treatment outcomes. Review of Educational Research, 68, 277–321. http://dx.doi.org/10.3102/00346543068003277 [Article]

Wigfield, A., Guthrie, J. T., & McGough, K. (1996). A questionnaire measure of children’s motivation for reading (Instructional Resource No. 22). Washington, DC: Office of Educational Research and Improvement.

Wigfield, A., Guthrie, J. T., Tonks, S., & Perencevich, K. C. (2004). Children’s motivation for reading: Domain specificity and instructional influences. Journal of Educational Research, 97, 299–310. http://dx.doi.org/10.3200/JOER.97.6.299-310 [Article]

Wright, B., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370.

Figure 1.

Categories of reading and dimensions of reading participation in the Inventory of Reading Occupations.

Full Size | Slide (.ppt)

Figure 2.

Vertical ruler of student ability, reading categories, and reading dimensions of the Inventory of Reading Occupations.

Full Size | Slide (.ppt)

Table 1.

Fit Analysis of Revised Inventory of Reading Occupations (IRO) Test Items

IRO Item	Measure	SE	Infit MnSq	ZStd	Outfit MnSq	ZStd
Reading categories
Magazines	0.27	0.05	0.82	−2.5	0.87	−1.2
Labels	0.30	0.04	0.88	−1.8	0.85	−1.7
E-readers	−0.15	0.05	0.99	0.0	0.86	−1.2
Computers	−0.03	0.04	0.87	−1.9	0.88	−1.2
Notebooks	−0.17	0.04	1.01	0.1	0.90	−0.9
Comic books	0.08	0.05	1.02	0.2	0.91	−0.8
Television	0.09	0.05	1.01	0.1	0.94	−0.5
Bulletin boards	0.17	0.05	0.97	−0.4	0.95	−0.4
Game boards	0.03	0.04	1.00	0.0	0.96	−0.3
Posters	0.03	0.04	0.99	0.0	0.99	−0.1
Subject books	−0.05	0.05	1.03	0.4	1.02	0.2
Signs	−0.08	0.04	1.09	1.2	1.03	0.3
Story books	−0.20	0.04	1.11	1.4	1.12	1.1
Chalkboard	−0.12	0.04	1.14	1.9	1.11	1.0
Worksheets	−0.01	0.04	1.01	0.1	1.20	2.1
Game consoles	−0.07	0.05	1.14	1.7	1.20	1.7
Cellphone	0.07	0.05	1.16	1.9	1.32	2.5
Reading dimensions
Mastery	−0.12	0.02	0.90	−3.4	0.92	−1.7
Preference	−0.12	0.02	0.93	−1.2	0.98	−0.3
Frequency	0.24	0.02	1.13	5.2	1.23	5.7
Physical contexts
Community	1.09	0.05	0.95	−2.5	0.92	−2.7
School	0.15	0.04	0.92	−5.2	0.94	−2.5
Home	−1.24	0.05	1.13	4.5	1.33	5.2
Social contexts
Teachers	0.38	0.05	0.97	−1.3	0.89	−1.8
Other family members	0.10	0.05	1.01	0.3	1.01	0.3
Parents	−0.93	0.05	0.99	−0.2	1.03	0.8
Friends and classmates	0.46	0.05	1.04	1.4	1.04	0.6

Note. MnSq = mean square; SE = standard error; ZStd = standard z.

View Large

Table 1.

Fit Analysis of Revised Inventory of Reading Occupations (IRO) Test Items

IRO Item	Measure	SE	Infit MnSq	ZStd	Outfit MnSq	ZStd
Reading categories
Magazines	0.27	0.05	0.82	−2.5	0.87	−1.2
Labels	0.30	0.04	0.88	−1.8	0.85	−1.7
E-readers	−0.15	0.05	0.99	0.0	0.86	−1.2
Computers	−0.03	0.04	0.87	−1.9	0.88	−1.2
Notebooks	−0.17	0.04	1.01	0.1	0.90	−0.9
Comic books	0.08	0.05	1.02	0.2	0.91	−0.8
Television	0.09	0.05	1.01	0.1	0.94	−0.5
Bulletin boards	0.17	0.05	0.97	−0.4	0.95	−0.4
Game boards	0.03	0.04	1.00	0.0	0.96	−0.3
Posters	0.03	0.04	0.99	0.0	0.99	−0.1
Subject books	−0.05	0.05	1.03	0.4	1.02	0.2
Signs	−0.08	0.04	1.09	1.2	1.03	0.3
Story books	−0.20	0.04	1.11	1.4	1.12	1.1
Chalkboard	−0.12	0.04	1.14	1.9	1.11	1.0
Worksheets	−0.01	0.04	1.01	0.1	1.20	2.1
Game consoles	−0.07	0.05	1.14	1.7	1.20	1.7
Cellphone	0.07	0.05	1.16	1.9	1.32	2.5
Reading dimensions
Mastery	−0.12	0.02	0.90	−3.4	0.92	−1.7
Preference	−0.12	0.02	0.93	−1.2	0.98	−0.3
Frequency	0.24	0.02	1.13	5.2	1.23	5.7
Physical contexts
Community	1.09	0.05	0.95	−2.5	0.92	−2.7
School	0.15	0.04	0.92	−5.2	0.94	−2.5
Home	−1.24	0.05	1.13	4.5	1.33	5.2
Social contexts
Teachers	0.38	0.05	0.97	−1.3	0.89	−1.8
Other family members	0.10	0.05	1.01	0.3	1.01	0.3
Parents	−0.93	0.05	0.99	−0.2	1.03	0.8
Friends and classmates	0.46	0.05	1.04	1.4	1.04	0.6

Note. MnSq = mean square; SE = standard error; ZStd = standard z.

Table 2.

Measurable Data Summary of the Different Test Items of the Inventory of Reading Occupations

Category	Score	Expected	Residual Value	Standard Residual
Reading dimensions
M (n = 5,993)	3.86	3.86	0.00	0.00
Population SD	1.40	0.74	1.20	1.00
Sample SD	1.40	0.74	1.20	1.00
Physical contexts
M (n = 6,998)	0.55	0.55	0.00	−0.02
Population SD	0.50	0.25	0.43	1.03
Sample SD	0.50	0.25	0.43	1.03
Social contexts
M (n = 9,340)	0.36	0.36	0.00	0.00
Population SD	0.48	0.25	0.41	1.00
Sample SD	0.48	0.25	0.41	1.00

Note. M = mean; SD = standard deviation.

View Large

Table 2.

Measurable Data Summary of the Different Test Items of the Inventory of Reading Occupations

Category	Score	Expected	Residual Value	Standard Residual
Reading dimensions
M (n = 5,993)	3.86	3.86	0.00	0.00
Population SD	1.40	0.74	1.20	1.00
Sample SD	1.40	0.74	1.20	1.00
Physical contexts
M (n = 6,998)	0.55	0.55	0.00	−0.02
Population SD	0.50	0.25	0.43	1.03
Sample SD	0.50	0.25	0.43	1.03
Social contexts
M (n = 9,340)	0.36	0.36	0.00	0.00
Population SD	0.48	0.25	0.41	1.00
Sample SD	0.48	0.25	0.41	1.00

Note. M = mean; SD = standard deviation.