David Schelly, Alisha Ohl; Examining Clinical Meaningfulness in Randomized Controlled Trials: Revisiting the Well Elderly II. Am J Occup Ther 2019;73(1):7301205120p1-7301205120p13. doi: 10.5014/ajot.2019.030874.
Download citation file:
© 2019 American Occupational Therapy Association
OBJECTIVE. Randomized controlled trial (RCT) interventions often rely on p values, where statistical significance is assumed to provide evidence of an intervention effect. This study provides a secondary data analysis of the Well Elderly II RCT using multiple approaches that examine clinical meaningfulness.
METHOD. We reanalyzed the Well Elderly II RCT using effect size, standard deviation, standard error of measurement, minimal difference, a fragility index, an assessment of poor scores at baseline, and an analysis with a small subgroup of participants removed.
RESULTS. Although some participants improved on several scales, most stayed the same, and a small subset declined. Omitting a small subgroup of participants led to nonsignificant p values.
CONCLUSION. There is evidence that disparities in baseline scores and regression to the mean may have created the appearance of an intervention effect. Our methods of considering clinical meaningfulness suggest improved approaches to analyzing RCT data.
For evidence-based practice, RCTs that only report p values should not be regarded as a high level of evidence without the consideration of effect size.
Researchers should assess clinical meaningfulness by considering participants and subgroups rather than only aggregate scores; we recommend simple approaches that rely on descriptive statistics, beginning with comparisons of percentages of participants who declined, stayed the same, and improved between the treatment and control groups.
Researchers should scrutinize baseline values, consider measurement error, and assess the fragility of their results.
This PDF is available to Subscribers Only
For full access to this pdf, sign in to an existing account, or purchase an annual subscription.