The rigor of occupation-based standardized assessments that rely on observational scoring procedures depends on proven reliability among test administrators. This study measured interrater reliability of the Cognitive Performance Test (CPT), a standardized, occupation-based assessment that measures cognitive–functional capacity in older adults with neurocognitive disorders. To capture a range of experience among test administrators, two sets of raters—four expert and three novice—scored video recordings of 10 patients administered the CPT. Interrater reliability results were strong among all raters (intraclass correlation coefficient [ICC] = .93), with expert raters (ICC = .97) yielding higher coefficients than novice raters (ICC = .93). Spearman’s ρ correlation coefficients were high among all raters (rs = .92–1.00). Practitioners can be confident that results of the CPT give accurate and consistent information to the health care team, family members, and patients when administered with fidelity using standardized protocols.