Observational Study
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Psychiatr. Jan 19, 2020; 10(1): 1-11
Published online Jan 19, 2020. doi: 10.5498/wjp.v10.i1.1
Validity, reliability, and psychometric properties of a computerized, cognitive assessment test (Cognivue®)
Diego Cahn-Hidalgo, Paul W Estes, Reina Benabou
Diego Cahn-Hidalgo, Internal Medicine of Brighton, Rochester, NY 14623, United States
Paul W Estes, Reina Benabou, Cognivue Inc., Victor, NY 14564, United States
Author contributions: Cahn-Hidalgo D drafted the initial manuscript; Estes PW and Benabou R performed data analysis for this research; and all three authors reviewed and revised the initial draft, and subsequently reviewed and approved the final, submitted manuscript.
Institutional review board statement: The study was reviewed and approved by the Western Institutional Review Boards (WIRB).
Informed consent statement: All study participants provided informed written consent about personal and medical data collection prior to study enrolment.
Conflict-of-interest statement: Dr. Cahn-Hidalgo has acted as a consultant and speaker for Cognivue Inc. Estes PW and Dr. Benabou are employees of Cognivue Inc.
Data sharing statement: Statistical analysis and dataset available from the corresponding author at rbenabou@cognivue.com. Study subjects gave informed consent for anonymized data sharing.
STROBE statement: The authors have read the STROBE statement checklist of items, and the manuscript was prepared and revised according to the STROBE statement checklist of items.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Reina Benabou, MD, PhD, Cognivue Inc., 7911 Rae Blvd, Victor, NY 14564, United States. rbenabou@cognivue.com
Received: June 20, 2019
Peer-review started: June 23, 2019
First decision: August 20, 2019
Revised: November 14, 2019
Accepted: November 26, 2019
Article in press: November 26, 2019
Published online: January 19, 2020
Abstract
BACKGROUND

Cognitive issues such as Alzheimer’s disease and other dementias confer a substantial negative impact. Problems relating to sensitivity, subjectivity, and inherent bias can limit the usefulness of many traditional methods of assessing cognitive impairment.

AIM

To determine cut-off scores for classification of cognitive impairment, and assess Cognivue® safety and efficacy in a large validation study.

METHODS

Adults (age 55-95 years) at risk for age-related cognitive decline or dementia were invited via posters and email to participate in two cohort studies conducted at various outpatient clinics and assisted- and independent-living facilities. In the cut-off score determination study (n = 92), optimization analyses by positive percent agreement (PPA) and negative percent agreement (NPA), and by accuracy and error bias were conducted. In the clinical validation study (n = 401), regression, rank linear regression, and factor analyses were conducted. Participants in the clinical validation study also completed other neuropsychological tests.

RESULTS

For the cut-off score determination study, 92 participants completed St. Louis University Mental Status (SLUMS, reference standard) and Cognivue® tests. Analyses showed that SLUMS cut-off scores of < 21 (impairment) and > 26 (no impairment) corresponded to Cognivue® scores of 54.5 (NPA = 0.92; PPA = 0.64) and 78.5 (NPA = 0.5; PPA = 0.79), respectively. Therefore, conservatively, Cognivue® scores of 55-64 corresponded to impairment, and 74-79 to no impairment. For the clinical validation study, 401 participants completed ≥ 1 testing session, and 358 completed 2 sessions 1-2 wk apart. Cognivue® classification scores were validated, demonstrating good agreement with SLUMS scores (weighted κ 0.57; 95%CI: 0.50-0.63). Reliability analyses showed similar scores across repeated testing for Cognivue® (R2 = 0.81; r = 0.90) and SLUMS (R2 = 0.67; r = 0.82). Psychometric validity of Cognivue® was demonstrated vs. traditional neuropsychological tests. Scores were most closely correlated with measures of verbal processing, manual dexterity/speed, visual contrast sensitivity, visuospatial/executive function, and speed/sequencing.

CONCLUSION

Cognivue® scores ≤ 50 avoid misclassification of impairment, and scores ≥ 75 avoid misclassification of unimpairment. The validation study demonstrates good agreement between Cognivue® and SLUMS; superior reliability; and good psychometric validity.

Keywords: Cognitive screening test, Dementia, Memory, Motor control, Perceptual processing, Visual salience

Core tip: This study was designed to address the question of how to identify early cognitive impairment and how to monitor cognitive impairment over time with high reliability. This is critical in patients at risk for age-related cognitive decline. The study results demonstrated that Cognivue - through its unique adaptive psychophysics technology - had good validity and psychometric properties, as well as superior test-retest reliability compared to the St. Louis University Mental Status examination. Therefore, as a quantitative, computerized, assessment tool, Cognivue represents a safe and effective method for clinicians to more conveniently and effectively identify and track cognitive impairment.