Validity, reliability, and psychometric properties of a computerized, cognitive assessment test (Cognivue®)

Abstract

BACKGROUND

Cognitive issues such as Alzheimer’s disease and other dementias confer a substantial negative impact. Problems relating to sensitivity, subjectivity, and inherent bias can limit the usefulness of many traditional methods of assessing cognitive impairment.

AIM

To determine cut-off scores for classification of cognitive impairment, and assess Cognivue^® safety and efficacy in a large validation study.

METHODS

Adults (age 55-95 years) at risk for age-related cognitive decline or dementia were invited via posters and email to participate in two cohort studies conducted at various outpatient clinics and assisted- and independent-living facilities. In the cut-off score determination study (n = 92), optimization analyses by positive percent agreement (PPA) and negative percent agreement (NPA), and by accuracy and error bias were conducted. In the clinical validation study (n = 401), regression, rank linear regression, and factor analyses were conducted. Participants in the clinical validation study also completed other neuropsychological tests.

RESULTS

For the cut-off score determination study, 92 participants completed St. Louis University Mental Status (SLUMS, reference standard) and Cognivue^® tests. Analyses showed that SLUMS cut-off scores of < 21 (impairment) and > 26 (no impairment) corresponded to Cognivue^® scores of 54.5 (NPA = 0.92; PPA = 0.64) and 78.5 (NPA = 0.5; PPA = 0.79), respectively. Therefore, conservatively, Cognivue^® scores of 55-64 corresponded to impairment, and 74-79 to no impairment. For the clinical validation study, 401 participants completed ≥ 1 testing session, and 358 completed 2 sessions 1-2 wk apart. Cognivue^® classification scores were validated, demonstrating good agreement with SLUMS scores (weighted κ 0.57; 95%CI: 0.50-0.63). Reliability analyses showed similar scores across repeated testing for Cognivue^® (R² = 0.81; r = 0.90) and SLUMS (R² = 0.67; r = 0.82). Psychometric validity of Cognivue^® was demonstrated vs. traditional neuropsychological tests. Scores were most closely correlated with measures of verbal processing, manual dexterity/speed, visual contrast sensitivity, visuospatial/executive function, and speed/sequencing.

CONCLUSION

Cognivue^® scores ≤ 50 avoid misclassification of impairment, and scores ≥ 75 avoid misclassification of unimpairment. The validation study demonstrates good agreement between Cognivue^® and SLUMS; superior reliability; and good psychometric validity.

Key Words: Cognitive screening test, Dementia, Memory, Motor control, Perceptual processing, Visual salience

Core tip: This study was designed to address the question of how to identify early cognitive impairment and how to monitor cognitive impairment over time with high reliability. This is critical in patients at risk for age-related cognitive decline. The study results demonstrated that Cognivue - through its unique adaptive psychophysics technology - had good validity and psychometric properties, as well as superior test-retest reliability compared to the St. Louis University Mental Status examination. Therefore, as a quantitative, computerized, assessment tool, Cognivue represents a safe and effective method for clinicians to more conveniently and effectively identify and track cognitive impairment.

INTRODUCTION

Tools for assessing cognitive function decline are often limited by issues of measurement efficacy[1-5], testing bias[5,6], inconsistent retest reliability[7], or cost[8]. In addition, some tests cannot be administered by non-clinicians, and excessive test length can make some tools impractical for routine use in clinical practice[6,9,10]. Cognivue^® is a physiological and psychophysical computerized tool for the automated assessment of cognitive functioning that is not dependent on traditional question-and-answer testing. It was developed based on extensive basic research into the neural mechanisms of functional impairment in memory, aging, and dementia. This was complimented by clinical research demonstrating letter, word, and motion perceptual deficits in patients with Alzheimer’s disease[11-14].

Cognivue^® consists of 3 sub-batteries of 10 separately scored sub-tests presented in a 10 min automated sequence. The first sub-battery (visuomotor and visual salience) measures adaptive motor control and dynamic visual contrast sensitivity. These results do not count towards the final score. They are used only to adapt the remaining sub-tests to the response characteristics of the individual subject. This ensures that only cognition is evaluated, and the subject is not at a disadvantage because of visual or motor deficits. The subsequent 2 sub-batteries include perceptual processing (letter, word, shape, and motion discrimination), and memory processing (letter, word, shape, and motion memory). Cognivue^® is Food and Drug Administration (FDA)-cleared for use as an adjunctive tool to aid in assessing cognitive impairment in subjects aged 55-95 years and is not intended to be used alone for diagnostic purposes.

This manuscript presents the full results of 2 clinical studies (previously presented at the 2019 annual meeting of the American Association of Geriatric Psychiatry) conducted to assess the efficacy and safety of Cognivue^®. The purpose of the first study was to establish Cognivue^® cut-off values (e.g., impaired, intermediate, and unimpaired cognitive function) compared to a reference standard, the St. Louis University Mental Status (SLUMS) Examination. The objective of the subsequent FDA pivotal trial was to clinically validate the agreement between the Cognivue^® classifications and the SLUMS classifications, examine test-retest reliability, and determine the psychometric properties of the tests.

MATERIALS AND METHODS

Cut-off score determination study

The study to determine Cognivue^® cut-off scores for the classification of cognitive impairment enrolled 92 adults 55 to 95 years of age from assisted and independent-living communities who were at risk for age-related cognitive decline or dementia. Subjects were invited via posters and email to complete both the SLUMS (reference standard) and Cognivue^® tests. Exclusion criteria included the presence of motor or visual disabilities and the inability to provide informed consent.

The SLUMS is an 11-item questionnaire with scores ranging from 0 to 30, and it is designed to measure orientation, memory, attention, and executive functions. As previously described, the Cognivue^® quantitative assessment tool includes 3 sequentially automated sub-batteries (visuomotor ability, perceptual processing, and memory processing) over a brief 10 min session (Table 1 and Figure 1). The first sub-battery is used to calibrate the subsequent tests based on the individual’s visual and motor abilities. Participants in this study were stratified according to SLUMS score (> 26 = unimpaired, 26-21 = mildly impaired, and < 21 = impaired)[15].

Table 1 Components of the Cognivue^® quantitative assessment tool.

Sub-battery and Sub-test	Description
Basic motor and visual ability
Adaptive motor control test	Assesses visuomotor responsiveness using speed and accuracy measures
	Measures subject’s ability to control the rotatory movement of the CogniWheel™ in response to rotational visual stimuli
Visual salience test	Assesses basic visual processing functions
	Measures the subject’s ability to identify a wedge filled by a random pattern of black and white dots shown on an neutral (gray) background
Perceptual processing
Letter discrimination	Measures the subject’s perceptual processing of different forms, despite the addition of increasing amounts of clutter
	Discriminate real English letters from a variety of non-letter, letter-like shapes
Word discrimination	As above
	Discriminate real 3-letter words from 3-letter non-words
Shape discrimination	As above
	Discriminate a circle filled with a common shape from the rest of the display filled with other common shapes
Motion discrimination	As above
	Discriminate a circle filled with one direction of dot motion from the rest of the display filled with another direction of dot motion
Memory processing
Letter memory	Assesses memory using specialized sets of visual stimuli
	Measures the subject’s ability to recall which letter was presented as a pre-cue, and then select that letter from a display of alternative items, despite the addition of increasing amounts of clutter
	Select the correct letter of the English alphabet
Word memory	As above
	Select the correct 3-letter word
Shape memory	As above
	Select the correct shape
Motion memory	As above
	Select the correct direction of motion

Figure 1 Cognivue^® quantitative sub-batteries. A: Visuomotor testing (motor control, visual salience); B: Perceptual processing (letter, word, shape, motion); C: Memory testing (letter, word, shape, motion).

Two different optimization methods were used to determine the cut-off values for Cognivue^® scores that corresponded to the SLUMS classifications unimpaired, intermediate (mildly impaired), and impaired. The first method used a minimization algorithm to optimize the negative percent agreement (NPA) and positive percent agreement (PPA) between Cognivue^® and SLUMS scores in the objective function: (1) NPA = [true negative (TN)/false positive (FP) + TN] × 100%; and (2) PPA = [true positive (TP)/false negative (FN) + TP] × 100%.

The second method used a minimization algorithm with two measures in the objective function, inaccuracy and error bias: (1) Inaccuracy = 1 - (TP + TN/total); (2) Error bias = contrast ratio (difference/sum) of FPs and FNs.

Clinical validation study

The validation study enrolled 401 adults 55 to 95 years of age at risk for age-related cognitive decline or dementia residing in independent-living communities who were invited via posters and email. Exclusion criteria included the presence of motor or visual disabilities and the inability to provide informed consent. Participants completed Cognivue^®, SLUMS, and an array of traditional neuropsychological tests (Table 2). Results of the cut-off determination study were used to group the participants, with subjects in the intermediate categories (low to moderate impairment) being combined with the unimpaired category for each testing modality.

Table 2 Summary of the purpose and analyses of the components of the Food and Drug Administration pivotal clinical trial of Cognivue^®.

Validation of classification scores	Purpose: Assess the validity of the previously defined Cognivue® cut-off scores in a larger sample of subjects
	Methods: Scores on Cognivue® and SLUMS were compared using regression and classification analyses. PPA and NPA were calculated
Assessment of retest reliability	Purpose: Compare scores from repeated administration of Cognivue® to assess retest reliability, compare findings to parallel results from SLUMS
	Methods: Repeated Cognivue® and SLUMS testing was conducted in 2 sessions 1-2 wk apart with regression and rank linear regression analysis being performed
Assessment of score psychometrics vs other neuropsychological tests	Purpose: Compare scores on Cognivue® and other neuropsychological tests to describe relationship and compare them to SLUMS
	Methods: 401 participants completed 10 different tests [SLUMS, SLUMS-clock drawing1, SLUMS-animal naming1, RAVLT, TMT-A, TMT-B, Benton JOLO, figural memory, PPB, HVCS, GDS (15-item)]; rank linear regression analysis and factor analysis performed

¹Scored separately from overall St. Louis University Mental Status. GDS: Geriatric Depression Scale; HVCS: Hamilton-Veale Contrast Sensitivity; JOLO: Judgment of line orientation; NPA: Negative percent agreement; PPA: Positive percent agreement; PPB: Purdue Peg Board; RAVLT: Rey Auditory Verbal Learning Test; SLUMS: St. Louis University Mental Status; TMT-A: Trail Making Test-A; TMT-B: Trail Making Test-B.

Agreement analysis of the impairment classifications as defined in the cut-off determination study was performed, along with an assessment of the retest reliability of Cognivue^®, and a comparison of its psychometric properties relative to other neuropsychological tests (Table 2). Regression analyses for agreement and retest reliability, and rank linear regression and factor analysis for psychometric comparisons, among others were performed (Table 2).

RESULTS

Cut-off score determination study

A total of 92 participants completed both the SLUMS and the Cognivue^® tests at one of five sites. Individual scores for study participants are shown in Figure 2. Based on SLUMS score, 50% were unimpaired (> 26), 38% were mildly impaired (26-21), and 12% were impaired (< 21) (Figure 2).

Figure 2 Scatterplot showing the St Louis University Mental Status and Cognivue^® scores for the 92 study participants. The table to the left of the scatter plot provides a key for relating the plot to participant classifications. Above the upper horizontal red line shows Cognivue^® scores of < 50 and to the left of the left vertical blue line shows St. Louis University Mental Status (SLUMS) scores < 21 denoting impaired. Below the bottom horizontal red line shows Cognivue^® scores of > 75 and to the right of the right vertical blue line shows SLUMS > 27 denoting unimpaired. Results of the analyses of classification are included in the table enclosed in the scatter plot. ACC: Accuracy; FN: False negative; FP: False positive; N%A: Negative percent agreement; P%A: Positive percent agreement; SLUMS: St. Louis University Mental Status; TN: True negative; TP: True positive.

In the analysis of NPA and PPA, the SLUMS cut-off score for impaired (< 21) minimized to 0.297 at a Cognivue^® cut-off score of 63.5 (NPA = 0.80; PPA = 0.79). The SLUMS cut-off score for unimpaired (> 26) minimized to 0.324 at a Cognivue^® cut-off score of 73.5 (NPA = 0.68; PPA = 0.67). In the analysis of optimization by accuracy and error bias, the SLUMS cut-off score of < 21 (impaired) corresponded to a Cognivue^® cut-off score of 54.5 (NPA = 0.92; PPA = 0.64). The SLUMS cut-off score of > 26 (unimpaired) corresponded to a Cognivue^® cut-off score of 78.5 (NPA = 0.5; PPA = 0.79).

Based on the 2 separate analysis techniques, it was determined that Cognivue^® scores between 55 and 64 corresponded to SLUMS scores indicating impaired (0 to 20), and Cognivue^® scores between 74 and 79 corresponded to SLUMS scores indicating unimpaired (27 to 30). Cognivue^® scores between the ranges classifying impaired and unimpaired (64 and 74) corresponded to SLUMS scores of 21 to 26. Therefore, Cognivue^® scores ≤ 50 provide a conservative standard consistent with cognitive impairment that will avoid misclassification of an individual that is impaired, and scores ≥ 75 provide a conservative cut-off consistent with no impairment that will avoid misclassification of an individual that is unimpaired (Table 3).

Table 3 Summary of Cognivue^® cut-off score analysis.

	SLUMS cut-off scores		Cognivue® cut-off scores
Impaired	< 21	→	≤ 50
Mildly impaired (intermediate)	21-26	→	51-74
Unimpaired	> 26	→	≥ 75

SLUMS: St. Louis University Mental Status.

Clinical validation study

Part 1: Validation analyses: A total of 401 participants completed at least 1 testing session, and based on SLUMS score, 30% were unimpaired (> 26), 43% were mildly impaired (26-21), and 27% were impaired (< 21). Validation analysis of the previously defined Cognivue^® classification scores yielded a PPA of 56% [95% Wilson interval (WI), 0.47-0.65] and a NPA of 95% (95%WI, 0.91-0.97), with a weighted κ = 0.57 [95% confidence intervals (CI): 0.50-0.63]. This suggests a significant categorical relationship between Cognivue^® and SLUMS scores. An analysis omitting the intermediate groups as being indeterminate showed a stronger relationship between Cognivue^® and SLUMS categories for impaired or unimpaired, with a PPA of 82% (95%WI, 0.72-0.89) and a NPA of 98% (95%WI, 0.93-0.99).

Part 2: Retest reliability analyses: Data were available for 358 participants who completed 2 Cognivue^® testing sessions, 1-2 wk apart. Regression analyses of test-retest reliability produced similar scores across repeat testing for Cognivue^® (regression fit: R² = 0.81; r = 0.90) (Figure 3A) and SLUMS (regression fit: R² = 0.67; r = 0.82) (Figure 3B).

Figure 3 Scatterplot showing first test scores (abscissa) and second test scores (ordinate) co-plotted with a Deming regression line (dashed) and 45° line (solid) for Cognivue^® and the St Louis University Mental Status. A: Cognivue^®; B: St. Louis University Mental Status. Cognivue^®: Intercept of line: 95% confidence interval (CI): 4.27-13.84 (SE = 2.433; P = 0.0002); slope of line: 95%CI: 0.880-0.993 (SE = 0.0285; P = 0.0264); regression fit: R² = 0.81 (r = 0.90). St. Louis University Mental Status: Intercept of line: 95%CI: 2.24-6.06 (SE = 0.970; P < 0.0001); slope of line: 95%CI: 0.82-0.97 (SE = 0.039; P = 0.039); regression fit: R² = 0.67 (r = 0.82). SLUMS: St. Louis University Mental Status.

Agreement analysis for Cognivue^® test-retest reliability revealed strong correlation between participant classification by first and second Cognivue^® tests (PPA = 89%; NPA = 93%), with an intraclass correlation (ICC) of tests 1 and 2 of 0.99 (P < 0.001). The agreement analysis for SLUMS also showed strong agreement between participant classification by first and second SLUMS tests (PPA = 87%; NPA = 87%; ICC, 0.87; P < 0.001).

The Cognivue^® classifications of impaired, intermediate, and unimpaired did not differ significantly across repeat testing. Analyzing the 3 classifications separately yielded PPAs of 89% for impaired, 57% for intermediate, and 87% for unimpaired (Table 4). Whereas for SLUMS, the relationships between scores and classifications across repeated testing were less robust than those for Cognivue^®. Analyzing the 3 SLUMS classifications separately yielded PPAs of 87% for impaired, 55% for mildly impaired, and 51% for unimpaired.

Table 4 Proportion of participants classified in each impairment category by first and second Cognivue^® tests.

2nd Test	1st Test
	Impaired	Intermediate	Unimpaired	Total
Impaired	42 (89%)	21	0	63
Intermediate	5	41 (57%)	32	78
Unimpaired	0	10	207 (87%)	217
Total	47	72	239	358

Psychometric analysis: This analysis was based on data from the 401 participants who had completed at least one testing session which included Cognivue^®, SLUMS, and a battery of other traditional neuropsychological tests. Rank scores on each psychometric test were plotted against their ranks on SLUMS scores and against their ranks on Cognivue^® scores with linear regression lines, the lines’ parameters, and their 95% confidence intervals.

Data were then condensed using a factor analysis of the various neuropsychological test scores. Tests were grouped according to relations between scores across participants. The factor analysis converged in six iterations to yield a five factor solution showing Cognivue^® scores most closely correlated with the following types of measures: verbal processing (animal naming and Rey Auditory Verbal Learning Test), manual dexterity and speed (Peg Board), visual acuity (contrast sensitivity), visuospatial and executive function (Trail Making Test-B and judgment of line orientation), and speed and sequencing (Trail Making Test-A) (Table 5).

Table 5 Factor analysis component matrix for the neuropsychological test scores indicating correlations with Cognivue^®.

	Component
	1	2	3	4	5
SLUMS-clock drawing	0.420	0.338	0.038	0.367	-0.049
SLUMS-animal naming	0.5291	0.346	0.146	0.365	-0.125
RAVLT-A-1	0.7181	0.209	0.034	0.128	-0.040
RAVLT-A-2	0.8201	0.204	0.080	0.157	-0.138
RAVLT-A-3	0.8321	0.193	0.120	0.190	-0.057
RAVLT-A-4	0.8471	0.200	0.143	0.184	-0.040
RAVLT-A-5	0.8631	0.210	0.080	0.182	-0.013
RAVLT-B-1	0.5791	0.213	0.104	0.178	-0.060
RAVLT-A-6	0.8521	0.134	0.093	0.170	-0.051
RAVLT-A-7	0.8601	0.159	0.117	0.164	-0.040
RAVLT-hits	0.6701	0.052	0.128	0.252	-0.003
RAVLT-fps	-0.408	-0.017	-.111	-0.041	0.125
Peg Board-Left	0.247	0.7962	0.297	0.120	-0.090
Peg Board-Right	0.297	0.7522	0.206	0.160	-0.186
Peg Board-Bimanual	0.293	0.8222	0.230	0.134	-0.137
Contrast-Left	0.146	0.133	0.8013	0.110	-0.068
Contrast-Right	0.160	0.156	0.8023	0.094	-0.106
Contrast-Binocular	0.189	0.183	0.8333	0.132	-0.153
TMT-B-Time	-0.312	-0.088	-0.213	-0.7884	0.116
TMT-B-Errors	-0.266	-0.072	-0.197	-0.8154	0.085
Benton JOLO	0.185	0.196	-0.024	0.4994	-0.344
TMT-A-Time	-0.115	-0.185	-0.158	-0.150	0.8625
TMT-A-Errors	-0.081	-0.058	-0.134	-0.068	0.9025
Figural memory	0.272	0.243	0.202	0.376	0.047
GDS	-0.119	-0.341	0.207	-0.329	-0.008

¹Cognivue^® scores most closely correlated with verbal processing (animal naming and Rey Auditory Verbal Learning Test);

²Cognivue^® scores most closely correlated with manual dexterity and speed (Peg Board);

³Cognivue^® scores most closely correlated with visual acuity (contrast sensitivity);

⁴Cognivue^® scores most closely correlated with visuospatial and executive function (Trails B and judgment of line orientation);

⁵Cognivue^® scores most closely correlated with speed and sequencing (Trails A). Fps: Frames per second; GDS: Geriatric Depression Scale; JOLO: Judgment of line orientation; RAVLT: Rey Auditory Verbal Learning Test; SLUMS: St. Louis University Mental Status; TMT: Trail Making Test.

The five factor scores for each participant were used in a multiple linear regression analyses for Cognivue^® scores and yielded an adjusted linear R² of 0.67 (Figure 4A). Similar analysis for SLUMS scores yielded an adjusted linear R² of 0.65 (Figure 4B).

Figure 4 Scatterplot showing regression standardized predicted values including all five factors and Cognivue^® scores or the St Louis University Mental Status scores co-plotted with linear regression lines. A: Cognivue^®; B: St. Louis University Mental Status. Avg: Average; R²: Coefficient of determination; SLUMS: St. Louis University Mental Status.

DISCUSSION

In the US, estimates of the prevalence of mild cognitive impairment range from 15%-20% among those ≥ 65 years of age[16], and 9.9%-35.2% among those ≥ 70 years of age[17-19], representing a substantial potential impact on both direct and indirect costs. Studies show cognitive issues such as Alzheimer’s disease and other dementias confer substantial negative impacts, including increased hospitalization, assisted living stays, home health care visits, risk of in hospital mortality, as well as increased costs compared to those without such impairments[16].

Cognivue^® was designed to overcome some of the limitations inherent in traditional paper and pencil tests for assessing cognitive impairment, such as sensitivity[1,2], subjectivity[3], test-retest reliability[7], as well as educational, language, gender, and cultural biases[5,6].

Cognivue^® eliminates these biases through its unique adaptive psychophysics technology, which focuses on whether the subject can detect and identify stimuli, including differentiating between different stimuli, and describing the magnitude or nature of the difference. This is unrelated to the subject’s level of education or social/economic status. Furthermore, the first sub-battery of tests is designed to assess the individual’s visual and motor abilities. These are used only to adapt subsequent tests, and therefore, eliminate bias related to visual and motor deficits and assessing only the individual’s cognitive function.

The dynamic, adaptive technology avoids memorization issues associated with current paper and pencil tests. The test-retest results of this trial suggest that Cognivue^® may help eliminate or reduce the effect of memorization and human error/variability on retest reliability[7], while the retests with SLUMS showed a lower degree of retest reliability. Cognivue^® subjects are unable to memorize answers due to the adaptive nature of the test.

Other barriers to cognitive testing in clinical practice include the impact on costs and scheduling flexibility that are related to the need for administration by the provider specifically[6,9,20]. Cognivue^® is self-administered by the patient and can be initiated by non-clinician support staff. Work load is also reduced by the ability of Cognivue^® to track and monitor test outcomes over repeat testing over time.

In a survey of primary care providers, diagnosis and treatment of mild cognitive impairment was impacted by a negative attitude toward the importance of early diagnosis[21]. Recent recommendations recognized that cognitive impairment occurs on a continuum, progressing from age-related decline, to mild cognitive impairment, and to dementia[22]. Early identification of cognitive impairment can lead to earlier management, which can lead to improved prognosis and decreased morbidity, and early discussion regarding decision-making[23,24]. In fact, patients often identify assistance in planning for future treatments as a factor influencing their willingness to be screened[22]. In clinical practice, physician clinical judgment is less effective than structured tools for the recognition of mild cognitive impairment[25]. The 2017 Geriatric Summit on Assessing Cognitive Disorders Among the Aging Population sponsored by the National Academy of Neuropsychology, emphasized the importance of ongoing screening, and the need for automated tools for assessing and recording patient’s results over time[26]. Cognivue^®, provides an easy-to-use reliable, computerized, testing tool, which can help clinicians meet these needs.

A limitation of the present studies validating Cognivue^® is the use of a single reference standard, SLUMS. However, although Cognivue^® has not yet been compared to Mini-Mental Status Examination (MMSE) or Montreal Cognitive Assessment (MoCA) tests—two of the more widely used paper and pencil cognitive assessments—SLUMS has been[15,27-29]. These studies demonstrated either the equivalence or superiority of SLUMS. Therefore, it is a reasonable to infer the likely equivalence, at a minimum, of Cognivue^® in terms of its sensitivity, specificity, and psychometric validity to these commonly used tools. Cognivue^® may be less likely to under diagnose (reported with MMSE)[28], or over diagnosis mild cases of impairment (10% with MMSE, 25% with MoCA)[22].

Longitudinal follow-up studies are underway to assess the ability of Cognivue^® to monitor cognitive deterioration over time. Cognivue^® is also being studied in other patient populations and specific disease states such as multiple sclerosis[30]. Additionally, a highly portable model of Cognivue^® with a Health Insurance Portability and Accountability Act-compliant data repository is scheduled to be launched in mid-2019.

In conclusion, Cognivue^® scores ≤ 50 and ≥ 75 were consistent with conservative standards for impaired and unimpaired, respectively, in the cut-off study. When these scores were used in the clinical trial also reported here, Cognivue^® was shown to have good validity and psychometric properties, and superior test- retest reliability compared to SLUMS. As a result, Cognivue^® received FDA de novo 510(k) clearance as an adjunctive tool for evaluating cognition through perceptual and memory function in individuals 55-95 years of age[31]. It is not intended to be used as a diagnostic tool.

These studies demonstrated the safety and efficacy of Cognivue^®, which is the first FDA-cleared test for the automated computerized assessment of cognitive functioning. This represents an improved approach to help in the early identification of cognitive impairment in patients at-risk for cognitive decline.

ARTICLE HIGHLIGHTS

Research background

The assessment of declining cognitive function due to age or dementia is often impeded by multiple factors such as testing bias, cost, poor measurement efficacy, test inconsistency, and test length. Some tools also specifically require administration by a clinician, adding further constraint to routine testing in clinical practice. Prior research on the neural mechanisms of functional impairment in memory, aging, and dementia has led to the development of a novel computerized testing method.

Research motivation

There is a need for a reliable, easy-to-use, novel method for the early identification, testing, and monitoring of cognitive impairment in at-risk patient populations. Cognivue^® is a safe and effective automated method for identifying and tracking cognitive impairment in a more convenient and efficient manner than traditional question and answer paper and pencil testing.

Research objectives

The main objective of the first clinical study was to establish cut-off values for Cognivue^® (e.g., impaired, intermediate, and unimpaired cognitive function) relative to the St. Louis University Mental Status (SLUMS). The main objectives of the second study were to clinically validate the agreement between Cognivue^® and SLUMS classifications, assess retest reliability of Cognivue^®, and assess Cognivue^®'s psychometric properties.

Research methods

Participants in the first study completed both the SLUMS and Cognivue^® tests. Optimization methods used to determine cut-off values included a minimization algorithm to optimize the negative percent agreement and positive percent agreement between test scores in the objective function and a minimization algorithm with two measures in the objective function (inaccuracy, error bias). Participants in the second study also completed both the SLUMS and Cognivue^® tests as well as other traditional neuropsychological tests. Regression analyses of agreement and retest reliability, as well as rank linear regression and factor analysis for psychometric comparisons were conducted, and agreement analysis of the impairment classifications derived from the first study was also performed.

Research results

It was found that Cognivue^® scores ≤ 50 would avoid misclassification of an impaired person while providing a conservative standard consistent with cognitive impairment, and that scores ≥ 75 would avoid misclassification of an unimpaired person while providing a conservative cut-off consistent with no impairment. In the second study, Cognivue^® demonstrated good validity and psychometric properties, as well as superior test-retest reliability compared to the SLUMS.

Research conclusions

With its unique adaptive psychophysics technology, Cognivue^® provides a computerized tool for the automated assessment of cognitive functioning free from many of the biases and limitations of the more traditional paper and pencil methods such as sensitivity, subjectivity, test-retest reliability, and educational, language, gender, and cultural biases. Improved prognosis and decreased morbidity may be possible with an earlier implementation of management strategies in patients with cognitive impairment identified earlier in its course. Because Cognivue^® is self-administered it provides clinicians with a safe, effective, and time-saving method to efficiently assist in the assessment and monitoring of cognitive impairment in their patients.

Research perspectives

Trials assessing the utility of Cognivue^® in other, specific patient populations are currently in progress. Additionally, the ability of Cognivue^® to monitor the deterioration of cognitive function over time is currently being assessed in longitudinal studies.