Standardization of apple cancellation test for neglect patients in Korea: An observational study

Abstract

BACKGROUND

Neglect can be divided into two types using apple cancellation test (apple test): Egocentric neglect (EN) and allocentric neglect (AN). However, in South Korea, apple test results and decision criteria are still largely dependent on tests by foreign countries.

AIM

To establish a new South Korea standard and improve the accuracy of neglect assessment, the apple experiment was standardized in this study.

METHODS

This study was conducted on 223 healthy subjects for a total of 7 mo from August 2021 to February 2022. Standardization was carried out using the original apple test developed by Bickerton in 2011. In scoring for the apple test, total omission error refers to the number of missed targets (full apple) in the entire test sheet (left, middle, and right area). The score for EN is the difference between the cor-rect number of right area and the correct number of left area (excluding the middle area). For AN, the score is difference between the number of left opening apples and number of right opening apples (including the middle area). Linear regression analysis was used for standardization using the general characteristics of subjects and the results of the apple test.

RESULTS

The cut-off score, which is the standard value indicating the pathological condition by combining the results of all subjects, is as follows: Total omission error (5), error for EN (2), and error for AN (2). Also, differences in cut-off score according to age were found.

CONCLUSION

This study will be helpful in facilitating a more accurate differential diagnosis of neglect.

Key Words: Apple cancellation test, Egocentric neglect, Allocentric neglect, Brain injury, Standardization

Core Tip: This study attempted to standardize the apple cancellation test for differential diagnosis of neglect in Korea. As a result, various cut-off scores are presented to establish criteria for diagnosing egocentric and allocentric neglect. In addition, the appropriate time to perform the test according to age and education level is also presented. We hope that this study will be useful for future studies.

INTRODUCTION

Neglect, the difficulty in recognizing bodily stimuli on the opposite side of the brain injury due to damage to the non-dominant hemisphere (mainly the right brain), is a common sequela of stroke[1]. Such neglect can be divided into egocentric and allocentric[2]. Egocentric neglect (EN) is characterized by having difficulties in recognizing half of an object in the body or space (mainly the left side) centered on the self. Allocentric neglect (AN) is characterized by having difficulties recognizing the half (mainly the left side) regardless of the object’s position[3]. Due to these characteristics, EN is sometimes called viewer-centered neglect, while AN is called object-centered neglect[3].

Various methods for neglect assessment have been developed and used[4-10]. For instance, cancellation tests, including Letter Cancellation Task[4], Bells Test[5], and Star Cancellation Test[6], have been created. These cancellation tests are conducted by checking targets among several letters or pictures of objects randomly scattered on the test sheet and then examining the presence and degree of neglect based on the difference in the left/right performance[4-6]. Line crossing, also called Albert’s test, included in another cancellation test is similar to other cancellation tests in that 36 small lines randomly scattered on the test sheet are provided, but the subject is required to mark the center of all lines, which differs from the previous tests in the method of execution[6]. The line bisection test is another example[7]. For this test, the presence and degree of neglect can be evaluated based on the extent of deviation from the center after marking the center of 20 horizontal lines arranged neatly on the test sheet[7]. Copying or drawing comprises another neglect test[8]. This test is conducted by instructing to imitate a specific picture presented or draw a butterfly, a person, a clock, etc., without showing a picture to evaluate the presence or absence of errors[8]. Finally, the Catherine Bergego scale was developed to assess the daily life performance of neglect patients[9,10]. This test consists of 10 items evaluated on a 0- to 3-point scale, with the higher score indicating more problems that neglect patients have in their daily life[9,10]. However, all of these evaluation tools were developed when AN was not differentiated, so only EN, which could not recognize the other side of brain injury centered on the self, can be evaluated.

The emergence of the apple cancellation test (apple test) developed in 2011 for differential diagnosis of neglect made the distinction between EN and AN smooth[11]. This apple test can be used not only for differential diagnosis of neglect but also for cases where two types of neglect exist simultaneously[11]. Based on this discrimination, it was found that AN had a more adverse effect on daily life performance than EN[12]. Moreover, this study allowed health care practitioners to understand that the existing neglect treatment was ineffective in treating AN and the need for developing a new treatment method[13]. It was possible to conveniently identify the degree of improvement of two types of neglect, which could not be identified before, through determination and selective treatment.

Accordingly, standardization studies were conducted in countries such as Italy, England, China, and Russia to use useful apple test[14-17]. However, in South Korea, apple test results and decision criteria still largely depend on tests by foreign countries.

Therefore, in this study, domestic standardization of the apple test was carried out to establish a new standard for South Korea and improve neglect assessment accuracy.

MATERIALS AND METHODS

Study subjects

This study evaluated 231 healthy and normal persons and analyzed 223 subjects, excluding 8 unfaithful subjects[11]. The age range was 20 to 80 years, and the exclusion criteria were as follows: (1) Those with neurological and psychiatric impairments; (2) those with cognitive impairment, such as communication problems; (3) those with visual problems, such as vision; and (4) those who did not give consent to participate in the study.

Data collected were investigated at six levels of education (i.e., illiteracy, elementary school, middle school, high school, junior college, and university graduation). Details are presented in the Supplementary Table 1. Before the start of the study, approval was obtained from the Institutional Review Board (KWNUIRB-2021-06-002-002), and all subjects were informed about the study, and they consented before participating (Table 1).

Table 1 General characteristics of subjects (n = 223) (mean ± SD).

General characteristic		n	%
Gender	Male	109	48.90	-
	Female	114	51.10
Age (yr)	20-29	38	17.04	48.80 ± 18.10
	30-39	38	17.04
	40-49	36	16.14
	50-59	36	16.14
	60-69	40	17.94
	70-	35	15.70
Level of education	Illiteracy	10	4.48	12.27 ± 4.00
	Elementary school	19	8.52
	Middle school	22	9.87
	High school	57	25.56
	Junior college	50	22.42
	University	65	29.15
Residence	Seoul/Gyeonggi province	88	39.46	-
	Gyeongsang province	10	4.48
	Jeolla province	1	0.45
	Gangwon province	116	52.02
	Chungcheong province	7	3.14
	Jeju province	1	0.45

Study tools/scoring methods

This study employed the original version of the apple cancellation test (apple test) (Bickerton et al[11], 2011). The apple test consists of 150 apples [50 target full apples and 100 open apples as distractors (left or right)][11,14]. Using an invisible grid, the page is divided into five areas (left, 2 areas; middle, 1 area; right, 2 areas). All 150 apples are pseudo-randomly scattered (30 apples in each area). Under the test method, only the target (full apple) should be checked regardless of the apple’s size, while a separate mark should not be added on the distractor (left or right open apple). A stopwatch is used, and the time limit is set at 5 min.

In scoring, total omission error refers to the number of missed targets (full apple) in the test sheet. The score for EN is the difference between the correct number of right area and the correct number of left area (excluding the middle area). For AN, the score is the difference between the number of left opening apples and the number of right opening apples (including the middle area) (Figure 1).

Figure 1 Apple test. Orange arrows indicate the complete targets, green arrows indicate the left open targets, and yellow arrows refer to the right open targets.

Study procedures

This study was conducted to prepare a standard for performing the apple cancellation test (apple test) in normal people and present a standard for differentiating pathological performance based on this. The study lasted 7 mo, from August 2021 to February 2022. The test was conducted by three evaluators who were trained in advance. The subject could familiarize himself/herself with the test method through a preliminary test before undergoing the apple test. Although the selecting method for the target (full apple) in the preliminary test sheet was the same, it was differentiated from the main test by presenting only 7 targets and distractors vertically in the middle of the test sheet.

Analysis

Statistical analyses were performed using IBM SPSS Statistics 25.0. Descriptive statistics were used for the subject’s general characteristics, while linear regression analysis examined the relationship between the subject’s general characteristics and the apple test score. The cutoff value of the confidence interval for discriminating pathological performance through the apple test score was also set. A 95%CI was used for the cutoff value. Data from those in their 20s or subjects who were in the 3^rd year or higher at a university were analyzed by converting their educational age to a junior college degree or higher.

RESULTS

Accuracy score-total omission errors

The results of analyzing the total omission errors of the subjects through linear regression analysis did not confirm significance according to age and level of education (F = 1.91; df = 2; r² = 0.02, P = 0.15). Consequently, a cutoff was calculated for the subjects’ pathological performance. The mean total omission error was 2.74 (standard error = 0.21; range from 0 to 14), and a cutoff of 4 was obtained with a 95%CI: (2.74 + 1.96 × 0.21 = 3.15) and by calculating in excess. Therefore, if there are 5 or more omission errors, the result should be considered a pathological condition. The pathological cutoff by age was 4 for those in their 20s, 5 for those in their 30s, 40s, and 60s, and 6 for those in their 50s and 70s or older (Table 2).

Table 2 Accuracy score and total omission errors.

Linear regression	F	r2	P value	Descriptive statistics
				Minimum value	Maximum value	Mean	Standard error	Cutoff	Pathological cutoff
Total	1.91	0.02	0.15	0.00	14.00	2.74	0.21	3.15	5
20-29	0.49	0.03	0.62	0.00	8.00	1.95	0.36	2.64	4
30-39	2.05	0.11	0.15	0.00	13.00	2.68	0.53	3.72	5
40-49	2.07	0.11	0.14	0.00	13.00	2.53	0.48	3.46	5
50-59	0.18	0.01	0.84	0.00	12.00	3.50	0.57	4.61	6
60-69	1.33	0.07	0.28	0.00	14.00	2.65	0.47	3.56	5
70-	0.26	0.00	0.93	0.00	13.00	3.23	0.60	4.41	6

Asymmetry score for EN-omission error difference

Significance was not confirmed according to education level and age in the linear regression model (F = 1.09; df = 2, r² = 0.01; P = 0.34). Consequently, a cutoff for the pathological performance was calculated for all subjects. The difference in the number of correct answers for all subjects (i.e., the number of correct answers in the right area - the number of correct answers in the left area) was 0.00 on average (standard error = 0.14, ranging from -3 to +3), and a cutoff of 1 was obtained with a 95%CI: (0.00 + 1.96 × 0.07 = 0.14) and by calculating in excess. Therefore, if the difference in the number of correct answers in the left and right areas, excluding the middle area, is 2 or more, the result should be regarded as a pathological condition. The pathological cutoff by age was 1 for those in their 20s and 2 for all other age groups. The sign indicates EN on the left side in the case of + and on the right side in the case of –(Table 3).

Table 3 Asymmetry score for egocentric neglect-omission error difference.

Linear regression	F	r2	P value	Descriptive statistics
				Minimum value	Maximum value	Mean	Standard error	Cutoff	Pathological cutoff
Total	1.09	0.01	0.34	-3.00	3.00	0.00	0.07	0.14	2
20-29	0.04	0.00	0.96	-2.00	1.00	-0.29	0.15	0.00	1
30-39	0.10	0.01	0.91	-2.00	3.00	0.13	0.19	0.50	2
40-49	0.51	0.03	0.60	-3.00	1.00	0.00	0.16	0.31	2
50-59	0.85	0.05	0.44	-3.00	2.00	-0.19	0.20	0.19	2
60-69	0.28	0.02	0.76	-2.00	3.00	0.25	0.17	0.58	2
70-	0.50	0.03	0.61	-3.00	3.00	0.57	0.21	0.99	2

Descriptive statistics and linear regression analysis.

Asymmetry score for AN-commission error difference

Significance was not confirmed according to the education level and age in the linear regression model (F = 1.67; df = 2; r² = 0.02, P = 0.19). Consequently, a cutoff for the pathological performance was calculated for 223 subjects. The difference between the number of the left opening apples and the number of the right opening apples was 0.03 on average (standard error = 0.03, ranging from -3 to 2), and a cutoff of 1 was obtained with a 95%CI: (0.03 + 1.96 × 0.03 = 0.09) and by calculating in excess. Therefore, if the value obtained by subtracting the number of the right opening apples from the number of the left opening apples among incorrect answers in all areas of the test sheet is 2 or more, the result should be regarded as a pathological condition. The pathological cutoff by age was 2 in all age groups. The sign indicates AN on the left side in the case of + and on the right side in the case of - (Table 4).

Table 4 Asymmetry score for allocentric neglect-commission error difference.

Linear regression	F	r2	P value	Descriptive statistics
				Minimum value	Maximum value	Mean	Standard error	Cutoff	Pathological cutoff
Total	1.67	0.02	0.19	-3.00	2.00	0.03	0.03	0.09	2
20-29	1.35	0.07	0.27	-1.00	1.00	0.05	0.06	0.18	2
30-39	0.58	0.03	0.57	0.00	1.00	0.08	0.04	0.17	2
40-49	0.29	0.02	0.75	-1.00	1.00	0.03	0.06	0.15	2
50-59	3.74	0.19	0.03a	0.00	1.00	0.03	0.03	0.08	2
60-69	2.25	0.11	0.12	-3.00	2.00	-0.05	0.09	0.14	2
70-	2.18	0.12	0.13	-1.00	2.00	0.57	0.09	0.75	2

^aP < 0.05.

Descriptive statistics and linear regression analysis.

Expected and maximum time of execution

The expected time spent by subjects completing the task was estimated. In the linear regression model, age and level of education had a significant effect on the time required to perform the task (F = 50.24, df = 2, r² = 0.31; P = 0.000). Based on this analysis, the following conversion formula was obtained:

The expected time of execution [106.54 + 0.76 × age + (-3.74) × education period] was calculated based on age and education level. Then, the maximum time of execution was estimated based on the calculated result of the expected time of execution. If this time is exceeded, a pathological performance is assumed.

Expected and maximum time of execution (maximum time = expected time + 1.96 × standard error of residuals (38.72) are shown in Table 5.

Table 5 Expected and maximum time of execution in seconds.

Time		Age
		20	25	30	35	40	45	50	55	60	65	70	75	80
Expected time
Level of education	0	122	126	129	133	137	141	145	148	152	156	160	164	167
	6	99	103	107	111	115	118	122	126	130	134	137	141	145
	9	88	92	96	99	103	107	111	115	118	122	126	130	134
	12	77	81	84	88	92	96	100	103	107	111	115	119	122
	14	69	73	77	81	85	88	92	96	100	104	107	111	115
	16	62	66	70	73	77	81	85	89	92	96	100	104	108
Maximum time
Level of education	0	198	201	205	209	213	217	220	224	228	232	236	239	243
	6	175	179	183	187	190	194	198	202	206	209	213	217	221
	9	164	168	172	175	179	183	187	191	194	198	202	206	210
	12	153	157	160	164	168	172	176	179	183	187	191	195	198
	14	145	149	153	157	160	164	168	172	176	179	183	187	191
	16	138	142	145	149	153	157	161	164	168	172	176	180	183

Linear regression analysis.

DISCUSSION

Neglect is divided into EN and AN. The concept of EN is well established, and various evaluation tools have already been suggested. However, awareness of AN is still lacking, and only a few evaluation tools have been developed and standardized. This study was conducted because the apple cancellation test (apple test) has yet to be standardized, so that the existing EN can be differentiated from the new AN using only one evaluation.

The results of linear regression analysis showed that total omission error and egocentric and AN are not significantly related to the subjects’ age and education level. Accordingly, a pathological cutoff of 5 was calculated based on the mean and standard error for the total omission error. In terms of the pathological cutoff by age, it was found that the frequency of omission of correct answers increased as the age increased from those in their 20s (4 points) to those in their 70s (6 points). Additionally, the fact that the total omission error exceeds the pathological cutoff indicates a problem with the subject’s evaluation accuracy.

For the pathological cutoff of EN, a cutoff of 2 was obtained overall. For each age group, a cutoff of 1 was obtained for those in their 20s and a cutoff of 2 for all other age groups. For those in their 20s, the standard error was the lowest due to the high-performance level, and the standard error became higher as the difference in performance between individuals gradually increased with age. In addition, there was a slight difference in the object recognition ability for space with increasing age. However, there was no significant difference in all age groups when considering the pathological cutoff.

For the pathological cutoff of AN, a cutoff of 2 was obtained overall. All age groups had the same reference value. This means that there is no significant difference in the ability to recognize objects in various positions, even with increasing age. However, based on the decision cutoff, it can be confirmed that it is a very sensitive evaluation tool in which EN and AN can be diagnosed even with one or two mistakes.

Lastly, the expected and maximum time of execution was significantly affected by the education level and age. Since the examination is based on domestic education level (elementary school, middle school, high school, junior college, and university graduation), it is expected that subjects in their 20s who graduated from university (16 years) will have the fastest execution time at 62 s, and those in their 80s without education will have the longest execution time at 167 s. Unlike previous results, it is greatly affected by the level of education because previous studies conducted the test with the level of education in three categories[14], while in this study, there were six categories (i.e., illiteracy, elementary school, middle school, high school, junior college, and university graduation). Another plausible reason is that cognitive evaluation is greatly influenced by the level of education, as shown in a study’s findings[18]. For example, in South Korea, the proportion of people with middle school or lower level of education (11%) as of 2020 was lower than the Organisation for Economic Cooperation and Development (OECD) average (21%), while the proportion of people with high school or higher level of education (51%) was higher than the OECD average (40%)[19]. These results indicate that South Korea has a very low distribution of illiteracy and has been significantly influenced by the overall high level of education.

In addition, in contrast to the existing standardization studies where data collection was performed only in limited places[14], this study was conducted with the general public living in the local community. Consequently, a more accurate normal reference value was obtained. The accuracy score-total omission errors provided by the study’s results can be used to reduce the number of cases where severe cognitive decline is misinterpreted as neglect. It will also allow for a more accurate assessment of EN and AN, and a more detailed understanding of the extent of improvement with intervention. More importantly, the presentation of test times according to age and education level may provide another basis for patient neglect research.

However, this study has the following limitations: (1) The regional distribution is not uniform; (2) those with high education are concentrated by age group due to South Korea’s characteristics; and (3) in the cutoff of total omission error, a specific reason for those in their 50s having a higher cutoff than the 60s was not presented.

Despite such limitations, this study can be meaningful since it is the first to establish South Korea’s discrimination criteria as an evaluation tool for neglect. Unlike previous studies, this study not only provides a cutoff for each age group but also suggests the expected execution time through the detailed classification of education level to compare the current performance level. Lastly, it is expected that more accurate patient evaluation and differential diagnosis will be conducted, and many interventions for AN can be developed by standardizing the apple cancellation test (apple test).

CONCLUSION

This study aimed to standardize the apple cancellation test (apple test). Cutoff scores for total omission error and EN and AN were presented. The expected execution time, considering the detailed age groups and education level, was also added. This study will help facilitate a more accurate differential diagnosis of neglect. Finally, it is hoped that more diverse treatments will be developed based on the apple test.

ARTICLE HIGHLIGHTS

Research background

Neglect has different symptoms depending on the type. Among the different types, egocentric neglect (EN) and allocentric neglect (AN) can be differentiated by the apple cancellation test.

Research motivation

Many countries have standardized the apple cancellation test and used it in their own countries. However, South Korea has not yet standardized the test.

Research objectives

In this study, we aimed to standardize the apple cancellation test for South Korea.

Research methods

We conducted the apple cancellation test on 223 normal people (20-80 years old) and wanted to present the cutoff score according to age.

Research results

The cutoff scores for diagnosing EN and AN were presented by age. At the same time, we also presented the time requirements for the test by age and education level.

Research conclusions

We have completed the standardization of the apple cancellation test for South Korea. We look forward to using it in more studies.

Research perspectives

It would be meaningful to standardize the various tests for each country to ensure accurate testing.