Non-guided self-learning program for high-proficiency optical diagnosis of diminutive and small colorectal lesions: A single-endoscopist pilot study

Abstract

BACKGROUND

The implementation of optical diagnosis (OD) of diminutive colorectal lesions in clinical practice has been hampered by differences in performance between community and academic settings. One possible cause is the lack of a standardized learning tool. Since the factors related to better learning are not well described, strong evidence upon which a consistent learning tool could be designed is lacking. We hypothesized that a self-designed learning program may be enough to achieve competency in OD of diminutive lesions of the colon.

AIM

To assess the accuracy of OD of diminutive lesions in real colonoscopies after application of a self-administered learning program.

METHODS

This was a single-endoscopist prospective pilot study, in which an experienced endoscopist followed a self-designed, self-administered learning program in OD of colorectal lesions. An assessment phase divided in two halves with a 6-mo period in between without performance of OD was developed in a population-based colorectal cancer screening program. The accomplishment of the Preservation and Incorporation of Valuable Endoscopic Innovations criteria and performance measures were calculated overall and in the two halves of the assessment phase, assessing their response to the 6-mo stopping period. The evolution of performance through blocks of 50 lesions was also assessed.

RESULTS

Overall, 152 patients and 522 lesions (≤ 5 mm: 399, and 6-9 mm: 123) were included. The negative predictive value for the OD of adenoma in rectosigmoid lesions diagnosed with high confidence was 91.7% [95% confidence interval (CI): 87.3-96.6]. The proportion of agreement on surveillance interval between OD and pathological diagnosis was higher than 95%. Overall accuracy for diminutive lesions diagnosed with high confidence was 89.5% (95%CI: 86.3-92.7). The overall accuracy of OD was similar in the two halves of the assessment phase [90.1 (95%CI: 85.6-94.7) vs 88.2 (95%CI: 87.9-95.9)]. All the other performance parameters were also equivalent, except for specificity. Specificity, negative predictive value and accuracy were the parameters most affected by the stopping period between the two halves. Upon analyzing trends on blocks of 50 lesions, an improvement on sensitivity (P = 0.02) was detected only in the first half and an improvement on accuracy (P = 0.01) was detected only in the second half.

CONCLUSION

A self-administered learning program is sufficient to achieve expert-level OD. To maintain performance, continuous practice is needed, with a refresher course following any long non-practice period.

Key Words: Optical diagnosis, Accuracy, Learning, Polyp, Colonoscopy, Education

Core tip: The learning process for optical diagnosis (OD) of diminutive colorectal polyps is not standardized, and this may influence the described differences in OD performance between community and academic settings. Our study shows that an individual following a self-designed and self-administered learning program is able to reach the expert level of OD performance completely fulfilling the criteria of Preservation and Incorporation of Valuable Endoscopic Innovations. However, continuous practice is needed to maintain performance and, if a non-practice period is expected, a refresher course is needed to avoid a significant drop in performance parameters.

INTRODUCTION

Optical diagnosis (OD) of gastrointestinal epithelial lesions has become a reality due to the development of new image enhancing technologies. The ability to perform in situ differentiation of adenomatous and hyperplastic colorectal lesions has led to the proposal of a resect-and-discard strategy for the management of diminutive (≤ 5 mm) polyps[1]. Following this strategy, diminutive lesions would be resected and discarded after an adenoma high-confidence OD has been made, while rectal diminutive lesions with an OD of hyperplastic would be left in place. This strategy has been shown to be cost-efficient[2].

An excellent accuracy of OD is a requirement for applying such a strategy, and it has been shown to be so in many studies, most of them performed in academic centers[3]. However, this good accuracy has not been well replicated in community settings[4,5]. Learning of OD is key for its implementation in clinical practice, and the lack of standardized learning tools may explain part of the problem. A wide variety of learning tools has been described, including classroom type[6], self-directed computer-based[3] or web-based teaching programs[7]. Still pictures, videos or both have been used to explain the optical features of each type of polyp[8,9]. However, there are no head-to-head comparisons between learning tools and most of them have not been validated.

Moreover, people learn at different rates, as has been shown by some studies monitoring the learning curve of OD. Some learners never get competency in OD, while others need long-term monitoring[4,6]. Unfortunately, since the factors related to better learning are not well described, the strong evidence upon which a consistent learning tool could be designed is lacking. Despite these challenges, we hypothesized that a self-designed learning program may be enough to achieve competency in OD of diminutive lesions of the colon.

Our study was designed according to the following aims: (1) to assess the accuracy of OD of diminutive lesions in real colonoscopies from a colorectal carcinoma (CRC) screening program using narrow band imaging (NBI) and the NBI International Colorectal Endoscopic (NICE) classification after following a non-guided self-administered learning program; and (2) to describe the OD learning curve by analyzing which parameters may be more suitable for monitoring competency.

MATERIALS AND METHODS

Study design and population

This was a single-endoscopist prospective pilot study, in which an experienced endoscopist (> 500 colonoscopies per year and adenoma detection rate of 68%) followed a self-designed, self-administered learning program for OD of colorectal lesions. In this learning program, the NICE classification was reviewed and a published set of still pictures[8] was used to identify the main optical characteristics of hyperplastic and adenomatous polyps under NBI. Then, the NICE classification was put into practice on 50 consecutive colorectal lesions identified in CRC screening colonoscopies. The endoscopist reviewed the pathological records, when available, comparing this diagnosis with the provided OD. A detailed evaluation of inconsistencies was performed and diagnostic disagreements were reviewed with the pathologist.

After completing the learning program, an assessment phase was begun in which individuals scheduled for colonoscopy in the setting of the Valencian Government Colorectal Screening Program were consecutively included. This screening program is based on results from the immunological fecal occult blood test administered every 2 years and colonoscopy administered in cases of positivity. Exclusion criteria were poor quality preparation (Boston < 2 in any colon segment), incomplete colonoscopy, inflammatory bowel disease, coagulopathy that precluded taking samples, or unwillingness to participate in the study. This assessment phase was divided in two halves, with a predefined stopping period of 6 mo in between, in which no OD was performed. No OD refresher course was given before the beginning of the second phase.

Colonoscopy procedure

For bowel preparation, a split-dose scheme using sodium picosulphate plus magnesium citrate (Citraflet®; Casen Recordati, S.L., Zaragosa, Spain) or 2-L polyethylene glycol (PEG) plus ascorbate (Moviprep®; Salix Pharmaceuticals, Bridgewater, NJ, United States) was administered. All colonoscopies were performed using high-resolution CF-HQ190AL or CF-H190L endoscopes (Olympus, Optical Co., Ltd., Tokyo, Japan) and a video endoscope system (EVIS EXERA III; Olympus).

Variables

Data on age, sex, and personal and familiar histories of colon polyps or CRC were recorded. For every lesion, data on size, morphology (following the Paris classification[10]), location, NICE classification[11] group, and final OD were also recorded. All data were prospectively included in a database built in Access 2003 (Microsoft Corp., Redmon, WA, United States). Pathological diagnosis was introduced in the database by a researcher involved neither in the colonoscopies nor in the OD process. Therefore, during this phase of the study, the endoscopist was blind to the pathological report and no feedback was provided. Only diminutive lesions (1-5 mm) or small lesions (6-9 mm) were considered for the analysis.

The optical and pathologic diagnostics were compared for the diagnosis of adenoma vs non-adenomatous lesions, considering pathology as the gold standard. For analysis purposes, hyperplastic polyps, sessile serrated polyps, inflammatory polyps, and biopsies informed as normal were considered as non-adenomatous lesions.

Study end-points

The primary end-point was the Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) criteria[1] accomplishment at the end of the study. The final surveillance recommendation when using OD was the combination of OD of diminutive lesions and the pathology report of larger lesions. The concordance between the recommended follow-up from OD and from pathology was calculated for the three main currently available guidelines (European Union[12], European Society of Gastrointestinal Endoscopy[13], and American Society for Gastrointestinal Endoscopy[14]). Patients in whom an in-situ surveillance recommendation could not be given (i.e., those with no diminutive lesions, with at least one polyp diagnosed with low confidence, or diagnosed with a CRC or a large polyp scheduled for endoscopic mucosal resection) were not included in this analysis. Secondary end-points were the evaluation of overall accuracy, sensitivity, specificity, positive and negative predictive values (PPV and NPV, respectively), and positive likelihood ratio for the diagnosis of adenoma. All performance values were calculated at the end of the first half, at the end of the study and during the assessment phase in groups of 50 lesions.

Sample size estimate on and statistical methods

To obtain a precision of 3% in the estimation of the accuracy of OD for diminutive lesions, using a bilateral 95% confidence interval (CI) and expecting an accuracy of 90%, at least 385 diminutive lesions had to be included.

Each patient’s and lesion’s characteristics were summarized by median (standard deviation) for continuous variables and by number (percentage) for categorical variables. Sensitivity, specificity, PPV and NPV, and positive likelihood ratio were calculated as measures of accuracy together with their 95%CIs. True positive and negative values were defined as an agreement between OD and histology. The Cochran-Armitage test for trend was used to determine if performance improved through blocks of 50 lesions in both halves of the study. P-values were two-sided, and differences were considered significant at P < 0.05. Analysis was performed by using the Stata statistical package, version 14.2 (Stata Corp, College Station, TX, United States). The results of this study are reported in accordance with Standards for Reporting of Diagnostic Accuracy guidelines[15].

RESULTS

From January 2015 to January 2017, 152 individuals who underwent a CRC screening colonoscopy were selected for study inclusion. Their main characteristics are summarized in Table 1. These patients harbored 522 lesions [1-5 mm in 399 (76.4%) and 6-9 mm in 123 (23.6%)], the main characteristics of which are summarized in Table 2.

Table 1 Patient characteristics, n = 152.

Characteristic	Value
Age, yr	61.1 ± 6.2
Female sex n (%)	56 (36.8)
Familiar history of CRC n (%)	33 (21.7)
Number of polyps	3.8 ± 3.0
Number of adenomas	2.7 ± 2.6
Number of advanced adenomas	0.6 ± 1.0

CRC: Colorectal cancer.

Table 2 Lesion characteristics n (%).

Characteristic		1-5 mm	6-9 mm
Number of lesions by size		399 (76.4)	123 (23.6)
Paris classification
	0-Ip	3 (0.7)	22 (17.9)
	0-Is	273 (68.4)	88 (71.5)
	0-IIa	117 (29.3)	11 (8.9)
	0-IIc	2 (0.5)	0
	0-IIa + IIc	3 (0.7)	0
	0-IIb	1 (0.2)	2 (1.6)
Pathology
	Adenoma	255 (63.9)	97 (78.9)
	Hyperplastic	106 (26.6)	17 (13.6)
	SSP	8 (2.0)	7 (5.6)
	Other	26 (6.5)	1 (0.8)
	Lost/not enough sample	4 (1.0)	1 (0.8)
Location
	Proximal	248 (62.2)	56 (45.5)
	Distal	151 (37.8)	67 (54.5)
Optical diagnosis1
	NICE 1	288 (72.2)	102 (82.9)
	NICE 2	110 (27.6)	20 (16.3)
High-confidence diagnosis		347 (87.0)	116 (94.3)

¹Only available for 520 lesions (1 lesion excluded because a poor approach to application of narrow band imaging, and 1 subepithelial lesion). NBI: Narrow band imaging; NICE: NBI International Colorectal Endoscopic classification system; SSP: Sessile serrated polyp.

PIVI criteria

Overall, 55 (59.8%) diminutive rectosigmoid lesions were diagnosed as hyperplastic and 34 (36.9%) as adenoma. One lesion was lost for analysis and two were categorized as normal mucosa. The NPV for the OD of adenoma in rectal lesions diagnosed with high confidence was 91.7% (95%CI: 87.3-96.6). In 59 patients (38.8%), an in-situ surveillance recommendation could not be given; these patients included 40 with at least one lesion diagnosed with low confidence, 7 with no diminutive lesion, 9 with a CRC or a malignant polyp diagnosed in the same colonoscopy, and 3 with large polyps suitable for endoscopic mucosal resection. The proportion of agreement on surveillance interval between OD and pathological diagnosis following the different guidelines for the remaining 92 patients is summarized in Table 3.

Table 3 Concordance between proposed surveillance interval between optical diagnosis and pathology, n = 93.

Guideline	Concordance		Too long		Too short
	n	% (95%CI)	n	% (95%CI)	n	% (95%CI)
EU	89	95.7 (91.9-100)	2	2.1 (0-21.4)	2	2.1 (0-21.4)
ESGE	90	96.8 (93.5-100)	2	2.1 (0-21.4)	1	1.1 (0-20.5)
ASGE	89	95.7 (91.9-100)	3	3.2 (0-22.3)	1	1.1 (0-20.5)

ASGE: American Society of Gastrointestinal Endoscopy; CI: Confidence interval; ESGE: European Society of Gastrointestinal Endoscopy; EU: European Union.

Accuracy of OD

Regarding the OD with NBI, 520 lesions were classified as adenomas or hyperplastic polyps, with 347 (87.0%) diminutive lesions and 116 (94.3%) small lesions diagnosed with high confidence (Table 2).

The performance values for the OD of small and diminutive lesions are summarized in Table 4. Overall accuracy for diminutive and small lesions diagnosed with high confidence was 89.5% (95%CI: 86.3-92.7) and 99.1% (95%CI: 97.4-100.0) respectively. Values were, as expected, much lower for lesions diagnosed with low confidence (Table 4). These values did not differ significantly when comparing location (distal vs proximal) and morphology (sessile vs flat) (data not shown).

Table 4 Overall diagnostic performance of narrow band imaging and the Narrow Band Imagig International Colorectal Endoscopic classification system.

Parameter	High-confidence		Low-confidence
	1-5 mm, n = 347	6-9 mm, n = 115	1-5 mm, n = 51	6-9 mm1, n = 7
Sensitivity	97.0 (95.2-98.8)	100.0	76.0 (64.3-87.7)	N/A
Specificity	74.3 (69.4-78.6)	94.4 (90.2-98.6)	46.1 (32.4-59.8)	57.1 (20.4-57.1)
PPV	88.5 (84.6-91.4)	99.0 (97.2-100.0)	57.6 (44.0-71.2)	0
NPV	92.3(89.1-94.8)	100.0	66.7 (53.8-79.6)	100.0
LR+	3.8 (1.2-4.8)	18.0 (10.1-23.9)	1.4 (-1.8-4.6)	N/A
Accuracy	89.5 (85.7-92.3)	99.1 (97.4-100.0)	60.7 (47.3-74.1)	57.1 (20.4-57.1)

Data are given as % (95%CI).

¹No adenoma was diagnosed with low-confidence.

CI: Confidence interval; LR: Likelihood ratio; N/A: Non-applicable; NPV: Negative predictive value; PPV: Positive predictive value.

Learning curve assessment

The overall accuracy of OD was similar in the two halves of the study [90.1% (95%CI: 85.6-94.7) vs 88.2 (95%CI: 87.9-95.9)]. All the other performance parameters were also equivalent, except for specificity (Table 5). The NPV for adenoma in rectosigmoid lesions and agreement on surveillance intervals were also similar between both halves of the study (Table 5).

Table 5 Comparison of the diagnostic performance measures and Preservation and Incorporation of Valuable Endoscopic Innovations criteria fulfillment between the two halves of the study.

Parameter		1st half, n = 1651	2nd half, n = 1821
Sensitivity		96.1 (93.1-99.0)	97.6 (95.4-99.8)
Specificity		82.0 (76.1-87.9)	65.4 (58.5-72.3)
PPV		90.1 (85.5-94.7)	87.2 (82.3-92.0)
NPV		92.6 (88.6-96.6)	91.9 (87.9-95.9)
Accuracy		90.1 (85.6-94.7)	88.2 (87.9-95.9)
NPV rectosigmoid lesions		92.3 (87.3-96.6)	90.5 (86.2-94.8)
Surveillance interval agreement
	UE	100.0	93.3 (89.7-96.9)
	ESGE	100.0	95.1 (92.0-98.2)
	ASGE	100.0	93.3 (91.6-98.2)

Data are given as % (95%CI).

¹Only for diminutive lesions diagnosed with high-confidence.

ASGE: American Society of Gastrointestinal Endoscopy; ESGE: European Society of Gastrointestinal Endoscopy; EU: European Union; NPV: Negative predictive value; PIVI: Preservation and Incorporation of Valuable Endoscopic Innovations; PPV: Positive predictive value.

Figure 1 depicts the evolution during time of OD performance of diminutive lesions. Specificity, NPV and accuracy were the parameters most affected by the stopping period between the two halves. However, sensitivity and the percentage of lesions diagnosed with high confidence are more robust parameters. Analyzing trends on blocks of 50 lesions showed an improvement in specificity (P = 0.0001) and NPV (P = 0.00001) in both halves. However, an improvement in sensitivity (P = 0.02) was detected only in the first half and an improvement in accuracy (P = 0.01) was detected only in the second half. There was no significant improvement in the percentage of lesions diagnosed with high confidence in either of the two halves as the trainee progressed through lesion batches.

Figure 1 Performance parameters by blocks of 50 lesions during the two halves of the study. The yellow line represents the 6-mo stopping period. NPV: Negative predictive value; PPV: Positive predictive value.

DISCUSSION

Our study shows that a good accuracy, reaching an expert level, and complete fulfillment of the PIVI criteria can be accomplished by self-learning. At the end of the study, the NPV for the OD of adenoma in rectal lesions was 91.7% and the proportion of agreement in the surveillance intervals between OD and pathology was higher than 95%.

Previous studies on OD learning have shown conflicting results. When it comes to performing OD in real colonoscopies, several studies and a meta-analysis have shown lower levels of performance (i.e., not fulfilling the PIVI criteria) in community hospitals than in academic centers, despite a structured learning program having been followed[4,5,16]. On the contrary, other authors have shown that trainees without previous experience in NBI can meet PIVI thresholds after following a standardized learning program[17]. One of the possible explanations for this discrepancy may be the different design of the learning tool.

We used a validated set of still pictures followed by a practice on real colonoscopies with auto-administered feedback, hypothesizing that the latter would ease the transition from still pictures to real practice and shorten the learning curve. For the initial learning steps, several training modules have been used in the literature, including classroom-type[8,18], computer training[19] and web-based[20] modules. All systems may have similar efficacy as it has been suggested in a recent report showing that self-learning using a computer-based program with pictures and videos is as efficient as a classroom-type teaching session for learning OD[21]. Therefore, the key to efficacy of the learning program may be more in other adds-on or modifications.

Other authors have also shown a good efficacy of learning when introducing an in vivo phase during the learning program, with a pre-defined number of colonoscopies[22] or lesions[6]. In our study, 50 lesions were sufficient to meet the PIVI criteria at the end of the assessment phase.

Other modifications that have been tested in the literature are refresher teaching sessions and periodic feedback. We did not introduce any refresher session, not even before the beginning of the second period of the assessment phase after the 6-mo stopping period, and it did not affect the final results on efficacy. Any feedback was allowed during the assessment phase and the endoscopist was blinded to the pathology results. Regarding these two modifications, there is some controversy in the literature. Paggi et al[23] introduced refresher teaching sessions every 2 mo and monthly feedback on individual performance, achieving an overall NPV for adenoma in rectosigmoid lesions of 91.3% and more than 90% of agreement on surveillance intervals. Patel et al[17] delivered periodic feedback to all the participants in a prospective study and were able to show an overall NPV for high-confidence diagnosis of rectosigmoid lesions of 94.7% and a surveillance interval agreement of 91.2%. However, a randomized trial was not able to show any influence of feedback on final performance[6].

We planned a stopping period at the middle of the study to investigate if a non-practice period could influence performance and to detect which parameters were affected the most. Following the stopping period (which was not followed by a refresher course), almost all performance parameters dropped significantly. Specificity was the most affected parameter, and it took 200 lesions to reach previous levels. On the other hand, sensitivity was very resistant to inactivity. Accuracy dropped from 0.89 to 0.77, and it took 150 lesions to reach 0.90. NPV for adenoma in rectal lesions also dropped significantly, from 0.90 to 0.67.

Regarding trends for improvement through blocks of 50 lesions, a significant improvement was detected in both halves for specificity and NPV, suggesting that the number of false positives and false negatives are only significantly reduced after ongoing practice. The significant trend for improvement of accuracy only in the second period suggests that if a long non-practice period has occurred, a refresher course in OD is needed. A previous study[24] of 12 endoscopists evaluating 80 videos at 12 wk apart found a significant improvement in accuracy in both periods; however, that study did not include real colonoscopies.

The strength of the current study described herein is its design as a single-endoscopist study, which allowed for detailed analysis of the learning process. Another strength is that the PIVI criteria on surveillance agreement has been calculated for the most widely applied international guidelines, showing that learning is consistently strong under different circumstances. However, some issues may limit generalizability. First, the single-endoscopist study design carries the risk of the results being dependent on the trainee´s characteristics. Studies including several endoscopists have shown that despite an overall good performance, many individuals do not reach the PIVI thresholds[4,22] and that in many cases a continuous monitoring is needed. Nonetheless, the statement that an efficient self-learning program is possible when the trainee is highly motivated seems conclusive.

Another limitation is that all patients belong to a FIT-positive population. In this situation, the probability of finding polyps is higher and this may enhance the learning process. The diagnosis of sessile serrated polyp was not considered, and these polyps were included in the non-adenomatous group. However, this only comprised 2% of samples and none of the hyperplastic lesions were more than 10 mm, having little relevance to the final results.

In conclusion, a self-administered learning program including real colonoscopies is sufficient to learn OD at an expert level. However, continuous practice is needed to maintain performance and a refresher course is needed if a long non-practice period occurs. Performance values behave differently after a stopping period, and this should be taken into account when planning a monitoring program.

ARTICLE HIGHLIGHTS

Research background

The resect-and-discard strategy for the management of diminutive colon polyps is a paradigm shift based on an accurate optical diagnosis (OD). Such a high accuracy has only been achieved by experts, while the performance in community hospitals does not reach thresholds that would allow its universal implementation. The lack of a standardized learning tool for OD of colon lesions may contribute to this problem.

Research motivation

Although several learning tools have been described, most of them are not validated and there is a great variability in their components and designs. We hypothesized that self-learning of OD is feasible and that accuracy thresholds can be achieved with a self-administered program. A detailed description of the learning process can provide valuable information for the design of an OD learning system.

Research objectives

We aimed to assess the accuracy of OD of diminutive lesions in real colonoscopies using the International Colorectal Endoscopic classification system for narrow band imaging after following a non-guided self-administered learning program. We also aimed to describe in detail the learning process by analyzing which parameters may be more suitable for monitoring competency.

Research methods

An experienced endoscopist followed a self-designed, self-administered learning program in OD of colorectal lesions. Then, OD was applied to lesions detected in colorectal cancer screening colonoscopies. The study period was divided in two halves, with a 6-mo period in between with no performance of OD. Sensitivity, specificity, predictive values and accuracy of the OD compared to the pathological report were calculated for overall results and for the two halves of the study. The accomplishment of the Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) criteria and the evolution of performance parameters through blocks of 50 lesions were also assessed.

Research results

Overall, 152 patients and 522 lesions were included in the analysis. Regarding the accomplishment of the PIVI criteria, the negative predictive value for the OD of adenoma in rectal lesions diagnosed with high confidence was 92.6% (95% confidence interval: 86.4-97.6) and the proportion of agreement on surveillance interval between OD and pathological diagnosis following the different guidelines was over 95%. Overall accuracy for diminutive lesions diagnosed with high confidence was 89.5% (95% confidence interval: 85.7-92.3). Specificity, negative predictive value and accuracy were the parameters most affected by the stopping period between the two halves. Analyzing trends on blocks of 50 lesions showed an improvement in sensitivity (P = 0.02) only in the first half of the study and an improvement on accuracy (P = 0.01) only in the second half.

Research conclusions

This study shows that a self-administered learning program based on still pictures plus an in vivo phase with auto-feedback is feasible to reach quality standards on OD of colorectal lesions. It also shows that a non-practice period deteriorates performance, and in that case a refresher course seems advisable. These results have practical implications in the design of OD learning tools and in the development of a quality monitoring system.

Research perspectives

These data have become the base for the design and validation of a self-administered learning tool that are currently in process. The efficacy of this kind of tool should be tested with endoscopists having different levels of experience and being from different backgrounds.