Inter- and intraobserver agreement in computed tomography enterography in inflammatory bowel disease

Abstract

AIM

To evaluate intra- and interobserver agreement in imaging features in inflammatory bowel disease and comparison with fecal calprotectin (FC) levels.

METHODS

Our institutional computed tomography enterography (CTE) database was retrospectively queried to identify patients who underwent CTE from January 2014 to June 2015. Patient inclusion criteria were confirmed inflammatory bowel disease (IBD) and FC collected < 4 mo after CTE without any change in clinical treatment or surgical treatment during this interval. The exclusion criterion was poor image quality. Two blinded abdominal radiologists, with 12 and 3 years of experience analyzed the CTE regarding localization (small bowel, colonic, both, or no disease detected); type of IBD (inflammatory, stenosing, fistulizing, > 1 pattern, or normal); and signs of active disease (present or absent). In 42 of 44 patients evaluated, routine CTE reports were made by one of the readers who re-evaluated the CTEs ≥ 6 mo later, to determine the intraobserver agreement. FC was considered a sign of disease activity when it was higher than 250 μg/g.

RESULTS

Forty-four patients with IBD (38 with Crohn’s disease and 6 with ulcerative colitis) were included. There was a moderate interobserver agreement regarding localization of IBD (κ = 0.540), type of disease (κ = 0.410) and the presence of active signs in CTE (κ = 0.419). There was almost perfect intraobserver agreement regarding localization, type and signs of active disease in IBD. The κ values were 0.902, 0.937 and 0.830, respectively. After a consensus between both radiologists regarding inflammatory activity in CTE, we found that 24 (85.7%) of 28 patients who were classified with active disease had elevated FC, and six (37.5%) of 16 patients without inflammatory activity in CTE had elevated FC (P = 0.003). The correlation between elevated FC and the presence of active disease in CTE was significant (κ = 0.495, P = 0.001).

CONCLUSION

We found almost perfect intraobserver and moderate interobserver agreement in the signs of active disease in CTE with concurrence of high FC levels.

Key Words: Crohn’s disease, Ulcerative colitis, Computed tomography, Fecal calprotectin, Inflammatory bowel disease activity

Core tip: Evaluation of active inflammation in inflammatory bowel disease (IBD) is not simple and demands a multidisciplinary approach. A few studies have evaluated the interobserver agreement in computed tomography enterography (CTE) findings in patients with active inflammation in IBD. Intraobserver agreement was only evaluated in other imaging modalities. This study showed for the first time intraobserver agreement for CTE signs of active IBD and its correlation with fecal calprotectin (FC) levels. We found almost perfect intraobserver and moderate interobserver agreement in the characterization of signs of active disease in CTE, in concurrence with high FC levels in patients with IBD.

INTRODUCTION

Inflammatory bowel disease (IBD) is considered an important healthcare problem worldwide, with a high morbidity and poor quality of life. The treatment of IBD is directed according to the analysis of clinical, endoscopic, laboratory and imaging features. The presence of active inflammatory disease plays a central role for tailored treatment planning. Nevertheless, the evaluation of active inflammation is not a simple task and demands a multidisciplinary approach.

Computed tomography enterography (CTE) has become an important imaging modality for evaluation of IBD due to its accessibility and reliability. CTE provides visualization of the entire gastrointestinal tract, allowing the differentiation in inflammatory, stenosing and fistulizing diseases, and enables the characterization of active disease[1]. Imaging features of active inflammatory disease include mucosal hyperenhancement, wall thickening, mural stratification, prominent vasa recta (comb sign), mucosal ulcerations, enlarged mesenteric lymph node and mesenteric fat stranding[2-5].

Magnetic resonance enterography (MRE) and CTE are equally accurate for assessment of disease activity[6]. However, CTE is more widely available, especially in developing countries, less time consuming, and more reproducible in terms of image quality[7]. Despite the need for intravenous contrast media and exposure to radiation, CTE is still widely used for evaluation of patients with IBD. The use of dose modulation can reduce CTE radiation dose, increasing the use of this method[8].

Fecal calprotectin (FC) is a zinc- and calcium-binding protein that is found in bowel-activated neutrophils during mucosal damage, and is considered to be one of the most important biomarkers for evaluation of disease activity in IBD. It is a noninvasive and low cost method, which measures FC directly from stool samples. Increased FC levels have been found in IBD, with close correlation with endoscopic scores of inflammation[5,9].

The aim of this study was to evaluate inter- and intraobserver agreement in detection of inflammatory signs in CTE, in comparison with FC levels.

MATERIALS AND METHODS

Study design

Institutional review board approval was obtained and the requirement for informed written consent was waived. Our institutional CTE database was retrospectively queried to identify patients who underwent CTE from January 2014 to June 2015.

Patient inclusion criteria for this study were confirmed inflammatory bowel disease and FC collected < 4 mo from the date of CTE, without any change in clinical treatment or surgical treatment during this interval. The exclusion criterion was poor image quality.

The CTE images of these patients were anonymized and reviewed by two abdominal radiologists (A.C.X. and C.D.Y. with 12 and 3 years of experience as an attending gastrointestinal radiologist) blinded for clinical, laboratory, endoscopic findings and previous reports of the patients. Despite the lower experience time in abdominal radiology, Reader 2 (A.C.X.) presented more experience in CTE.

In 42 of 44 patients evaluated, the routine CTE reports were made by Reader 2, who re-evaluated the CTEs ≥ 6 mo, to minimize the recall bias, in order to determine the intraobserver agreement.

CTE technique

CTE examinations were performed using a standardized clinical protocol on a 64-channel CT scanner (Brilliance, Philips Medical Systems, Eindhoven, the Netherlands; and Discovery HD 750, General Electric Healthcare, Waukesha, WI, United States). Patients fasted for ≥ 6 h and ingested 1.5 L of a polyethylene glycol solution in 50 min to distend the small bowel. Each patient received 10 mg intravenous N-butylhyoscine bromide, to reduce bowel peristalsis and 8 mg intravenous ondansetron to reduce nausea and vomiting.

CTE images were acquired after intravenous injection of 2.0 mL/kg contrast agent (iopromide; Bayer, Berlin, Germany), containing 623 mg/mL iodine, at a rate of 4 mL/s, followed by 25 mL saline. Bolus-tracking software was used to trigger the arterial phase scans at 20 s after contrast enhancement of the upper abdominal aorta to an attenuation threshold of 150 HU. The enterographic phase was timed to start at 60 s after the start of contrast injection. Contrast-enhanced CT was performed using the following scanning parameters: 250 mA, 120 kVp, 0.5-s tube rotation time, and pitch 1.375. A 2.0-mm section thickness was used and images were reconstructed after every 1.5 mm.

Image evaluation

The two abdominal radiologists after a specific training analyzed the CTE in terms of localization (small bowel, colonic, both or no disease detected); type of IBD (inflammatory, stenosing, fistulizing, > 1 pattern, or normal); and signs of active disease (present or absent). Active disease was defined as the presence of ≥ 2 of the following findings: (1) mucosal hyperenhancement; (2) wall thickening with mural stratification; (3) hypervascularity of the involved mesentery (comb sign); (4) mucosal ulcerations; (5) enlarged mesenteric lymph node; and (6) mesenteric fat stranding (Figure 1).

Figure 1 Computed tomography enterography shows signs of inflammatory activity in patients with inflammatory bowel disease. A: Mucosal hyperenhancement, wall thickening with mural stratification (arrowheads), mesenteric fat stranding (arrows); B: Mucosal ulcerations (arrow); C: Hypervascularity of the involved mesentery (comb sign) (arrows); D: Enlarged mesenteric lymph nodes (arrowheads).

FC

Collected fecal samples used for FC measurements were stored and shipped on ice to Alvaro Laboratory (Cascavel, Brazil), where the FC levels were determined using a quantitative ELISA (BÜHLMANN fCAL^® ELISA), using a standard method. The detection limits of this ELISA kit for FC range from 30 to 1800 μg/g. Levels above 250 μg/g were interpreted as disease activity.

Statistical analysis

The data were analyzed using the statistical program SPSS version 22.0 and MINITAB 16.0. The χ² test and Mann-Whitney test were used to compare variables between two groups. For all tests, P < 0.05 was considered statistically significant. Interobserver agreement was assessed using weighted κ with statistics. κ values were interpreted as follows: 0-0.20, slight agreement; 0.21-0.40, fair agreement; 0.41-0.60, moderate agreement; 0.61-0.80, substantial agreement; and 0.81-1.0, almost perfect agreement. The statistical methods of this study were reviewed by Mr. Valdecir Marvulle.

RESULTS

One hundred and fifty-one patients underwent CTE during the selected period. Fifty-seven patients had confirmed IBD. We excluded 13 patients: 12 from whom FC was measured > 4 mo after CTE and one patient who had poor CTE image quality. The final study population consisted of 44 patients (Figure 2). The median interval between CTE and FC measurement was 58.7 (range: 0-120) d.

Figure 2 Selection of patients for the retrospective study with all the patients undergoing fecal calprotectin within 4 mo. CTE: Computed tomography enterography; IBD: inflammatory bowel disease.

Among 44 patients, 25 were women (56.8%), with a mean ± SD age of 49 ± 25.4 years, with Crohn’s disease (CD) (n = 38) and ulcerative colitis (UC) (n = 6). Thirty patients (68.2%) had elevated FC (> 250 μg/g), and the mean FC value was 496 ± 706.31 μg/g (Table 1).

Table 1 Patient characteristics (n = 44) n (%).

Variable	Value
Sex
Male	19 (43.2)
Female	25 (56.8)
Age at CTE (yr)	49.0 ± 25.4
IBD
Crohn’s disease	38 (86.3)
Ulcerative colitis	6 (13.7)
Fecal calprotectin
Minimum	30
Maximum	1800
Mean ± SD	496 ± 706
> 250 μg/g (%)	30 (68.2)

CTE: Computed tomography enterography; IBD: Inflammatory bowel disease.

Localization of the disease was defined by Reader 1 in the small bowel in 18 patients (40.9%), four (9.1%) in the colon, nine (20.5%) in both, and no disease in 13 (29.5%). By Reader 2, the classification was: small bowel in 12 patients (27.3%), colon in seven (15.9%), 17 (38.6%) in both, and no disease in eight (18.2%). There was a moderate interobserver agreement regarding localization of the disease (κ = 0.540) (Table 2).

Table 2 Interobserver agreement in computed tomography enterography (n = 44).

CTE variables	Reader 2						%1	κ value	P value
Disease localization
Reader 1	SB	C	B	ND	Total		65.9	0.54	< 0.001
SB	10	0	8	0	18
C	0	3	0	1	4
B	0	0	9	0	9
ND	2	4	0	7	13
Total	12	7	17	8	44
Type of IBD
Reader 1	I	S	F	M	N	Total	54.5	0.41	< 0.001
I	9	1	0	2	1	13
S	1	6	0	1	0	8
F	0	0	1	2	0	3
M	2	2	1	2	0	7
N	5	0	2	0	6	13
Total	17	9	4	7	7	44
Signs of active disease
Reader 1	Present		Absent		Total		70.4	0.419	0.002
Present	19		2		21
Absent	11		12		23
Total	30		14		44

¹Percentage of agreement. B: Both; C: Colon; CTE: Computed tomography enterography; F: Fistulizing; I: Inflammatory; IBD: Inflammatory bowel disease; M: More than one pattern; N: Normal; ND: No disease detected; S: Stenosing; SB: Small bowel.

Regarding the type of IBD, Reader 1 classified 13 (29.5%) patients as inflammatory, eight (18.3%) as stenosing, three (6.8%) as fistulizing, seven (15.9%) as > 1 pattern, and 13 (29.5%) as normal. Reader 2 classified 17 (38.6%) as inflammatory, nine (20.5%) as stenosing, four (9.1%) as fistulizing, seven (15.9%) as > 1 pattern, and seven (15.9%) patients as normal. The interobserver agreement regarding the type of IBD was moderate (κ = 0.410) (Table 2).

Reader 1 classified 21 (48%) patients as having active disease and Reader 2 classified 30 (68%) (Table 2). The weighted quadratic κ value for classifying the IBD as active or not was 0.419, indicating moderate agreement (Table 2).

There was almost perfect intraobserver agreement regarding localization, type and signs of active disease in IBD. The κ values were 0.902, 0.937 and 0.830, respectively (Table 3).

Table 3 Intraobserver agreement in computed tomography enterography (n = 42).

CTE variables	Reader 2 (2nd)						%1	κ value	P value
Disease localization
Reader 2 (RE)	SB	C	M	ND	Total		92.8	0.902	< 0.001
SB	12	0	3	0	15
C	0	7	0	0	7
B	0	0	12	0	12
ND	0	0	0	8	8
Total	12	7	15	8	42
Type of IBD (n = 44)
Reader 2 (RE)	I	S	F	M	N	Total	95.1	0.937	< 0.001
I	15	0	0	0	0	15
S	0	8	0	0	0	8
F	0	0	3	0	0	3
M	2	0	0	6	0	8
N	0	0	0	0	8	8
Total	17	8	3	6	8	42
Signs of active disease
Reader 2 (RE)	Present		Absent		Total		92.9	0.83	< 0.001
Present	28		3		31
Absent	0		11		11
Total	28		14		42

¹Percentage of agreement. 2^nd: Second evaluation; SB: Small bowel; B: Both; C: Colon; CTE: Computed tomography enterography; F: Fistulizing; I: Inflammatory; IBD: Inflammatory bowel disease; M: More than one pattern; N: Normal; ND: No disease detected; RE: Routine evaluation; S: Stenosing.

After a consensus between both radiologists regarding signs of active disease in CTE, we found that 24 (85.7%) of 28 patients who were classified as having active disease had elevated FC, and six (37.5%) of 16 patients without inflammatory activity in CTE had elevated FC (P = 0.003). The correlation between elevation of FC (> 250 μg/g) and presence of active disease in CTE was significant (κ = 0.495, P = 0.001). As such, using a Mann-Whitney test, the FC levels were significantly higher in patients deemed as having active disease in CTE (P = 0.004) (Figure 3).

Figure 3 Fecal calprotectin levels in patients with and without signs of active inflammatory disease in computed tomography enterography. CTE: Computed tomography enterography.

DISCUSSION

Our study showed almost perfect intraobserver and moderate interobserver agreement in classifying IBD as active disease. We considered that the intraobserver was better than the interobserver agreement probably due to the greater experience in CTE of Reader 2. However, it must be considered that, despite the interval of ≥ 6 mo from routine evaluation to the second one, and previous anonymization of the patients’ data, a recall bias might have occurred.

A few studies evaluated that the interobserver agreement for each CTE finding of active inflammation resulted in a moderate to substantial concordance, with κ values ranging from 0.43 to 0.83[7,10]. Their interobserver agreement was higher for mural hyperenhancement[10]. However, in clinical practice, the final interpretation of the radiologists usually is more relevant than the presence of each imaging feature alone.

Siddiki et al[11] evaluated the interobserver agreement regarding the final interpretation of the radiologists as active or inactive, which is similar to our study, and demonstrated a substantial interobserver agreement (κ = 0.76). One possible reason for a higher interobserver agreement is the fact that they classified the patients into four groups (definitely active, suspicious, inactive and absent) and then the suspicious subtype was considered as active for statistical analysis, which may have improved the concordance.

In contrast, we found almost perfect intraobserver agreement regarding localization, type, and inflammatory activity. To the best of our knowledge, this is the first study to evaluate intraobserver agreement in CTE, but there have been a few evaluations of other imaging modalities. De Franco et al[12] showed a substantial intraobserver agreement (κ = 0.71) in contrast-enhanced ultrasound parameters of active disease in patients with CD in the terminal ileum. Another MR enteroclysis study showed high intraobserver agreement in the evaluation of each active criterion alone (κ ranged from 0.61 to 1.0)[13].

The differences in interobserver agreement in our study in comparison with others may reflect the difference in CTE experience of the two radiologists, which, moreover, reflect the reality of most hospitals. This reinforces the need for objective and structured reports, such as magnetic resonance index of activity (MaRIA) used in MRE, which can improve the reproducibility of the reports, mainly between radiologists with different levels of experience in CTE[14]. Moreover, the better intraobserver agreement strengthens the need for a multidisciplinary team with experience in IBD in all specialties, including radiology. IBD is a complex condition, with a high morbidity, in which the patients benefit from being treated in a reference hospital by an engaged team with reproducible results.

After a consensus between the radiologists, we found a significant correlation between active inflammatory disease on CTE and high levels of FC (κ = 0.495, P = 0.001). Our findings are in line with those of prior studies that demonstrated good correlation between high levels of FC with endoscopic scores and CTE[15,16]. Arai et al[16] evaluated the correlation between FC, CTE and balloon-assisted enteroscopy in patients with IBD. The authors created a novel CTE score in which four imaging variables were evaluated in five predefined ileal-colonic segments, and each variable was scored from 0 to 4 per segment. The authors showed that the FC levels were well correlated with CTE score (r = 0.4018, P = 0.0011).

We also found that 85.7% of the patients who were classified as having active disease had elevated FC, opposed to 37.5% of patients without active inflammation on CTE who had elevated FC. FC is a biomarker that reflects intestinal mucosal damage, and using a cut-off point of 250 μg/g, as in our study, the sensitivity and specificity of detecting active inflammation in IBD are about 80%, when compared with endoscopy[15-17]. However, some other studies have shown that FC presents a better sensitivity than specificity, which could explain the false-positive results[4,18-20]. Furthermore, the best area under the curve was demonstrated in studies that correlated low FC levels with inactive disease[21,22]. Additionally, other authors have shown that an increase in FC levels may precede the onset of inflammation[23], but we did not follow-up the patients. The combination of these factors may have influenced these discordant results.

There were several potential limitations to our study. First, the small sample size and retrospective nature of the study, not allowing FC measurement and CTE to be performed on the same day. In addition, there was no correlation with the standard reference values, such as endoscopic or histological findings, and the interobserver agreement was only evaluated by one reader. Finally, we did not perform a follow-up of the patients with no inflammatory signs on CTE and high FC levels. Therefore, further prospective studies with larger patient populations, with multireader evaluation and with other correlations (e.g., laboratory, endoscopic and histological analysis) are needed to evaluate the role of each marker in the evaluation of patients with IBD.

In conclusion, we found almost perfect intraobserver and moderate interobserver agreement in the characterization of signs of active disease in CTE in concurrence with high FC levels in patients with IBD.

ACKNOWLEDGMENTS

The authors would like to thank Mr. Valdecir Marvulle, for his generous statistical advice for this manuscript, Mr. Joao Horvat and Mr. Lincoln Costa, for assistance in editing the manuscript.

COMMENTS

Background

The evaluation of active inflammation in inflammatory bowel disease (IBD) patients is not a simple task and demands multidisciplinary evaluation; being an important tool in patient management. Computed tomography enterography (CTE) provides visualization of the entire gastrointestinal tract, enabling the characterization of disease activity in IBD.

Research frontiers

A few studies have evaluated the interobserver agreement in CTE findings of active inflammation in IBD patients. However, the intraobserver agreement was only evaluated for other imaging modalities. In the present study, we aimed to evaluate the inter- and intraobserver agreement in the characterization of signs of active disease in CTE in comparison with fecal calprotectin (FC) levels.

Innovations and breakthroughs

This study evaluated for the first time intraobserver agreement in CTE signs of active IBD and their correlation with FC levels. The authors found almost perfect intraobserver agreement in the characterization of signs of active disease in CTE and significant correlation between active signs in CTE and high levels of FC.

Applications

This study strengthens the importance of CTE and FC in the evaluation of patients with IBD and reinforces the need for a multidisciplinary team with experience in IBD in all specialties, including radiology.

Terminology

The presence of active inflammatory disease plays a key role in tailored treatment planning in patients with IBD and CTE and FC which are important methods for evaluating IBD.

Peer-review

This is an interesting study. CTE is becoming a diagnostic modality for IBD recently due to its easy accessiblity, especially for Crohn’s disease. FC is confirmed correlation to the mucosal inflammation of the IBD.