Evaluation of a handheld creatinine measurement device for real-time determination of serum creatinine in radiology departments

Abstract

AIM: To assess the feasibility/accuracy of a commercial handheld device in the context of increased demand for point-of-care serum creatinine (SCr) determination.

METHODS: In this institutional review board-approved, prospective study, 401 patients referred for contrast-enhanced computed tomography were included at two centres. Capillary (c)SCr was determined using two devices A+B and venous (v)SCr was determined in the centre’s laboratory. Method comparison statistics for both centres and for vSCr<>1.2 mg/dL, receiver operating characteristic analysis, negative predictive values (NPV), sensitivity and specificity were calculated pre-/post-curve offset correction with vSCr.

RESULTS: Pearson’s coefficients for cSCr vs vSCr were: centre 1-A:0.93/B:0.92; centre 2-A:0.85/B:0.82 (all P < 0.0001). Overall correlation was better for vSCr > 1.2 mg/dL. The area under the receiver operating characteristic curves showed a high accuracy for cSCr, but the device underestimated SCr, which was confirmed by Bland-Altman plot. Addition of the offset correction factor to the original data from centre 1 resulted in an improvement in sensitivity for detecting patients at risk (> 1.2 mg/dL), whilst maintaining acceptable specificity and improving NPV.

CONCLUSION: This study showed the feasibility of SCr determination using the evaluated handheld device in a routine clinical setting. The device showed high sensitivity and high NPV, but may significantly underestimate SCr without offset correction to local laboratories.

Key Words: Contrast-medium-induced nephropathy, Nephrogenic systemic fibrosis, Point of care testing, Serum creatinine determination, StatSensor

INTRODUCTION

Contrast-medium-induced nephropathy (CIN)[1] and nephrogenic systemic fibrosis (NSF)[2] are two potentially life-threatening complications of iodinated contrast agents and gadolinium-based contrast agents (GBCA), respectively, which have attracted increasing attention over the last few years. In particular, patients with impaired renal function are at a high risk of developing CIN or NSF. Serum creatinine (SCr) is an important surrogate parameter for renal function and thus offers an opportunity for risk stratification by detecting patients at risk. The new European Medicines Agency guidelines also require mandatory SCr testing for some GBCA[3].

CIN occurs, especially in patients with heavily impaired renal function, after the application of iodinated contrast agents, e.g., in computed tomography (CT) or catheter angiography and leads to a significant increase in mortality and morbidity[4]. CIN is defined as a temporary percentage increase in SCr of more than 25% or as an absolute increase in SCr of more than 0.5 mg/dL in the first 24 h and up to 5 d after contrast medium application. The incidence of CIN in the general population is estimated to be 1%-2%, whereas up to 70% of patients with severe chronic kidney disease are at risk of developing clinically significant CIN[5]. A direct relationship with increased length of hospitalisation, cost and long-term morbidity has been reported. For those patients who require dialysis, a 30% in-hospital mortality rate and 80% 2-year mortality rate can be expected[6]. The first step in the prevention of CIN is therefore to identify patients at risk.

NSF is a rare systemic fibrosing disorder that was first described in the literature in 2000[2]. Currently, there are at least 500 recorded cases worldwide. The first cases were observed in 1997. Occurring almost exclusively in patients with heavily impaired renal function, NSF shows a very heterogeneous clinical appearance ranging from indurations of the skin as a hallmark of the disease to rare and potentially fatal cardiac fibrosis[7,8]. The reported 24-mo mortality rate of NSF was found to be between 20% and 48%. A causative relationship between GBCA and NSF has not been proven but seems likely at this time[9]. There is no consistent successful treatment for NSF.

Consequently, the determination of renal function before contrast-enhanced CT or magnetic resonance imaging (MRI) examinations or interventions is mandatory. A surrogate parameter for renal function in widespread use is SCr in conjunction with the modification of diet in renal disease (MDRD) formula to assess the estimated glomerular filtration rate (eGFR)[10]. Clinical laboratories can provide SCr values within approximately one hour, which is sufficient for scheduled examinations, e.g., inpatients, but insufficient for outpatients and emergency patients. Furthermore, routine blood sampling from outpatients seems inappropriate due to invasiveness, disruption of the workflow, waiting time and additional expense. The immediate availability of SCr values for patients in radiology with a non-specific history concerning potentially impaired renal function should foster patient safety and an improved workflow in radiology. Recently, a new handheld device for the determination of SCr and estimation of the eGFR, called StatSensor™ Creatinine (NovaBiomedical, Waltham, MA, United States) has become commercially available[11,12]. The aim of this study was to assess the feasibility of its broad clinical application in a radiology department and the accuracy of SCr measurements and estimation of the GFR in daily point-of-care testing compared with standard laboratory diagnostics.

MATERIALS AND METHODS

Ethic statement and patients

After approval of the local Institutional Review Board and informed written consent was obtained, 401 consecutive patients scheduled for contrast-enhanced CT were included in this prospective study performed at two large university hospitals: the Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim (referred to as centre 1) and the Institute of Clinical Radiology-Grosshadern, Munich (referred to as centre 2). Two hundred and one consecutive patients were included in centre 1 (127 male, 74 female; mean age = 62 ± 15 years; mean weight = 77.3 ± 18.1 kg) and 200 consecutive patients were included in centre 2 (127 male, 73 female, mean age = 62 ± 13 years; mean weight = 77.7 ± 14.9 kg).

Blood sampling

For the purpose of the determination and comparison of SCr values, three different SCr measurements were performed for each patient: two capillary (cSCr - StatSensor™) and one venous blood sample (vSCr - laboratory reference). The capillary blood (3-6 μL) was obtained by a small lancet puncture in the fingertip. The two capillary samples were performed as point-of-care testing, using two different handheld StatSensor™ devices (referred to as A and B) in each study centre, to allow measurement of interdevice agreement. As the patients already had venous access for GBCA administration, this access was used to draw blood for the venous blood sample (5 mL). SCr of the venous blood samples (vSCr) was determined by the local hospital laboratory.

Methods of creatinine determination and correction

The laboratory gold standard for SCr measurement is the Jaffé method, a colorimetric assay, which measures the reaction between creatinine and picric acid and can be accomplished using commercially available autoanalysers. The laboratories at the study centres were equipped with the following autoanalysers based on a device-specific modified Jaffé method: Siemens Dimension RXL (Siemens Healthcare, Erlangen, Germany) at centre 1 and Olympus AU2700 (Olympus, Tokio, Japan) at centre 2.

The technology of the creatinine handheld device is based on a different, enzymatic amperometric pathway: first creatinine is hydrolysed in creatine (catalysed by the enzyme creatinine amidohydrolase) and then hydrolysed to sarcosine (catalysed by amidinohydrolase). The oxidation of sarcosine (catalysed by sarcosine oxidase) produces hydrogen peroxide which oxidises the terminal indicator (Fe³⁺) at the working electrode to produce a current. This current, measured electrochemically, is proportional to the creatinine concentration in the sample. This method was standardised by Nova Biomedical against a laboratory enzymatic method (Vitros Kodak DT60; Ortho-Clinical Diagnostics, Rochester, NY, United States)[13]. The handheld device (360 g, 15.3 mm × 82.5 mm × 46 mm; Nova Biomedical, Boston, MA, United States) allows the measurement of SCr level within a range of 0.3-12 mg/dL and calculation of the GFR in about 30 s. Therefore, a small capillary blood sample of a few microlitres is sufficient.

Because of the lack of standardisation between the methodologies for creatinine determination in the laboratories and the thereby likely bias of the results, a re-evaluation of the obtained data was planned for centre 1. To evaluate, and if applicable, correct this bias a follow-up assessment was undertaken to calculate and implement a curve offset for the handheld device in order to standardise with the Siemens Dimension RXL used in the clinical laboratory at centre 1. Therefore, a method correlation and curve offset were calculated using the original data from centre 1 generated from the patient evaluation. The curve offset implemented was then added as a correction factor to the original data and the data re-analysed.

Statistical analysis

For statistical analyses, SAS 9.1 (Cary, NC, United States) and Microsoft Excel 2007 (Redmond, WA, United States) were used. The normal distribution of the data was confirmed beforehand by a Kolmogorov-Smirnov test. Descriptive statistical analyses of the creatinine determinations were performed including means and standard deviations for cSCR, vSCr and the differences. Throughout the entire analysis, statistical significance was assumed at P < 0.05. Further statistical analyses were subdivided into different parts. First, the correlations between the two determination methods (cSCr vs vSCr) were assessed with regard to statistical significance by Pearson’s correlation coefficients (r). Subgroup-wise analyses (stratified by centre, creatinine level and gender) were also performed. The cut-off for patients with a normal creatinine level was vSCr = 1.2 mg/dL (laboratory reference); higher values were considered to indicate renal impairment. Additionally, the correlation between cSCr and vSCr was assessed by linear regression. The linear regression is presented figuratively as a scatter plot, regression equation and correlation coefficient. Second, the agreement between the two methods was analysed. Therefore, to visualise the agreement, Bland-Altman plots were performed. The Bland-Altman plot shows the deviation between the two methods with different SCr levels. The red line indicates the average value of the differences between the two methods; the two yellow lines form the 95% reference range. Furthermore, paired t-tests and the corresponding mean difference confidence interval approach were used to assess deviations in the location of the two methods; the test procedure of Maloney and Rastogi was used for comparison of precision in paired data. Third, to estimate the accuracy of cSCr measured by the handheld device, receiver operating characteristics (ROC) analyses were performed. Fourth, to test the congruence of cSCr determination between the two handheld devices used (A/B), Pearson’s correlation coefficients and the test of Maloney and Rastogi were performed. In order to correct the bias between the two methods, the patient data from centre 1 were re-assessed after implementation of a curve offset correction. Slope and y-axis offset correction factors were computed. After applying the offset correction factor, the correlation and agreement between the two methods and the agreement between the two devices were re-analysed. Finally, before and after implementing the curve offset, sensitivity, specificity, positive and negative predictive values (PPV and NPV) were calculated for the cut-off level of 1.2 mg/dL.

RESULTS

All measurements were performed successfully. The means, standard deviations of cSCr (StatSensor™), vSCr (laboratory reference) and the differences between these two methods, for the two devices and for both centres before and after offset correction are summarised in Table 1. In a comparison of the two methods, the handheld device underestimated SCr compared with the laboratory reference in both study centres, which was confirmed by the Bland-Altman plot (Figure 1D). In centre 1, the Pearson correlation coefficient of cSCr vs cSCr was higher than in centre 2: r = 0.93 for handheld device A (P < 0.0001) in centre 1, r = 0.92 for handheld device B (P < 0.0001) (mean correlation coefficient at centre 1: r = 0.93), centre 2 r = 0.85 for handheld device A (P < 0.0001) and r = 0.82 (P < 0.0001) for handheld device B (mean correlation coefficient at centre 2: r = 0.84). The stratification with regard to gender revealed a slightly higher correlation for male patients (Table 2). The subgroup-wise analysis as a mean for both centres for a vSCr > 1.2 mg/dL showed a better correlation (device A 0.91, P < 0.0001 and device B 0.90, P < 0.0001) compared with vSCr values < 1.2 mg/dL (device A 0.66, P < 0.0001 and device B 0.59, P < 0.0001). The results of the linear regression corresponded to the Pearson correlation and are presented figuratively in Figure 1A for both centres and stratified for vSCr >< 1.2 mg/dL (Figure 1B, C). In the inter-method agreement, paired tests showed significant differences between cSCr and vSCr (P < 0.0001) in both centres (Table 2). Furthermore, the comparison of the precision of both methods showed significant differences (P < 0.0001, Table 1). The AUC values (area under the curve) of the ROC analysis for cSCr (centre 1: 0.915 (device A), 0.926 (device B) and centre 2: 0.919 [device A), 0.911 (device B)] demonstrated a high accuracy. The inter-device agreement in both study centres was excellent (P < 0.0001, Table 1).

Table 1 Means, standard deviations, calculated difference between the measured serum creatinine level (mg/dL) using StatSensor™ creatinine and the laboratory reference, Pearson's correlation coefficients, paired t-test and Maloney-Rastogi test, subdivided by study centre, pre- and post-offset correction.

			Centre 1	Centre 2	Mean (centre 1 +2)	Pearson correlation coefficient	P value (paired t-test)	P value (Maloney-Rastogi-test)
StatSensor™
cSCr	Pre offset	Device A	0.80 ± 0.34	0.89 ± 0.27	0.85 ± 0.31	Inter-device evaluation (Device A vs B)
		Device B	0.81 ± 0.34	0.89 ± 0.26	0.85 ± 0.30	Center 1 = 0.97	Center 1 = 0.404	Center 1 = 0.480
		Difference between Device A vs B	-0.01 ± 0.09	0.00 ± 0.09		Center 2 = 0.95	Center 2 = 0.689	Center 2 = 0.521
		Mean (Device A + B)	0.80 ± 0.34	0.89 ± 0.26	0.85 ± 0.31
	Post offset	Device A	0.97 ± 0.50
		Device B	0.96 ± 0.50
Laboratory reference
vSCr			0.96 ± 0.42	1.02 ± 0.26	0.99 ± 0.35	Inter-method evaluation (cSCr vs vSCr)
Difference
cSCr - vSCr	Pre offset	Mean (Device A + B)	-0.16 ± 0.16a	-0.14 ± 0.15b	-0.15 ± 0.15c		< 0.0001a,b,c	< 0.0001a,c; 0.9962b
		Device A	-0.16 ± 0.16			0.93	< 0.0001	< 0.0001
		Device B	-0.15 ± 0.17			0.92	< 0.0001	< 0.0001
	Post offset	Device A	0.01 ± 0.19			0.93	0.638	< 0.0001
		Device B	0.00 ± 0.20			0.92	0.955	< 0.0001

cSCr: Capillary serum creatinine; vSCr: Venous serum creatinine.

Figure 1 Scatter plots, regression equations and correlation coefficients for the correlation for capillary point-of-care serum creatinine vs venous point-of-care serum creatinine for both centres (A), for creatinine values below (B) and above (C) the cut-off-level of 1. 2 mg/dL (venous point-of-care serum creatinine). The Bland-Altman plot for the differences between capillary point-of-care serum creatinine (cSCr) and venous point-of-care serum creatinine (vSCr) for both centres (D).

Table 2 Pearson's correlation coefficients for creatinine values measured using the laboratory reference (venous serum creatinine) and StatSensor™ creatinine, subdivided by study centre, renal function and gender.

vSCr (laboratory) vs	cSCr (StatSensor™)		Overall
	Device A	Device B
Study centre
Centre 1	0.93	0.92	0.93
Centre 2	0.85	0.82	0.84
Overall	0.90	0.88	0.89
Renal function
vSCr < 1.2 mg/dL	0.66	0.59	0.63
vSCr > 1.2 mg/dL	0.91	0.90	0.91
Gender
Male	0.91	0.89	0.90
Female	0.81	0.78	0.80

cSCr: Capillary serum creatinine; vSCr: Venous serum creatinine.

Re-assessment of the data from centre 1 revealed the following results. For device A, a slope correction factor of 1.45 and of 1.46 for device B was calculated. The y-axis correction factor was 0.193 (device A) and 0.215 (device B). Consequently, the correlation for vSCr vs cSCr for both devices improved after implementation of the offset correction. The regression graph showed an approximation to the graph with the equation y = x (Figure 2A, B). After applying the offset correction, no change in the correlation coefficient for device A and B (Figure 2A, B) was observed, but a minimisation of the percentage bias between the two methods was achieved (Table 3). The previous significant differences in the paired t-test for the agreement between the two methods were non-significant after the offset correction (Table 4). No significant changes were found for the Pearson correlation coefficient and the Maloney-Rastogi testing. Addition of the offset correction factor to the original data from centre 1 resulted in an improvement in the sensitivity (device A: 35.48% to 80.65%; device B: 41.94% to 70.97%) for detecting patients at risk (> 1.2 mg/dL), whilst maintaining acceptable specificity (device A: 99.41% to 98.26%; device B: 99.41% to 94.12%). Furthermore, the NPV improved (device A: 89.42% to 96.57%; device B: 90.37% to 94.67%).

Figure 2 Scatter plots, regression equations and correlation coefficients for the correlation for capillary point-of-care serum creatinine vs venous point-of-care serum creatinine for centres 1 and device A (A) and device B (B), each before and after implementation of the offset correction. C: The Bland-Altman plot for the differences between capillary point-of-care serum creatinine (cSCr) and venous point-of-care serum creatinine (vSCr) for centre 1 and device A after curve offset correction; D: The Bland-Altman plot for the differences between cSCr vs vSCr for centre 1 and device B after curve offset correction.

Table 3 Results of the re-analysed data from centre 1 (n = 201), shown pre- and post-implementation of the curve offset, separated for the two devicespre- and post-offset correction.

	Device	Slope	Intercept (mg/dL)	r2	Mean difference (mg/dL) (method-difference)	STDV of differences (method-difference)	% bias
Pre offset	A	0.765	0.065	0.870	-0.16	0.16	-15.54
	B	0.745	0.089	0.845	-0.16	0.17	-14.76
Post offset	A	1.110	-0.099	0.870	0.01	0.19	0.39
	B	1.088	-0.084	0.845	0.00	0.20	-0.02

The results are given as slope, intercept, mean difference, standard deviation and percentage bias.

Table 4 Sensitivity, specificity, positive predictive value, negative predictive value and mean values for the cut-off value of 1.2 mg/dL, of the re-analysed data from centre 1 (n = 201), shown pre- and post-implementation of the curve offset, separated for the two devices (%).

	Device	Sensitivity	Specificity	PPV	NPV
Pre offset	A	35.48	99.41	91.67	89.42
	B	41.94	99.41	92.86	90.37
Post offset	A	80.65	98.26	89.29	95.57
	B	70.97	94.12	68.75	94.67
	Mean (A + B)	77.42	94.71	72.37	95.83

PPV: Positive predictive values; NPV: Negative predictive values.

DISCUSSION

Chronic renal disease is a widespread problem, which can lead to potential life-threatening complications (CIN/NSF) after the administration of iodinated contrast agents, e.g., in the framework of CT or cardiology procedures and of GBCA in MRI. Particularly, in emergency and out-patients who often lack available laboratory values and clinical history, an estimation of the glomerular filtration rate from SCr level would be helpful. In such clinical situations requiring rapid decision-making based on renal function, a rapid SCr determination could identify patients at risk and lead to a different treatment procedure. For the prevention of CIN and of NSF, the assessment of renal function is recommended or rather mandatory[3,14,15], due to the limited treatment options. Our study showed the feasibility of creatinine determination using the new handheld device (StatSensor™) in a routine clinical setting in the radiology departments of two large university hospitals. The comparison demonstrated a significant correlation between the capillary measurements of the handheld device StatSensor™ and the venous laboratory reference method. In centre 1, the correlation was slightly higher than in centre 2 (r = 0.93 vs r = 0.84), which can potentially be explained by the two different laboratory autoanalysers and/or by the different handheld devices used. The method correlation in both centres was apparently better for normal renal function (vSCr ≥ 1.2 mg/dL; r = 0.91) compared with impaired renal function (r = 0.63). This inaccuracy for lower SCr values was recently described for the Jaffé method[16]. StatSensor™ yielded significantly lower values throughout and significant differences were found in the agreement between the two methods. Nevertheless, the measurements of the StatSensorc seem to be reliable because the inter-device agreement was excellent. After implementation of the curve offset correction, the data for centre 1 were re-analysed and showed a significantly improved correlation between the two methods. No statistically significant differences with the paired t-test were observed, however, significant differences in the precision (Maloney and Rastogi-test) remained. In addition to minimisation of bias, the NPV and PPV improved and were higher than the laboratory reference. Although maintaining high specificity, the curve offset correction led to high sensitivity and NPV for patients at risk (> 1.2 mg/dL). Due to the higher sensitivity and NPV, the discrimination between normal and impaired renal function improved.

To date, only a small number of studies have evaluated the accuracy of the new handheld device[11,12,17,18], however, similar results have been described, although Aumatell et al[17] and Korpi-Steiner et al[18] used an enzymatic reference method and venous full blood for StatSensor™. To our knowledge, our study is the first to use a colorimetric reference laboratory method. However, negative differences between SCr determinations using StatSensor™ compared with the enzymatic laboratory reference were also described in our study. Schnabl et al[11] also observed a good correlation for whole blood creatinine compared with laboratory plasma measurements (r² = 0.933), but described a negative proportional bias. In an in vitro study, high levels of creatine and urea falsely elevated creatinine measurement. The evaluation of StatSensor™ by Shephard et al[12] compared with an enzymatic laboratory reference in 100 patients confirmed our results of a statistically significant underestimation of SCr. These authors also used an alignment with the laboratory results. They found that for eGFRs above or below 60 mL/min, 100% and 87% of the results, respectively, agreed with the laboratory eGFR (79% and 96% post-alignment). Similar to our results, Shephard et al[12] concluded that the handheld device will identify most patients with an eGFR < 60 mL/min, but there will be many false undereGFR results that require laboratory validation.

Potential limitations of this study were, as mentioned above, the different autoanalysers used in the two study centres, which may partially explain the different results. A further problem was the lack of standardisation between the methods, which hampered comparability. The general limitations of SCr evaluation (e.g., late rise from about 50% renal function) are widely known and are not discussed here. The standardisation could be improved by calculating the GFR using the MDRD formula[10]. The differences between the laboratory reference method and the handheld device could be based on the known bias of the colorimetric Jaffé method vs enzymatic methods, which result in systematically higher creatinine values due to non-creatinine chromogens[19,20]. However, corresponding to this bias it seems to be mandatory to analyse potential interfering substances using the StatSensor™ in a larger patient population. Furthermore, to achieve a higher validity for the decision-making cut-off range of creatinine values, a larger collective should be included in a further study. In our study, the number of patients at risk with elevated creatinine values was relatively small for adequate evaluation. Finally, the curve offset correction significantly improved the results, but did not actually represent the real performance of the StatSensor™ handheld device.

In conclusion the StatSensor™ seems to be a rapid, cost-effective method for the determination of SCr with a high sensitivity and NPV. This could contribute to improving the workflow in a radiology department, especially in patients with an unknown history of renal diseases. The high sensitivity and NPV predict the differentiation between normal vs impaired renal function. However, without offset correction to the local specific laboratory, StatSensor™ may significantly underestimate SCr and should be corrected technically.

COMMENTS

Background

Contrast-medium-induced nephropathy (CIN) and nephrogenic systemic fibrosis (NSF) are two potentially life-threatening complications of iodinated contrast agents and gadolinium-based contrast agents (GBCA). According to increased requests over the last few years, the demand for point-of-care serum creatinine (SCr) determination in radiology departments has increased.

Research frontiers

Before implementing a decision-making, commercially available handheld device (StatSensor™) into clinical practice, the feasibility and accuracy of the device need to be tested in a large collective in the daily routine of a radiology department. This study showed the feasibility of serum creatinine (SCr) determination using this handheld device in a routine clinical setting. The device showed high sensitivity and high negative predictive value, but may significantly underestimate SCr without offset correction to local laboratories.

Innovations and breakthroughs

Recent reports have highlighted the importance of possible point-of-care determination of SCr in radiology departments. To the knowledge, the study is the first to compare StatSensor™ with a colorimetric reference laboratory method. The method correlation was apparently better for normal renal function than for impaired renal function. StatSensor™ yielded significantly lower values throughout and significant differences were found in the agreement between the methods. Nevertheless, the measurements using StatSensor™ seem to be reliable as the inter-device agreement was excellent. After implementation of the curve offset correction, the data showed a significant improvement in the correlation between the two methods.

Applications

By knowing the strengths and the potential pitfalls in the determination of SCr using the handheld device (StatSensor™), this may represent a future strategy for point-of-care testing of patients undergoing contrast-enhanced computed tomography (CT) or magnetic resonance imaging (MRI).

Terminology

CIN and CT are two complications of iodinated contrast agents and GBCA (MRI). Both are associated with impaired kidney function, and therefore the a priori determination of SCr is crucial in patients at risk.

Peer review

This is an interesting and critical report. It is a well written scientific paper.