Radiological clinical trials: Proposal of a problem-finding questionnaire to improve study success

Abstract

AIM

To develop a survey to help define the main problems in radiological clinical trials.

METHODS

Since 2006, we have managed seven different radiological clinical trials recruiting patients in academic and non-academic centres. We developed a preliminary questionnaire using a four-round Delphi approach to identify problems occurring in radiological clinical trials run at our centre. We investigated the recruitment experience, involvement of all multi-disciplinary team members and main obstacles to completing the projects. A final round of Delphi processes elucidated solutions to the identified problems.

RESULTS

Among 19/20 (95%) respondents, 10 (53%) were young physicians (under 35 years old), and the respondents included non-faculty members, fellows, residents, and undergraduate students. Ninety-four percent (18/19) of respondents showed interest in conducting clinical trials. On a scale of 1 to 10, the problems with higher/worse scores (8-9) were related to technical or communication problems. The most frequent problems across all studies were technical problems related to clinical trial equipment, insufficient willingness to participate, obstacles to understanding the design of electronic-case report form and extra work.

CONCLUSION

The developed questionnaire identified the main recurring problems in radiological clinical trials as perceived by end-users and helped define possible solutions that are mostly related to having dedicated clinical trial research staff.

Key Words: Clinical trials, Data management, Magnetic resonance imaging, Mammography, Ultrasonography

Core tip: Clinical data management (CDM) is important for efficiently managing and completing a clinical trial. CDM is the process of controlling, processing, validating and querying data generated in a clinical study. In this paper, we developed a questionnaire identifying the main recurring obstacles in radiological clinical trials as perceived by end-users. We tried to define possible solutions that are mostly related to having dedicated clinical trial research staff. This topic is relatively well-known by clinicians, while it is less well-known by radiologists and could be useful for radiological centres that are currently involved or will be involved in conducting or participating in radiological clinical trials. For this reason, we suggested a problem-solving questionnaire and reported our experience in managing seven multi-centre national and international radiological clinical trials.

INTRODUCTION

Clinical data management (CDM) is important for efficiently managing and completing a clinical trial. CDM is the process of controlling, processing, validating and querying data generated in a clinical study. Recommendations indicate that a specialized research unit may be useful for conducting clinical trials[1-3]. As indicated by Farrell et al[3], the success of a clinical trial depends on the presence of an efficient trial team consisting of various experts with different roles and responsibilities. In addition, it is important to have the resources to manage the study workflow stages, as defined by the coordinating centre. The presence of a dedicated trial manager is also important to collecting high-quality clinical data in healthcare studies. Indeed, the collection of poor quality data or the collection of a lower level of data than expected may contribute to underpowered, inconclusive or misleading results.

A good study design and efficient CDM Plan (CDMP) are important for taking full advantage of research project budgets, especially in multi-centre and international collaborative trials. The essential components of a CDMP include the following: Details of study personnel involved in the study and data access roles assigned to each, database design and database location, data entry procedure, methods of data collection - paper or electronic-case report form (e-CRF), data preparation before entry into the electronic system, and data flow and tracking to ensure optimal data completion and facilitate reporting.

The efficiency of the CDMP is crucial to optimizing patient recruitment and follow-up, increasing the percentage of completed e-CRFs, and using processes ensuring that high-quality data are collected with minimal or no missing data. As recently reported[4-6], investigators conducting randomized controlled trials (RCTs) use different strategies to avoid biases in data collection. However, many trials do not recruit sufficient participants, limiting the use of research results and translation of research findings into practice[7-9]. Additionally, an audit is necessary to regularly monitor the randomization process[10,11]. Standardized procedures are necessary to handle errors or problems in the randomization process and data acquisition, which is crucial to the overall trial quality.

The medical literature lacks a structured description of the main problems affecting clinical trials that specifically deal with imaging and are led by a radiological unit[12]. Imaging in research is increasingly involved. The use of imaging data in clinical research can provide many scientific benefits, but it can result in additional complexities that contribute to risks, biases and errors[13]. As indicated by Erickson et al[14], the use of imaging data in clinical trials may be a part of the solution for reducing the cost and increasing the efficiency to conduct a timely clinical trial. A frequent problem with a radiological clinical trial consists of the quality of the clinical trial data; multi-centre clinical trials need reproducible, quality assured data with post-processing methods supported by an operational infrastructure.

In the hospital, the medical subject’s imaging data are managed in the clinical picture archiving and communication system (PACS) via the digital imaging and communications in medicine (DICOM) protocol. Clinical PACS could be separate from the research PACS.

PACS is extremely limited in its support for research imaging. The system is DICO-centric and generally does not support the alternative file formats used in research. It is essential to guarantee the high quality of the entire process that images for clinical trials are collected using uniform image acquisition and measurement methods to minimize the variability.

To address this knowledge gap, we performed a survey-based study to identify the main problems in conducting radiological clinical trials and to help find solutions, including roles for staff dedicated to ongoing radiological clinical trials. The aim of this study was to identify potential barriers to conducting clinical trials in imaging.

This work is a pilot study. The survey was performed as an internal questionnaire survey at our centre, which is involved in several multi-centre clinical trials, and the preliminary results could help all centres involved in radiological clinical trials find solutions to the main problems and improve the progress and outcomes of future radiological clinical trials.

MATERIALS AND METHODS

Clinical trials

Data for this study were derived from staff involved in seven different radiological national and international multi-centre clinical trials employing cancer imaging. The clinical trials are listed as indicated in the Supplementary Information. The first study was performed in 2006 and the most recent in 2015[15-18].

All studies included in this work were already approved by the respective Ethical Committee and all participants signed a written informed consent form before enrolment. The studies were performed according to the principles outlined in the Declaration of Helsinki.

The studies codified as ASTOUND[15], Tomo-micro[16], BP-US[17] and BP-MRI[18] in the Supplementary Information were already published.

Development of the survey

The survey was developed using a 4-step consensus approach by the Delphi method[19,20]. The personnel of the University Hospital and all teams that participated in the seven radiological trials were invited to respond to the survey and participate in the Delphi method. The Delphi method is based on the premise that collective beliefs are more trustworthy than the beliefs of a single person; therefore, it is considered an efficient procedure to generate thematic knowledge[20]. By this method, opinions, expertise and critical thinking are systematized. Individual feedback on a topic, the judgment of the group’s work, and opportunities to change opinion were given in an anonymous form[19]. The questionnaire focused on the key issues identified by the personnel directly involved in the trials to reduce the influence of department chairs.

The first step consisted of a review of the existing literature up to July 2015 and the development of the first draft of the survey. The subsequent three steps each included a Delphi round to develop the final survey. A series of discussions (face-to-face meeting and e-mails) among the participants was performed. The survey investigated several stages of clinical trials, including the recruitment experience, effective involvement of all multi-disciplinary teams (MDTs), the main obstacles faced in clinical trials, and the background of each team member. After the survey, critical issues were identified and summarized; then, possible solutions were suggested by the same Delphi method.

The questionnaire consisted of 12 items that were written in English, as indicated in Supplementary Figure 1.

We have classified each issue of the survey given to the participants with a score of 0 to 10 (1 = no problems observed, 10 = several problems can negatively affect the results and induce the participants to quit). The characteristics are listed in Table 1.

Table 1 List of main issues and problems identified when conducting clinical trials¹.

Problem	Score (mean ± SD) among respondents	Effect on clinical trial conduction	Suggested solution	No. of surveys scored from 19 completed surveys
Principal investigator	9 ± 0.5	Lack of team consistency and participation	The principal investigator should be PERSONALLY involved and have a pro-active approach to the study	15/19
Administrative impediments (ethics committee, insurance) affect the beginning of clinical trials	6 ± 0.37	Delay in starting the study	Employ a coordinator from administrative staff with no clinical burden	13/19
Technical problems with instruments used in the study	6 ± 0.62	Delay in conducting the study	Identify a key person to regularly check instrumentation	12/19
Insufficient willingness to be part of a team and to collaborate in the trial	7 ± 0.41	Lack of interest and enthusiasm and inability to progress or finish in time	Organise frequent investigator meetings, conference calls and study checks	15/19
Slightly different clinical practices of the involved centres	7 ± 0.42	Risk of missing or non-standardized data	Discuss and standardize practical, methodological data-related aspects of the study	14/19
Difficulties to complete a complex e-CRF	7 ± 0.46	Incomplete e-CFR and missing data	Simplify the e-CRF	17/19
Perform quantitative evaluations	8 ± 0.38	Delay in quantitative radiological data acquisition	Have dedicated trained personnel and workstations	18/19
Extra work required to comply with study inclusion criteria	9 ± 0.32	Loss of patients potentially eligible for the study	Check inclusion criteria in advance by available patient data review	18/19

¹The score system ranges from 1 (no problem) to 10 (serious problem). e-CRF: Electronic-case report form.

Survey participants

The survey involved investigators who were participants belonging to the MDT, including personnel of the University Hospital and of all teams who took part in the seven radiological trials as described above. They were asked to complete the questionnaire, highlighting the main problems faced during clinical trials.

The survey was sent to all clinical team members, including the principal investigators (PIs), research nurses, nursing staff, and technicians. The anonymous questionnaire had to be returned to the identified PI’s delegate to record the responses, as normally done in a Delphi process. We performed further rounds of Delphi processes to solve all encountered difficulties.

Statistical analysis

The mean experience of team members in radiological clinical trials as well as the percentage of questionnaires returned was recorded. Group agreement with the clinical condition under consideration was defined as total cumulative agreement > 67% after the second or third Delphi round. Group consensus was defined if the consensus level of agreement (CLA) was > 90% for each issue of the survey. The results are presented as the total cumulative agreement after the last Delphi round by a four-point simplified Likert scale (agree, agree with minor reservation, agree with major reservation, and disagree).

RESULTS

Characteristics of survey participants

Nineteen of 20 team members (95%) returned the questionnaires. Ten of nineteen of survey participants were young physicians and non-faculty members (fellows, resident and undergraduate students). The other members (9/19) were staff-doctors, principal investigators, and co-investigators. Additionally, 18/19 of respondents showed interest in conducting clinical trials. Among these, a large proportion of physicians with previous clinical trial experience (14/18) and many residents, data managers, and nurses without clinical trial experience expressed high interest in conducting clinical trials. Only one participant was involved in a clinical trial that had terminated before the completion of the present survey.

Main problems encountered

The main barriers faced in conducting a radiological clinical trial (with a score of 8-9) were the time commitment to perform quantitative evaluations of radiological exams that are already reported and the extra work required to comply with the clinical trial’s inclusion criteria. A score of 6, reflecting a significant but not severe problem, was the need to deal with administrative impediments, such as the need to prepare all the documents for the local ethics committee and insurances for research studies. Indeed, these problems can delay the beginning of the radiological clinical trial. A low score of 6 was also due to a technical problem with the instruments (for example, new software applications) needed in a study and the lack of organized support from the hospital facilities. A score of 7 indicated a possible lack of interest to conduct the clinical trial and several difficulties to complete the e-CRF. From participating in multinational clinical trials, 15/19 of respondents assigned a score of 8 or 9 for the PI, indicating that the role of the PI is crucial to conducting a radiological clinical trial. After problem identification, possible solutions suggested from the final Delphi round are reported in Table 1.

DISCUSSION

Clinical trials have rapidly evolved during the past decade. As we discussed above, radiological clinical trials can be affected by different types of bias concerning imaging technology and recruitment strategies. Bias can result from differences in the methods in which information is collected or in the manner in which data are obtained during the recruitment process. In the past, radiologists have had limited direct patient interaction and have depended on other specialists to refer patients for enrolment; in this way, inadequate approaches to patient recruitment could introduce bias. The main strategies for recruitment were flyer distribution, brochure pick-up, internet posting-ads or poster distributions without direct patient contact. Current technology has allowed us to take a different approach, directly interact with the patients, and monitor the follow-up or response.

In this work, we developed a preliminary survey to elucidate knowledge on obstacles or problems in running radiological clinical trials from all participating in various radiological studies at our centre, and we hoped that the acquired information could improve the conduct of radiological clinical trials. We observed that several obstacles (related to administrative, technical/equipment, or resourcing issues) could hamper the development of relatively feasible radiological clinical trials. Using the same survey-based/Delphi process, we also sought to define possible solutions to the main problems that had to be overcome during several radiological clinical trials.

We tried to differentiate serious problems from less serious or minor problems. It is not surprising that the majority of problems that received a high score were related to the lack of resourcing and, specifically, to the lack of dedicated research personnel without a clinical burden. Indeed, busy daily radiological clinical practices have limited time for the additional work generated by conducting or contributing to a clinical trial. In our survey, the highest scores (“bigger problems”) were assigned to issues that typically go well beyond the radiological report, such as performing a quantitative evaluation on radiological images as part of the research protocol, or becoming familiar, and complying with the study inclusion criteria (patient eligibility). Indeed, for prospective trials, respecting the inclusion criteria of the study is crucial for several reasons, such as reaching the required number of patients and collecting reliable and unbiased data. Consequently, the suggestion given by the last Delphi round was to have dedicated clinical trial personnel who are not involved in the clinical routine undertake the role of checking and ensuring compliance with the inclusion criteria.

In larger multicentre studies requiring that data are gathered from different centres, even minor differences in the population, culture, nomenclature and medical practice can be causes of variability. As indicated by Willis-Shattuck et al[21] in a systematic review facing the issues related to recruitment in developing countries, the authors reviewed all studies investigating the link between motivation and retention of health workers in developing countries. The authors concluded that motivational factors are influenced by the context, and the successful completion of a study depends on the number of available resources.

It is commonly thought that the public health sectors of many countries suffer from a surplus of workers who are not particularly productive because they have not received adequate training. In fact, a survey presented at RSNA 2013 by Rehani et al[22] confirms that radiologists in developing countries need an accurate training program.

Indeed, in a single country, multi-centre trial, there can be variability and bias, but some of the possible sources of bias can be controlled with an appropriate trial design. An important result of our survey is that standardized trial planning and the identification of a key figure managing several phases of a radiological clinical trial is very important for ensuring a timely start and correct development of the trial.

Through our experience of being involved in seven different studies on cancer imaging and collaborating with several research groups from different contexts, we investigated how many problems can arise when developing clinical studies. Unfortunately, we did not evaluate the hospital due to a lack of funding.

We found it very useful to monitor monthly enrolment progress by site and permit sites to compare and discuss their progress. We organized collaborative workshops with all investigators from the included studies for all periods of the studies. These meetings were valuable to discuss practical, methodological and data-related aspects of each original study and to build trust among investigators. During these workshops, we discussed and refined the study protocol in advance, examined patient characteristics and information from diagnostic tests that are to be analysed, and agreed on data checking procedures and the main analyses to be performed.

In conclusion, this study could be a valuable preliminary survey that can elucidate the critical key points identified in radiological clinical trials. Obviously, this study does not solve all problems that a radiologist could face during a clinical trial. However, the main problems in oncology clinical trials or in imaging are not very different, and they are in common with what has previously been described as essential to successfully concluding a clinical trial. It is important to identify the crucial role of key people who are capable of connecting different expertise levels and responsibilities. Indeed, each person involved in conducting a trial should be instructed and qualified to tailor his or her respective task(s), taking advantage of previous cultural backgrounds. Our problem-solving approach may improve the organization of radiological clinical trials, especially in non-academic centres.

ACKNOWLEDGMENTS

The authors are very grateful to all our dedicated study staff (technicians and research personnel) for their enthusiasm, recruitment, assistance and retention efforts.

COMMENTS

Background

Clinical data management (CDM) is the process of controlling, processing, validating and querying data generated in a clinical study. Recommendations indicate that a specialized research unit may be useful for conducting clinical trials. The presence of a dedicated trial manager is important for collecting high-quality clinical data in healthcare studies. Indeed, the collection of poor quality data or of a lower level of data than expected may result in underpowered, inconclusive or misleading results. The essential components of a CDM plan (CDMP) include: Details of study personnel involved in the study and data access roles assigned to each, database design and database location, data entry procedures, methods of data collection - paper or e-CRF, data preparation before entry onto electronic system, and data flow and tracking to ensure optimal data completion and facilitate reporting.

Research frontiers

The efficiency of the CDMP is crucial to optimizing patient recruitment and follow-up, increasing the percentage of completed electronic-case report forms, and using processes ensuring that high-quality data are collected with minimal or no missing data. Usually, investigators conducting randomized controlled trials employ different strategies to avoid biases in data collection. Standardized procedures are necessary to handle errors or problems in the randomization process and data acquisition, which is crucial to the overall quality of the trial.

Innovations and breakthroughs

A good study design and an efficient CDMP are important for taking full advantage of research project budgets, especially in multi-centre and international collaborative trials. The information from this study might allow all centres involved in radiological clinical trials to find solutions to the main problems as well as help improve the progress and outcomes of future radiological clinical trials.

Applications

The medical literature lacks a structured description of the main problems that affect clinical trials specifically dealing with imaging in a radiological unit. The aim of this study was to identify potential barriers to conducting clinical trials in imaging.

Peer-review

The authors investigated problems faced when conducting clinical trials. This work clarified the issues for improving the efficiency of clinical research.