Technological developments enable measuring and using patient-reported outcomes data in orthopaedic clinical practice

Abstract

Patient-reported outcomes measures form the backbone of outcomes evaluation in orthopaedics, with most of the literature now relying on these scoring tools to measure change in patient health status. This patient-reported information is increasingly collected routinely by orthopaedic providers but use of the data is typically restricted to academic research. Developments in electronic data capture and the outcome tools themselves now allow use of this data as part of the clinical consultation. This review evaluates the role of patient reported outcomes data as a tool to enhance daily orthopaedic clinical practice, and documents how develop-ments in electronic outcome measures, computer-adaptive questionnaire design and instant graphical display of questionnaire can facilitate enhanced patient-clinician shared decision making.

Key Words: Patient reported outcome measures, Electronic patient-reported outcome, Computer adaptive design, Outcomes evaluation

Core Tip: Utilising modern information technology, data collection, processing and intuitive graphical data display in real-time, electronic patient-reported outcome assessment can be implemented in daily clinical practice.

INTRODUCTION

Patient-reported outcome measures (PROMs) are widely used in orthopaedic research as they provide important and detailed information on patients' perception of symptoms and function in everyday life[1]. These metrics are central to evaluating success in orthopaedics as factors such as pain and satisfaction are only accessible by self-report. Most orthopaedic units collect these to some degree and clinical staff will be familiar with these scores from the literature if not their personal practice. In the main, outcomes are assessed via pen-and-paper questionnaires, but electronic PRO questionnaire administration is increasingly employed in clinical studies and more recently also in daily clinical practice[2].

PROMs allow for objective measurement of the patient’s subjective view of their health status[3] which complement other clinical or image-based evaluations. In clinical practice, numerous benefits have been cited for using PROM data, including enhanced screening, diagnosis, and longitudinal monitoring of conditions, along with the promotion of patient centred care[4-6]. Greenhalgh et al[7] suggest that collecting and using PROM data, along with clear feedback and dissemination of this information, will stimulate and incentivise health professionals and ultimately healthcare providers to provide better care[7]. The challenge though is how to take the mean outcome score from a patient questionnaire and modify care delivery.

Today’s patients certainly want greater involvement in decisions regarding their care[8]. Shared decision-making i.e. the conversation that happens between a patient and health professional to reach a healthcare choice together, has been advocated and been embedded in clinical practice for some time. Patients should always make the ultimate decision - especially in elective surgery - about their care but can only do so if they are fully informed about the options and are encouraged to participate[9]. In orthopaedics, surveys suggest that surgeons are strong advocates of shared decision-making, but also that they have concerns regarding logistics, practicality and in particular burden to the clinical workflow relating to engaging the patient and providing in-depth personalised feedback and discussion[10]. Being able to interpret how well an individual is performing relative to the wider knee replacement population can form a meaningful part of the consultation process, and potentially influence the overall experience of the surgical pathway[11].

Use of patient reported outcomes data as part of clinical care

To date, the integration of PROM data in routine orthopaedic care is less evolved than in other specialties such as oncology. Here several clinical studies have emphasized the positive effects of regular collection and use of this data in patient management[12-14]. Regular collection of health data has demonstrated a positive impact on symptom control, overall quality of life and emotional well-being without burdening clinical management. Physicians report that over time the clinical use of PROM information improved their ability to recognize symptoms and health issues and enabled crucial conversation with the patient[15-17]. These positive effects should be similarly replicable in the orthopaedic field.

The routine collection of PRO data in daily practice can provide valuable information for research purposes. The collection of pre- and post-operation PRO questionnaire data is routine for many orthopaedic interventions in many healthcare systems. In fact, combined PRO assessments of general health and joint-specific outcomes have been integrated with national registries in the United Kingdom, Sweden, and the Netherlands and numerous local or regional registries[18]. This data that evaluates all patients at scale, can add valuable real-world context to help interpret the results of tightly controlled studies that may, for example, not encompass the typical case-mix presentation in routine practice. However, these routinely collected registry data are typically not directly available for the clinical teams, but only used for later analysis for research purposes or quality assurance. Thus, analyses of collected PROMs is generally at the population level and published primarily for the benefit of academic readers. Only summed information is occasionally fed back into clinical practice at the clinical governance level in the Figure 1. Therefore, the time and resources spent to collect PROM data do not help inform the individual patient or clinician and the full potential of these data is poorly exploited. Clinical assessment is separately based on the clinician’s individual discussion with the patient as to their symptomology and medical history and does not incorporate the standardized information provided by patients via PROMs. The latter is particularly useful for the longitudinal evaluation of symptoms and functional levels, as patients may have difficulties to provide valid information on the degree of deterioration or improve-ment of symptoms over time and clinical documentation may capture such change insufficiently; depending on the individual viewpoints, expectations and experience of the treating clinician. Alongside a physical analysis, imaging and potentially lab tests, PRO data may therefore have a key role in driving clinical management.

Figure 1 Current usage of patient information data flows.

PROM data can be used to promote patient centred care[19] and to facilitate patient and clinician’s understanding of how different treatments affect patient functioning and wellbeing over time, informing treatment decision making and, most importantly, improving expectation management[20]. Feeding back PRO data at the level of the individual patient during the consultation combines these two, usually separate streams of information in the Figure 2.

Figure 2 Potential usage of patient information data flows.

We are increasingly using prediction models for estimating eventual post-operative scores as part of pre-operative surgical expectation management. Unlike paper-pencil assessments, electronic data capture provides immediate availability of all collected data in a database that may be used for establishing and constant updating of regression models that allow for a prognosis of treatment outcome for individual patients based on baseline PRO assessments and patient characteristics, such as age, body mass index, or comorbidities[21,22]. Such models based on the data collected at an individual centre or taken from the literature may be helpful when dealing with interpreting individual PROM scores. Prediction models based on the routine data of a specific centre, may be more precise for predicting scores of new patients at that centre than models from the literature, as in such a scenario the sample on which the model is based is more similar to the patients for which predictions are desired.

Contextualising PROM data for the patient in terms of that individual’s position within the wider expected levels for people of their specific demographics, at a particular point in time, makes the PROM data relevant to the presenting complaint and planned management discussion. It facilitates the patient-clinician discussion of symptomology, expected trajectories and clinical management options; directly feeding into clinical management and outcomes. The use of these reports during the consultation can help to identify important health issues that might otherwise not be disclosed during the consultation[12]. Engaging the patient has the further benefit of enhancing inclusion and shared decision making contributes to patient satisfaction with clinical services.

While incorporating patient outcome scores in this manner as part of the clinical consultation has been an attractive idea for quite some time, it has only recently become a feasible proposition with a transformative step change in the way we can collect and use information technology in the clinic.

Electronic proms

As noted, PROMs are traditionally pen and paper-based, collected en-masse and manually uploaded for processing at the national, institutional or project level. Logistics can be challenging, and time consuming, often delayed, and manual data entry opens further issues with imputation errors.

Electronic PROMs (ePROMs) collected via a tablet computer connected to a local network to collate, process and store the data, clearly, offers several advantages; electronic administration of outcome questionnaires using tablet PCs or smartphones improved feasibility of data collection, decreased responder burden and sophisticated survey administration with real-time processing[23-25]. The International Society for Pharmacoeconomics and Outcomes Research PRO Mixed Modes Task Force[26] highlights additional advantages of using electronic data collection including avoidance of secondary data entry errors, date and time stamping, edit checks, and more accurate and complete data. Further, electronic systems allow remote questionnaire completion (e.g., at home), and automatic messages that remind patients of due assessments[27].

But most importantly, ePROM collection is necessary for using these data in the clinical consultation with the individual patient, as data collected on paper can hardly be made available timely in an accessible format without electronic means. In addition, regular PRO monitoring from home allows patients to observe their recovery process independently, to assess how they compare with a similar sample population and to remain in constant contact with surgical and therapy centres between outpatient appointments via increasingly available institutional web portals. This supports self-management by helping patients to better assess and contextualise their state of health/recovery. In a next step, if appropriate, patients can be invited to attend for radiographs or clinical review based on changes in their remote data presentation.

The successful use of PROMs in routine care in this way clearly requires careful planning and adequate logistics[28-30]. Implementation is supported by international guidelines such as the “User’s Guide for Implementing Patient-Reported Outcomes Assessment in Clinical Practice” published by the International Society for Quality of Life Research[31]. In detail, this process includes tasks such as: development of training and information materials regarding the use of PRO data during the clinical encounter, identifying the appropriate PRO measure that allows valid and reliable assessment of the relevant health issues, definition of time points during the treatment and disease trajectory when assessments should take place, and establishing pathways for linking clinically relevant scores to interventions and referrals.

Computer-adaptive measures and ‘intelligent’ questionnaires

The technological move to electronic data collection also facilitates a wider change in the use of PROMs, offering enhanced “intelligent” questionnaires. Traditionally, patient reported outcomes are “static” questionnaires, where all patients complete a defined set of the same fixed questions[32]. The questionnaire tools themselves can be lengthy, as many questions are required to allow accurate measurement[33]. For example, the well-used Knee injury and Osteoarthritis Outcome Score comprises 42 questions[34]. Lengthy questionnaires can cause high drop-off rates, leading in turn to fewer complete PROMs[35,36]. This static “one-size fits all” structure is suboptimal for multiple reasons. The questionnaire length can pose a burden to the patient as these measures require a considerable number of questions to cover the whole measurement range of the outcome parameter of interest (pain, functional limitation etc.). Following this, patients often find themselves confronted with questions that are repetitive or that are not appropriate to their current condition (e.g., inquiring as to sporting activities they could clearly not partake in). Such inappropriate questions can be irritating to complete and impair compliance. Frustratingly, these “inappropriate” questions provide no or little additional information to the clinician nor the researcher interpreting the questionnaire (e.g., if a patient reports barely being able to walk, further questions on various sports activities provide little or no further information).

A major step forwards in addressing these issues is the development of computer-adaptive test (CAT) measures[37-39]. Computer-adaptive testing uses large sets (or item banks) of potential questions that cover the spectrum of the issue in question and an algorithm for tailoring individual sets of questions. Based on the response to the initial item the CAT algorithm calculates a first estimate and confidence interval of the outcome score and selects the next most appropriate item (that offers the highest item information in the range of that estimated score) to be administered to specific the patient. This procedure continues until a predefined measurement precision has been reached or a maximum number of items have been asked. In this way, patients only need to answer a fraction of the possible questions – and not necessarily the same questions that others will complete. Thanks to the underlying mathematical model, the scores calculated by different questions within the same item bank are perfectly comparable.

The underlying probabilistic model of a CAT tool is based on item response theory (IRT). An essential prerequisite for running these models is that the outcome questionnaire is based on a strictly unidimensional item sets (i.e. questions that measure the same thing; be that pain or function or general health)[40]. IRT frameworks have been developed in various fields such as cancer and respiratory medicine[41-44]. Since internal consistency (unidimensionality) of orthopaedic questionnaires tends to be high they naturally lend themselves to IRT model application and CAT designs[45]. Although still a relatively new field in orthopaedics, a few studies have applied IRT modelling approaches to orthopaedic outcome measures for functional status, pain, and rehabilitation outcome[46-51] including studies on various aspects of CAT measures[32,52,53].

The largest initiative on the development of item banks for health outcomes measurement is the United States-led Patient-reported Outcome Measure Information System (PROMIS) group. PROMIS has released a substantial number of item banks, including an item bank for the assessment of physical functioning in all types of patient groups[54-56]. The PROMIS measures have been tested and validated in large reference populations, making them suitable for research on different health conditions[57]. For orthopaedic outcome research PROMIS provides useful measures for physical function (including versions for upper extremity, mobility, and mobility aid users)[54,58]. To date there is no joint-specific PROMIS tool available. The computer-adaptive pilot version of the FJS-12[32] for hip and knee assessment is the first computer-adaptive joint-specific measure for orthopaedics. However, because of its limited item bank, this pilot version represents primarily a proof of principle rather than an elaborate CAT instrument with an extensive item bank.

Clearly, CAT based outcome measures require infrastructure; hardware such as a computer tablet interface and a software package to run the questionnaires electronically, to manage item banks and to employ the CAT algorithm for item administration. As IT infrastructure for electronic data capture is increasingly available in hospital settings, the use of CAT measures becomes more feasible and efficient[59].

Compared to static versions, CAT questionnaires take a fraction of the time to complete and, because of the large item banks in the background, can be even more reliable, valid and sensitive to change compared to their paper-based counterparts. Advantages of the CAT approach include instant score calculation and no missing data. The beneficial use of this technology is perhaps most readily seen in clinical trials, where efficiency is enhanced by the direct entry of data by the patient, reducing transcription error rates in processing but most importantly in the enhanced accuracy in terms of confidence intervals facilitating reduction in required sample sizes and costs[60,61].

Scoring, interpretation, and feedback

For PROMs to contribute meaningfully to clinical care, patients and clinicians must be able to readily understand and interpret these data. Unfortunately, interpretation of the results presented by these instruments is not always easy, especially when different instruments are used to measure similar things, and each instrument is scored and scaled differently. For example, the score range of the Oxford Hip Score[62] ranges from 0 to 48 points, whereas the FJS-12[63] is reported on a 0-100-point scale. Scale direction is not consistent; some outcome tools report higher scores as “better” health whereas others indicate problems. The Western Ontario MacMaster score[64] is an example of a tool that is inversely scored, as it is a measure of symptom burden. This can make it difficult to compare information across questionnaires in daily practice. Both patients and clinicians report that variation in PROM scoring, scaling and presentation poses obstacles to interpretation and application[4].

Questionnaire scores taken at isolated time points are somewhat abstract and additional information is required from the literature or from manuals to interpret these. Various approaches have been taken to solve this; wider comparison can be straightforward by simply contrasting the individual score to the population mean or converting the score to percentiles. A further method is to convert scores to a standardized T-score metric with a fixed mean of 50 points and a standard deviation of 10 points. The T-score of an individual patient informs directly to what degree (measured in standard deviation units) the patient deviates from the mean of the reference population[11].

With today's widespread electronic data collection and processing, this data can be visualized immediately after the questionnaire has been completed. Using electronic data capture and display, patients can be presented with a normalised graph (targeted for their specific demographics) showing the expected change and standard deviations around the mean change following surgical interventions such as knee arthroplasty.

PROs are usually assessed repeatedly, prior to and following intervention, across multiple outpatient visits. As such it makes sense to present results longitudinally. This allows a quick overview as to the course of symptoms or functional impairments over time. Few studies have evaluated how to present outcome score results graphically[65,66]. These have shown that longitudinal line charts may be optimal for presenting individual patient data and that it is important to clearly indicate the scale direction (i.e., if high scores indicate poor or good health). Using color-coding or specific percentiles from reference populations can be integrated into charts to guide interpretation[67,68].

The transformation to T-values enables both patients and clinicians to understand the individual score results more easily, as results can be presented in the context of the results reported by others of a similar demographic and intervention. This feedback somewhat akin to the population height and weight growth charts that are routinely used in primary care, allows the patient to see what change would be expected for them in terms of outcomes that matter to them in Figure 3, and monitor their progress against their direct peers. Presenting the typical pre-operative scores also highlights the relative improvement that can be made from the patient’s individual starting point. This data should be captured sequentially, as it can be difficult to accurately deduce symptom burden retrospectively from memory and non-standardised clinic notes.

Figure 3 OKS T-scores feedback model stratified to patient demographics . Figure showing the longitudinal course of expected patient outcomes. The exemplary course (green line) of an individual patient undergoing total knee arthroplasty at 3 different points prior to and following surgery is presented in terms of t-scores against a background of an age and gender matched population. The dark grey area is the range for +/- 1 standard deviation of the appropriate matched population and the light grey area the range for +/- 2 standard deviation.

Challenges in moving to ePROMs

As with all new developments there is a reluctance to embrace new technology[3] and there are real challenges to address. There are issues to consider in terms of data security and privacy[23], as well as the feasibility of developing the local infrastructure required[26]. With regard to software there are essentially two possibilities, the necessary features can either be provided by an extension of the clinical information system, or stand-alone software developed specifically for the purpose of routine PRO monitoring may be used. Whereas the first option has the advantage of easier integration of PRO results into medical charts and linkage of clinical and PRO data, the latter option usually has more refined features for facilitating questionnaire administration and a more sophisticated presentation of results. Currently, it is also unclear whether the computer platforms and servers required increase the provider costs compared to employing a pen-and-paper model with manual transcription and upload. This may also depend on the individual institutional situation.

Electronic questionnaire administration requires the patient to have a basic level computer literacy. However, the last decade has seen more and more elderly people engaging with technology such as using smartphones and social media on a day-to-day basis. It has been suggested that vulnerable patients and elderly groups may struggle to fully engage with this technology. However, these patients currently engage in PROM surveys and in principle should be perfectly able to contribute data[69]. While the orthopaedic literature on implementation studies is still scarce, there are encouraging results, such as the findings by Slover et al[70] who reported question-naire completion rates of about 95% in osteoarthritis patients using a web portal. Further optimization of the graphical user interfaces may be required to promote wide uptake and to promote efficient usage of electronic questionnaires as in the early days of smartphones with full-field touchscreens. In practice, a person well-integrated within the clinic team could be designated as the local ePROM facilitator for introducing patients to the electronic assessments, responding to questions, and discussing patient concerns[71].

When migrating a PROM to an electronic format, great care needs to be taken that standardization of the questionnaire is maintained and no major changes in wording and formatting are introduced. Guidance on such migration and the assessment of equivalence of paper and electronic questionnaires has been developed by an ISPOR task force[72], which also highlights the type of evidence needed to demonstrate equivalence of both formats.

CONCLUSION

Utilising modern information technology, data collection, processing and intuitive graphical data display in real-time, electronic PRO assessment can be implemented in daily clinical practice. With home monitoring, the individual recovery process of the patients can be observed via an easy-to-use web portal and, if necessary, focused appointments can be made for clinically important parameters. With this transition to daily practice, PROMs expand from their original homeland of clinical studies and data registries and enter routine outpatient appointments to facilitate shared decision making, help manage patient expectations and complement follow-up.