Patient Engagement
Capturing the Patient Voice

The Cost of Patient-Reported Outcomes in Medicine

Article · January 25, 2018

We define Patient-Reported Outcomes (PROs) as assessments that directly capture the medical symptoms of the patient such as physical function, pain, and mental and social health well-being without interpretation of the patient’s responses by their clinician. The direct entry of this PRO electronically into the health record allows for graphic display of the data for easy interpretation of individual progress as well as aggregation of the data over common grouping such as diagnosis, surgery type, or patient characteristics. Capturing the voice of the patient through PROs and immediately incorporating this information into the electronic health record is critical to advance quality medical care.

Numerous publications and multispecialty consensus guidelines highlight the important organizational steps regarding PRO collection including governance, staff training, visualization, frequency, and ethical and legal considerations. Despite these endorsements and resources, the adoption of PROs by medical providers for immediate use in clinical settings has been slow. A program of real-time utilization of PROs can have profound positive influences on shared decision-making, quality improvement, and strategic allocation of institutional resources. Currently, technological barriers as well as perceived cost challenges obstruct the wider adoption of PROs in clinical settings.

There are three common methods to import PRO data into the electronic health record (EHR): custom-built local (institutional) solutions, EHR manufacturer–embedded PROs, and independent commercial constructed products. All have both hardware and software requirements. The factors that determine which of these options is best for a program include the available resources (personnel and hardware) for information technology (IT) construction, the ability to locally customize the product with PRO content experts, and, of course, the financial budget allocated for the program. If there is a larger, diverse provider and patient population, it may be more economical to build an institutional custom-built system.

The Medicare Merit-based Incentive Payment System (MIPS) reimbursement and bundled payment pilots have made routine collection of PROs a useful strategy in justifying and optimizing compensation for patient care. As more patients and conditions are covered under these reimbursement models, greater investment by health care systems in PRO collection can be anticipated. Additionally, as alternative payment models begin to specify the collection of specific PRO instruments, it is likely that a more customized platform (either institutional or commercial) may be required to meet that demand. Table 1 provides the advantages and challenges with each PRO distribution and collection platform.

Advantages and Challenges of Different Patient-Reported Outcome PRO Collection Systems

Table 1. Click To Enlarge.

Examples of institutional custom-built programs include the University of Rochester, University of Utah, and Washington University. The advantages of this type of program include flexibility in content, ability to tailor workflow and integration, and the capacity to change collection over time. The data display for both review and analysis can also be altered and customized. Scoring and updates can be completed in real time. Data sharing with other groups is easily performed.

The challenges include the need for IT expertise, provider content knowledge and interest, and ongoing resources for program maintenance. Institutions that employ IT specialists and can identify a central IT PRO leader may opt for this strategy. Clinician champions are required to help select content, and they will require salary support. The computer hardware requirements for a custom PRO program may seem extensive; however, much of the equipment is likely already in place for other institutional endeavors. Table 2 lists the hardware and personnel costs associated with a unique, custom-built PRO collection system.

Example Cost of Custom-Built PRO Collection Platform

Table 2. Click To Enlarge.

Despite the plethora of commercially available EHR platforms utilized in health care, embedded PROs have been limited to static forms or simple branching questionnaires that have been found to take more time and be more burdensome to the patient than newer IT using Computer Adaptive Technology (CAT) capabilities within the EHR programming. One such PRO is the Patient-Reported Outcome Measurement Information System available as static forms or CAT methodology, the PROMIS short form 10 (PROMIS 10), available in two of the most widely used EHR vendors for over 5 years. This assessment is free; however, the data analysis is limited to raw scoring only. This limits the utility of this instrument unless the raw score is secondarily converted to a t-score for comparison. PROMIS CAT methodology capitalizes on item response theory (IRT), in which the next question presented is based on the previously answered question. This PROMIS CAT capture increases the accuracy and efficiency over short forms. There are a wide array of PROMIS CAT assessments on focused domains such as physical function, pain, and depression. PROMIS CAT assessments will be available in one EHR in 2018 — however, this will require both an EHR software upgrade and additional usage fees. Unlike the short forms, a numeric score will be available; however, further customization for different patient populations and different instruments will be limited. Exporting PRO data from within the EHR system, as well as linking PRO data to other personal health identifiers for research, may be difficult.

There are numerous commercial vendor “add-on” products for PRO collection and display. As might be expected, these can range from very basic to extremely customized programming, and the cost varies accordingly. These products distribute the outcome assessments and registration materials directly to the patient in the clinic, via email or through electronic text messaging. The patient’s contact information must initially be collected by medical personnel and linked to the commercial product, which can be burdensome. Some systems solicit the patient via text or email to open a web-enabled link, and to complete the PRO assessments. This approach may allow integration within the EHR using HL7 feeds. In this strategy, the raw data collected is maintained on the vendor’s server. Viewing of the data is usually via the manufacturer’s platform and not within the institution’s electronic record. Standard reports are available; however, data linkage with the EHR fields for research exploration is both slow and limited in scope.

Implementation of routine PRO collection is paramount to measuring and maximizing value in health care. Although there is understandable concern over the IT costs inherent in incorporating PRO into the clinical care workflow, there is a substantial return on investment seen through improved patient engagement, advancement in clinical research, and the ability to influence the health care value equation. PRO implementation projects require both leadership and institutional support to advance. The University of Rochester Medical Center leadership, as well as leaders at the University of Utah, the Cleveland Clinic, and Partners HealthCare made PRO collection across the health care system a priority and part of both the strategic and business plans of the institution. As pay-for-performance initiatives and bundled payment projects continue to advance, the patient’s voice, through PROs, will tell providers, insurers, and health care administrators what surgeries, medications, and other therapies improve their symptoms and quality of life. Institutions with robust PRO data will be well positioned to compete in the marketplace.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
Barriers to Providing an Oustanding Patient Experience

Buzz Survey Report: Patient Experience

An independent NEJM Catalyst report sponsored by University of Utah Health on barriers to achieving an excellent patient experience.

The Patient Engagement Capacity Framework

The Patient Engagement Capacity Model: What Factors Determine a Patient’s Ability to Engage?

Patient engagement assessments often don’t dig deep enough to identify why patients don’t participate in their own health care. We present a new model to help providers pinpoint the reasons for lack of engagement and address them more effectively.

Organizational Mindset Is the Biggest Barrier to Engaging Patients as Consumers

Survey Snapshot: The Patient-Physician Relationship Is Key

Both parties involved in a consumer-facing transaction have access to important information about the product or service — but this isn’t the case with health care.

What High-Need, High-Cost Patients Say About How to Reduce High Utilization of ED and Inpatient Services

High-Need, High-Cost Patients Offer Solutions for Improving Their Care and Reducing Costs

More home health care and after-hours clinics, telemedicine, and home delivery of medications are among top solutions.

Top Physician Pain Points Identified by Chronic Patients

Unmet Needs: Hearing the Challenges of Chronic Patients with Artificial Intelligence

With natural language processing and machine learning, researchers are identifying patient emotional and medical needs that are not being met by clinicians and patient advocacy groups.

Health Care Can Learn a Lot from Other Consumer-Facing Industries - Especially About Customer Service - Consumerization of Health Care

Patient Engagement Survey: Health Care Has a Lot to Learn from Consumer-Friendly Industries

The consumerization of health care continues to reshape the way that patients engage with providers and experience care. Most providers see this fundamental change in the health care model as a necessary response to changing patient demands, and have embraced the need to learn from other industries.

Bilazarian01_pullquote patient engagement both sides of the bed

Patient Engagement from Both Sides of the Bed

When patients and families are included in medical rounds as valued members of the team, the quality of care improves.

Ten Guiding Principles for Patient-Centered Care as Described by Mountain States Health Alliance to consider with patient-centered outcomes research (PCOR)

Complementary Approaches to Patient Engagement in Patient-Centered Outcomes Research

It’s important for providers to understand and apply the key principles foundational to patient-centered outcomes research efforts.

Selected Advantages of Improved Patient Electronic Access to Health Data

Patient-Led Data Sharing — A New Paradigm for Electronic Health Data

The pieces are in place for a truly disruptive shift in how patients can access and use their own clinical data to improve their health.

Collapse of Two Online Health Communities as Result of Removal of Users Starting from Superusers

Online “Superusers” as Allies of the Health Care Workforce

Three proposed steps for integrating peer-driven online health communities with traditional services to improve health outcomes for people with chronic conditions.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Patient Incentives

70 Articles

Survey Snapshot: Is Transparency the Answer…

NEJM Catalyst Insights Council members say that while transparency might be necessary, we have to…

Primary Care

184 Articles

New Evidence on Stemming Low-Value Prescribing

New research suggests that deploying strong and surprising messages could help to address overprescribing of…

Digital Health Care in China: Benefits…

China’s experience offers insights for integrating digital and non-digital health care services.

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now