Patient Engagement
Capturing the Patient Voice

The Cost of Patient-Reported Outcomes in Medicine

Article · January 25, 2018

We define Patient-Reported Outcomes (PROs) as assessments that directly capture the medical symptoms of the patient such as physical function, pain, and mental and social health well-being without interpretation of the patient’s responses by their clinician. The direct entry of this PRO electronically into the health record allows for graphic display of the data for easy interpretation of individual progress as well as aggregation of the data over common grouping such as diagnosis, surgery type, or patient characteristics. Capturing the voice of the patient through PROs and immediately incorporating this information into the electronic health record is critical to advance quality medical care.

Numerous publications and multispecialty consensus guidelines highlight the important organizational steps regarding PRO collection including governance, staff training, visualization, frequency, and ethical and legal considerations. Despite these endorsements and resources, the adoption of PROs by medical providers for immediate use in clinical settings has been slow. A program of real-time utilization of PROs can have profound positive influences on shared decision-making, quality improvement, and strategic allocation of institutional resources. Currently, technological barriers as well as perceived cost challenges obstruct the wider adoption of PROs in clinical settings.

There are three common methods to import PRO data into the electronic health record (EHR): custom-built local (institutional) solutions, EHR manufacturer–embedded PROs, and independent commercial constructed products. All have both hardware and software requirements. The factors that determine which of these options is best for a program include the available resources (personnel and hardware) for information technology (IT) construction, the ability to locally customize the product with PRO content experts, and, of course, the financial budget allocated for the program. If there is a larger, diverse provider and patient population, it may be more economical to build an institutional custom-built system.

The Medicare Merit-based Incentive Payment System (MIPS) reimbursement and bundled payment pilots have made routine collection of PROs a useful strategy in justifying and optimizing compensation for patient care. As more patients and conditions are covered under these reimbursement models, greater investment by health care systems in PRO collection can be anticipated. Additionally, as alternative payment models begin to specify the collection of specific PRO instruments, it is likely that a more customized platform (either institutional or commercial) may be required to meet that demand. Table 1 provides the advantages and challenges with each PRO distribution and collection platform.

Advantages and Challenges of Different Patient-Reported Outcome PRO Collection Systems

Table 1. Click To Enlarge.

Examples of institutional custom-built programs include the University of Rochester, University of Utah, and Washington University. The advantages of this type of program include flexibility in content, ability to tailor workflow and integration, and the capacity to change collection over time. The data display for both review and analysis can also be altered and customized. Scoring and updates can be completed in real time. Data sharing with other groups is easily performed.

The challenges include the need for IT expertise, provider content knowledge and interest, and ongoing resources for program maintenance. Institutions that employ IT specialists and can identify a central IT PRO leader may opt for this strategy. Clinician champions are required to help select content, and they will require salary support. The computer hardware requirements for a custom PRO program may seem extensive; however, much of the equipment is likely already in place for other institutional endeavors. Table 2 lists the hardware and personnel costs associated with a unique, custom-built PRO collection system.

Example Cost of Custom-Built PRO Collection Platform

Table 2. Click To Enlarge.

Despite the plethora of commercially available EHR platforms utilized in health care, embedded PROs have been limited to static forms or simple branching questionnaires that have been found to take more time and be more burdensome to the patient than newer IT using Computer Adaptive Technology (CAT) capabilities within the EHR programming. One such PRO is the Patient-Reported Outcome Measurement Information System available as static forms or CAT methodology, the PROMIS short form 10 (PROMIS 10), available in two of the most widely used EHR vendors for over 5 years. This assessment is free; however, the data analysis is limited to raw scoring only. This limits the utility of this instrument unless the raw score is secondarily converted to a t-score for comparison. PROMIS CAT methodology capitalizes on item response theory (IRT), in which the next question presented is based on the previously answered question. This PROMIS CAT capture increases the accuracy and efficiency over short forms. There are a wide array of PROMIS CAT assessments on focused domains such as physical function, pain, and depression. PROMIS CAT assessments will be available in one EHR in 2018 — however, this will require both an EHR software upgrade and additional usage fees. Unlike the short forms, a numeric score will be available; however, further customization for different patient populations and different instruments will be limited. Exporting PRO data from within the EHR system, as well as linking PRO data to other personal health identifiers for research, may be difficult.

There are numerous commercial vendor “add-on” products for PRO collection and display. As might be expected, these can range from very basic to extremely customized programming, and the cost varies accordingly. These products distribute the outcome assessments and registration materials directly to the patient in the clinic, via email or through electronic text messaging. The patient’s contact information must initially be collected by medical personnel and linked to the commercial product, which can be burdensome. Some systems solicit the patient via text or email to open a web-enabled link, and to complete the PRO assessments. This approach may allow integration within the EHR using HL7 feeds. In this strategy, the raw data collected is maintained on the vendor’s server. Viewing of the data is usually via the manufacturer’s platform and not within the institution’s electronic record. Standard reports are available; however, data linkage with the EHR fields for research exploration is both slow and limited in scope.

Implementation of routine PRO collection is paramount to measuring and maximizing value in health care. Although there is understandable concern over the IT costs inherent in incorporating PRO into the clinical care workflow, there is a substantial return on investment seen through improved patient engagement, advancement in clinical research, and the ability to influence the health care value equation. PRO implementation projects require both leadership and institutional support to advance. The University of Rochester Medical Center leadership, as well as leaders at the University of Utah, the Cleveland Clinic, and Partners HealthCare made PRO collection across the health care system a priority and part of both the strategic and business plans of the institution. As pay-for-performance initiatives and bundled payment projects continue to advance, the patient’s voice, through PROs, will tell providers, insurers, and health care administrators what surgeries, medications, and other therapies improve their symptoms and quality of life. Institutions with robust PRO data will be well positioned to compete in the marketplace.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
PROMs Usage Is Limited but Growing

Patient Engagement Buzz Survey: PROMs Use Is Growing, but Implementation Takes Effort

While clinical outcomes lend themselves to measurement, quantifying how an individual patient experiences symptoms or a loss of function can be more challenging. One promising approach is the use of Patient-Reported Outcome Measures (PROMs).

Taxonomy of the Patient Voice Table

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care, the terms used to describe this goal are proliferating — and muddying the discourse. This taxonomy attempts to classify some of these terms and make some distinctions.

Agarwal01_pullquote handcrafting the patient experience

Handcrafting the Patient Experience

Health care organizations can take cues from consumer-facing companies like Airbnb to creatively insert convenience and surprise into patient encounters.

Screenshot of Tidepool daily diabetes data

A Taxonomy to Engage Patients: Objectives, Design, and Patient Activation

Health information initiatives will succeed only if they focus on patients’ motivation to engage and reflect the type of engagement they seek.

Ghafur01_pullquote -digital health health care consumer patient experience

Engaging Patients Using Digital Technology — Learning from Other Industries

Providers can benefit patients and disrupt health care by learning from the experience of other industries.

Mission Hospital Virtual Sitter - Drawing of the Virtual Bed Zone and Rails

Reducing Inpatient Falls and Injury Rates by Integrating New Technology with Workflow Redesign

How Mission Hospital scaled a virtual sitter pilot and reduced unassisted falls by 44% and fall-related injuries by 40%.

Health Care Providers Should Incentivize Patients

Survey Snapshot: Patient Financial Incentives — There Are No Quick Fixes

The NEJM Catalyst Insights Council agrees that while financial incentives are a common strategy to engage patients in healthy behaviors, they are not necessarily effective.

Support of Family and Friends Is More Effective Than Clinician Support in Realizing Health Goals - From the Patient Engagement Insights Report: Why No Single Health Incentive Works.

Patient Engagement Survey: Why No Single Health Incentive Works

Initiatives to improve patient engagement come in a variety of forms. While insurers, employers, and health care providers are all involved in using financial incentives and penalties for engagement efforts, improvement in health outcomes has been elusive. Achieving that ultimate goal will usually require a combination of financial and social approaches.

Health Systems Attending the Nudge Units in Health Care Symposium - Penn Medicine

Key Insights on Launching a Nudge Unit within a Health Care System

Leaders are finding that making higher-value choices easier through subtle changes to choice architecture can have an outsized impact on medical decision-making.

Barriers to Providing an Oustanding Patient Experience

Buzz Survey Report: Patient Experience

An independent NEJM Catalyst report sponsored by University of Utah Health on barriers to achieving an excellent patient experience.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Patient Incentives

75 Articles

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care,…

Ripe for Disruption: Why and How…

For big tech companies like Amazon, Apple, and Google, the health care sector looks ripe…

Patients As Customers

141 Articles

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now