Patient Engagement
Capturing the Patient Voice

The Cost of Patient-Reported Outcomes in Medicine

Article · January 25, 2018

We define Patient-Reported Outcomes (PROs) as assessments that directly capture the medical symptoms of the patient such as physical function, pain, and mental and social health well-being without interpretation of the patient’s responses by their clinician. The direct entry of this PRO electronically into the health record allows for graphic display of the data for easy interpretation of individual progress as well as aggregation of the data over common grouping such as diagnosis, surgery type, or patient characteristics. Capturing the voice of the patient through PROs and immediately incorporating this information into the electronic health record is critical to advance quality medical care.

Numerous publications and multispecialty consensus guidelines highlight the important organizational steps regarding PRO collection including governance, staff training, visualization, frequency, and ethical and legal considerations. Despite these endorsements and resources, the adoption of PROs by medical providers for immediate use in clinical settings has been slow. A program of real-time utilization of PROs can have profound positive influences on shared decision-making, quality improvement, and strategic allocation of institutional resources. Currently, technological barriers as well as perceived cost challenges obstruct the wider adoption of PROs in clinical settings.

There are three common methods to import PRO data into the electronic health record (EHR): custom-built local (institutional) solutions, EHR manufacturer–embedded PROs, and independent commercial constructed products. All have both hardware and software requirements. The factors that determine which of these options is best for a program include the available resources (personnel and hardware) for information technology (IT) construction, the ability to locally customize the product with PRO content experts, and, of course, the financial budget allocated for the program. If there is a larger, diverse provider and patient population, it may be more economical to build an institutional custom-built system.

The Medicare Merit-based Incentive Payment System (MIPS) reimbursement and bundled payment pilots have made routine collection of PROs a useful strategy in justifying and optimizing compensation for patient care. As more patients and conditions are covered under these reimbursement models, greater investment by health care systems in PRO collection can be anticipated. Additionally, as alternative payment models begin to specify the collection of specific PRO instruments, it is likely that a more customized platform (either institutional or commercial) may be required to meet that demand. Table 1 provides the advantages and challenges with each PRO distribution and collection platform.

Advantages and Challenges of Different Patient-Reported Outcome PRO Collection Systems

Table 1. Click To Enlarge.

Examples of institutional custom-built programs include the University of Rochester, University of Utah, and Washington University. The advantages of this type of program include flexibility in content, ability to tailor workflow and integration, and the capacity to change collection over time. The data display for both review and analysis can also be altered and customized. Scoring and updates can be completed in real time. Data sharing with other groups is easily performed.

The challenges include the need for IT expertise, provider content knowledge and interest, and ongoing resources for program maintenance. Institutions that employ IT specialists and can identify a central IT PRO leader may opt for this strategy. Clinician champions are required to help select content, and they will require salary support. The computer hardware requirements for a custom PRO program may seem extensive; however, much of the equipment is likely already in place for other institutional endeavors. Table 2 lists the hardware and personnel costs associated with a unique, custom-built PRO collection system.

Example Cost of Custom-Built PRO Collection Platform

Table 2. Click To Enlarge.

Despite the plethora of commercially available EHR platforms utilized in health care, embedded PROs have been limited to static forms or simple branching questionnaires that have been found to take more time and be more burdensome to the patient than newer IT using Computer Adaptive Technology (CAT) capabilities within the EHR programming. One such PRO is the Patient-Reported Outcome Measurement Information System available as static forms or CAT methodology, the PROMIS short form 10 (PROMIS 10), available in two of the most widely used EHR vendors for over 5 years. This assessment is free; however, the data analysis is limited to raw scoring only. This limits the utility of this instrument unless the raw score is secondarily converted to a t-score for comparison. PROMIS CAT methodology capitalizes on item response theory (IRT), in which the next question presented is based on the previously answered question. This PROMIS CAT capture increases the accuracy and efficiency over short forms. There are a wide array of PROMIS CAT assessments on focused domains such as physical function, pain, and depression. PROMIS CAT assessments will be available in one EHR in 2018 — however, this will require both an EHR software upgrade and additional usage fees. Unlike the short forms, a numeric score will be available; however, further customization for different patient populations and different instruments will be limited. Exporting PRO data from within the EHR system, as well as linking PRO data to other personal health identifiers for research, may be difficult.

There are numerous commercial vendor “add-on” products for PRO collection and display. As might be expected, these can range from very basic to extremely customized programming, and the cost varies accordingly. These products distribute the outcome assessments and registration materials directly to the patient in the clinic, via email or through electronic text messaging. The patient’s contact information must initially be collected by medical personnel and linked to the commercial product, which can be burdensome. Some systems solicit the patient via text or email to open a web-enabled link, and to complete the PRO assessments. This approach may allow integration within the EHR using HL7 feeds. In this strategy, the raw data collected is maintained on the vendor’s server. Viewing of the data is usually via the manufacturer’s platform and not within the institution’s electronic record. Standard reports are available; however, data linkage with the EHR fields for research exploration is both slow and limited in scope.

Implementation of routine PRO collection is paramount to measuring and maximizing value in health care. Although there is understandable concern over the IT costs inherent in incorporating PRO into the clinical care workflow, there is a substantial return on investment seen through improved patient engagement, advancement in clinical research, and the ability to influence the health care value equation. PRO implementation projects require both leadership and institutional support to advance. The University of Rochester Medical Center leadership, as well as leaders at the University of Utah, the Cleveland Clinic, and Partners HealthCare made PRO collection across the health care system a priority and part of both the strategic and business plans of the institution. As pay-for-performance initiatives and bundled payment projects continue to advance, the patient’s voice, through PROs, will tell providers, insurers, and health care administrators what surgeries, medications, and other therapies improve their symptoms and quality of life. Institutions with robust PRO data will be well positioned to compete in the marketplace.

New call for submissions ­to NEJM Catalyst

Now inviting longform articles


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
Bilazarian01_pullquote patient engagement both sides of the bed

Patient Engagement from Both Sides of the Bed

When patients and families are included in medical rounds as valued members of the team, the quality of care improves.

Ten Guiding Principles for Patient-Centered Care as Described by Mountain States Health Alliance to consider with patient-centered outcomes research (PCOR)

Complementary Approaches to Patient Engagement in Patient-Centered Outcomes Research

It’s important for providers to understand and apply the key principles foundational to patient-centered outcomes research efforts.

Selected Advantages of Improved Patient Electronic Access to Health Data

Patient-Led Data Sharing — A New Paradigm for Electronic Health Data

The pieces are in place for a truly disruptive shift in how patients can access and use their own clinical data to improve their health.

Collapse of Two Online Health Communities as Result of Removal of Users Starting from Superusers

Online “Superusers” as Allies of the Health Care Workforce

Three proposed steps for integrating peer-driven online health communities with traditional services to improve health outcomes for people with chronic conditions.

The PEACE Project - Patient Experience for Acute Care Elders at Sarasota Memorial Hospital

The PEACE Project

Improving the acute care experience for our eldest patients.

Rabson01_pullquote barriers to patient engagement in primary care practices

What Does It Take to Increase Patient Engagement in Primary Care Settings?

Massachusetts primary care practices are beginning to invest in programs to increase patient engagement, but these strategies can be challenging to implement and remain underutilized and undervalued.

Many Barriers to Engaging Patients in Treatment for Obesity

Survey Snapshot: Approaches to Address Clinician and Societal Roles in Obesity

The obesity problem is getting larger and larger. Some solutions lie beyond the traditional medical setting.

Esch01_pullquote patient activation

OpenNotes, Patient Narratives, and Their Transformative Effects on Patient-Centered Care

The development of standardized tools and techniques are enhancing the ability of providers to interact with patients, but true patient engagement requires a common understanding of its theoretical foundation and an open mind for including patients’ needs and beliefs — in their language — as part of each encounter.

Patients and PCPs Hold Primary Responsibility for Addressing Obesity

Patient Engagement Survey: The Failure of Obesity Efforts and the Collective Nature of Solutions

Patients hold responsibility for their actions, but providers’ approaches to address obesity often fall short as well.

Items from the Shared Decision-Making Process Survey for Elective Surgical Procedures - patient decision aids - decision quality - informed consent

Shared Decision-Making: Staying Focused on the Ultimate Goal

Despite growing acceptance and enthusiasm for patient involvement in their health care decisions, clinicians, at times, lose focus on the ultimate goal of shared decision-making: better health decisions from the informed patient’s perspective.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Patient Incentives

68 Articles

Online “Superusers” as Allies of the…

Three proposed steps for integrating peer-driven online health communities with traditional services to improve health…

Patient-Centered Care

245 Articles

Engaging Stakeholders to Produce Sustainable Change…

How an initiative designed to improve patient outcomes and satisfaction while containing costs led to…

Patients As Customers

123 Articles

Patient Engagement from Both Sides of…

When patients and families are included in medical rounds as valued members of the team,…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now