Patient Engagement
Technology to Empower Patients

Patient-Reported Outcomes — Are They Living Up to Their Potential?

Article · August 14, 2017

As part of a nationwide movement toward giving patients more of a voice in their health care, an increasing number of organizations are collecting and assessing patient-reported outcomes (PROs). There is a growing chorus of support from clinicians, researchers, and payers for embracing PRO measurement instruments in clinical care. But there are still important practical questions about how data on these outcomes should be collected, visualized, shared, and used to improve the quality of care.

At the orthopedic surgery department at the University of Rochester Medical Center, we have collected PROs during every outpatient clinic visit for the past 2 years, a practice that was expanded throughout 30 departments and divisions over the past year.1 Our decision to commit to PRO assessments was inspired by a study that compared physical function scores obtained in the office using the Patient-Reported Outcomes Measurement Information System (PROMIS) with the GAITRite temporal and spatial gait-analysis system, which measures walking speed, cadence, stride length, and other gait parameters directly — and costs $52,000.2,3 The study included 106 patients who underwent knee-ligament reconstruction. It showed that the PROMIS physical function assessment took 1 minute and was more precise than the gait-analysis approach, which took 10 to 15 minutes to complete. What’s more, the PROMIS assessment had less of a ceiling effect: none of the participants had the system’s highest possible score, whereas some using the gait-analysis approach did (see graphs).

Physical Function Assessments after Knee-Ligament Reconstruction. Mean PROMIS physical function T scores (Panel A) and GAITRite velocity scores (Panel B) were obtained at baseline and over 1 year. Error bars indicate the standard error, and asterisks a significant difference from baseline (P

Physical Function Assessments after Knee-Ligament Reconstruction. Mean PROMIS physical function T scores (Panel A) and GAITRite velocity scores (Panel B) were obtained at baseline and over 1 year. Error bars indicate the standard error, and asterisks a significant difference from baseline (P<0.001). Modified from Papuga et al. Click To Enlarge.

The validated PROMIS measurement tool uses computer adaptive technology and item-response theory. Each question is selected using a patient’s previous responses, allowing the system to assign a score from a limited amount of information. Patients answer an average of four to seven questions on a Wi-Fi–enabled tablet, and the system leverages a larger database — in the case of the physical function assessment, one with 121 validated items — to produce an accurate, reproducible score. An independent interface allows physicians to instantly view patient scores, compare them with scores from a reference population, and use them to support shared decision making with the patient. To permit more nimble access, PRO data are stored on a separate server rather than in the electronic health record (EHR), but they can be linked to personal health information in the EHR for the purposes of research and aggregate data assessment.

The University of Rochester collects scores from 80% of patients on three PROMIS domains — physical function, pain interference, and depression — through in-clinic testing that requires an average of 2.4 minutes to complete. Individual departments can choose to collect patient responses on additional domains; for example, physicians in our cancer center decided it was important to assess their patients’ anxiety and fatigue. Each additional domain increases completion time by approximately 1 minute, and the total number of domains is limited to five to avoid burdening patients. In 2 years, 148,000 unique patients have completed over 1.1 million PROMIS assessments.

After developing a pragmatic, efficient mechanism for collecting, visualizing, and sharing PROMIS scores, we evaluated how these data could be used to improve the quality of care. For physicians to determine whether a particular treatment option will be worthwhile for a given patient, they must understand the patient’s expectations, his or her current functional status, and how much improvement the treatment can be expected to produce. PRO data can be linked with diagnosis codes, surgical codes, and information on coexisting conditions, medications, physical therapy, and other variables in the EHR. Using the large PROMIS database, we were able to assess the effect of commonly performed surgeries on physical function, pain, and depression over the course of an episode of care.

We then performed receiver operator characteristic analysis to determine whether preoperative PROMIS scores could predict the likelihood that a patient would obtain a clinically meaningful benefit from foot and ankle surgery.4 We found that a patient with a PROMIS physical function T score above 42, for example, has a 94% chance of not experiencing a minimal clinically important difference in function after surgery. Similarly, a patient with a preoperative pain T score below 55 has a 95% chance of not obtaining a meaningful benefit in terms of pain interference. Similar assessments have been conducted for spine surgery, spinal injections, total joint replacement, and various other surgical interventions. This information can help guide decisions about surgery: discussions between surgeons and patients can focus on the expected benefit of surgery for the specific patient, rather than on the average benefit in a patient population.

Other institutions have also been incorporating PRO collection into clinical care. Health care organizations in England and Scotland have extensive experience assessing condition-specific PROs and patient scores on the EuroQol 5-Dimension Self-Report Questionnaire (EQ5D) and reporting these data publicly. In the United States, Dartmouth–Hitchcock Medical Center has assessed spine-surgery outcomes using the RAND 36-Item Short-Form General Health Survey (SF-36) for years and was an early champion of using PROs, having demonstrated the link between preoperative depression and poor surgical outcomes. This finding led the hospital to implement presurgical counseling to prepare patients for spine surgery.

The University of Utah sends PRO assessments to patients at scheduled times through a link sent to the patient’s e-mail address and receives responses from approximately 30% of patients before their appointments; scores for the remaining patients are collected in the clinic. The university also uses a supplemental application to provide clinicians with PROMIS data for various treatments, alongside validated cost data, to help inform treatment decisions. Northwestern, Stanford, Washington University, Partners HealthCare, and many other institutions are also using PROs to incorporate patients’ perceptions of their health into the medical record.

At the patient level, PRO data allow people to understand what to expect during recovery. For example, patients who have had surgery often want to know when they can return to work or participate in sports. By comparing an individual patient’s preoperative scores with prospective population-level PROMIS data, our system can create a roadmap of recovery that predicts functioning in specific areas over time to help answer patients’ questions and set appropriate expectations.

At the aggregate level, PRO data can be used to minimize variation in patient care. For example, institutions can compare data from different surgical procedures performed for the same condition to determine which ones have the best outcomes from the patient’s perspective. For procedures with similar outcomes, other factors such as costs, risks, and time to full recovery after surgery can be compared. When certain procedures are found to have less favorable outcomes, institutions can determine whether an individual surgeon’s technique needs improvement or the treatment approach should be abandoned completely.

PROs are already helping to improve patient care. By mastering the efficient measurement of these outcomes in the clinic, minimizing the reporting burden for patients, displaying PRO information at the point of care, and using outcomes predicted from population-level data to inform patient expectations, we can continue to ensure their benefits. Such a strategy allows us to help surgeons identify areas where they need improvement, eliminate procedures with less favorable outcomes, and avoid performing surgeries on patients who are unlikely to benefit from them. It also enhances patient satisfaction with care by helping physicians set appropriate expectations regarding a patient’s return to work, school, or sports. Most important, PROs place the patient’s voice at the forefront of health care delivery.


From the Department of Orthopedics, University of Rochester Medical Center, Rochester, NY.

1. Papuga MO, Dasilva C, McIntyre A, Mitten D, Kates S, Baumhauer JF. Large-scale clinical implementation of PROMIS computer adaptive testing with direct incorporation into the electronic medical record. Health Syst (Basingstoke) 2017 April 12 (Epub ahead of print).
2. Papuga MO, Beck CA, Kates SL, Schwarz EM, Maloney MD. Validation of GAITRite and PROMIS as high-throughput physical function outcome measures following ACL reconstruction. J Orthop Res 2014;32:793-801. CrossRef | Web of Science | Medline
3. Cella D, Yount S, Rothrock N, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care 2007;45:Suppl 1:S3-S11. | Web of Science | Medline
4. Ho B, Houck JR, Flemister AS, et al. Preoperative PROMIS scores predict postoperative success in foot and ankle patients. Foot Ankle Int 2016;37:911-918. CrossRef | Web of Science |

This Perspective article originally appeared in The New England Journal of Medicine.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement

Why People Stick With or Abandon Wearable Devices

Embedding fitness trackers in eyeglasses demonstrates that personality and motivational feedback play important roles in activity levels.

Patient Satisfaction in Anesthesiology Matters

A good patient experience is part of a successful anesthetic.

Walking with Our Patients in the Community

Mobile integrated health, a disruptive home-based delivery model, repurposes existing resources and addresses critical care gaps for high-risk, high-cost populations.

Home-to-Home Time — Measuring What Matters to Patients and Payers

Although discharging patients to a post-acute care facility might help hospitals reduce their average length of stay, it’s unclear when post-acute care improves patient outcomes and when it doesn’t.

EASE Orlando Health mobile app patient view

Improving Communication from the Operating Room: A New Mobile Application to Enhance the Family Experience

How Orlando Health’s new mobile app resulted in a 98% reduced anxiety rate for patients and their families in the waiting room.

Data Graphic: What Are the Most Effective Technologies in Patient Engagement Initiatives?

Members of the NEJM Catalyst Insights Council say that biometric measurement devices such as wireless scales or glucometers are the most effective.

In Your Words: Where Is the Greatest Potential for Technology to Enable Patient Engagement?

Responses from a survey of NEJM Catalyst Insights Council members.

Four Challenges of Launching a Telehealth Program

NewYork-Presbyterian tackled liability, licensure, reimbursement, and workflow to launch their successful telehealth program.

If We Can See the Outcome, We Can Change the Outcome

How Penn Medicine’s Chief Innovation Officer parlayed success at one of the nation’s biggest software companies into meaningful outcomes for health care.

Why Real-World Results Are So Challenging for Digital Health

User engagement outside of clinical trials is the critical factor.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Differences in Ambulatory EHR Use Patterns…

UCSF Health found that women providers spent more time in the EHR and documented longer…

Patient Incentives

75 Articles

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care,…

Patients As Customers

141 Articles

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now