Patient Engagement

Implementing Patient-Reported Outcome Measures

Article · September 5, 2017

In addition to saving lives, our primary mission in health care is to improve the lives of patients. This means reducing symptoms like chest pain or incontinence, or helping people walk better or open a jar by themselves. But how do you measure symptoms or function besides just asking a patient how she feels? Patient-Reported Outcome Measures (PROMs) are powerful because they use validated questionnaires to turn a symptom into a numerical score. With PROMs, for the first time we can use numbers to describe how much a knee replacement helps someone walk or to quantify the average difference in outcome between a biologic treatment versus conventional pharmacotherapy.

As we seek to provide better value in health care, we will succeed only if we can define what constitutes a good outcome. Despite the dizzying proliferation of clinical performance measures, these often miss the goal:

  • Most capture the process of care, including drawing labs or starting a medication, but fail to capture the very reason that most patients seek care, which is to improve their symptoms.
  • Current outcome measures, including mortality and hospital readmission, are important but are often multifactorial and not significantly under providers’ control.
  • Many exhibit little meaningful variation across providers and, therefore, fail to differentiate between good and exceptional care.
  • Many focus on particular diagnoses, leaving substantial gaps in our ability to measure quality across the full spectrum of care.

PROMs are precisely the missing link in defining a good outcome. They capture quality-of-life issues that are the very reasons that most patients seek care: to address a bothersome symptom, limited function, or ailing mental health. PROMs results vary tremendously among treatment choices, institutions, and providers. And PROMs are the outcome of relevance in nearly every diagnosis, including many in specialties that currently lack good quality measures. By making PROMs an integral part of clinical care, providers can use them to improve an individual patient’s care as well as in aggregate to improve care of a population.

Nevertheless, adoption of PROMS by large health systems has been slow due to several common challenges:

  • Technology barriers: To rapidly administer surveys, calculate scores, and trend results, the data need to be electronic. There is simply no time to manually collect and transcribe handwritten data. This requires an electronic platform that works with the patient portal via Wi-Fi–connected tablets. The platform must also be integrated into the electronic health record (EHR) system so that results flow into the point of care in real time in order to be actionable. And it must work nearly perfectly because neither patients nor providers have the patience for glitches.
  • Operational barriers: Increasing demands on all participants in health care have squelched any appetite for tasks that aren’t required, especially if they take time. Even if the technology works perfectly, patients, clinic staff, and clinicians — busy people already — must work together to make PROMs a reality. Convincing them that the small effort is worthwhile is half the battle. The other half is strategically using PROMs to achieve net time savings by making other activities faster.

Though these barriers loom large, Partners HealthCare has concluded that PROMs are essential to real-time clinical care and to how we measure, compare, and improve care as a system. Our large health system now collects PROMs in 21 specialties and across 64 clinics in orthopedics, urology, psychiatry, and cardiac surgery. We have collected over 200,000 PROMs and are collecting at a rate accelerating quickly past 12,000 per month.

For example, after cardiac catheterization, we collect the Seattle Angina Questionnaire and the Rose Dyspnea Scale to quantify not just how much longer patients live after the procedure, but how much their chest pain has dissipated and how much better they feel in daily life. To determine which subset of patients are most likely to benefit from surgery on their lumbar spine, we collect the PROMIS® domains, measuring functional status, pain intensity and interference, and depression and anxiety. For prostate cancer patients undergoing surgery or radiation, we collect the EPIC-CP to measure symptoms of incontinence and impotence in addition to survival.

These are just a few examples of PROMs in our organization. Less than five years into this journey, we’re beginning to see the promise of how this will help us guide and improve our clinical care.

We certainly have the battle scars you’d expect from such a complex endeavor — asking people to do yet one more thing, to incorporate yet one more piece of technology. Our efforts have taught us which design and operational choices provide the best chances for success. Even when you get everything right, it doesn’t always work, and perhaps the lesson we have learned more than any other is humility. Despite the challenges, PROMs clearly are an integral part of health care’s future. What follows are the lessons we have learned.

How to Engage Patients

Today’s processes ask more of our patients with intake forms, screening questionnaires, patient experience and satisfaction scores, research studies, and more. Increasingly patients begin to doubt the value of the time spent. Without further engagement and education, patients approach PROMs warily. We have found three design principles to be critical for success with patients:

Make it easy: Reach patients where it’s convenient for them, and provide a simple user interface.

Multimodal PROMs collection is essential. The patient portal is the most obvious way to collect PROMs, and it should be the cornerstone of collection. We recommend sending out PROMs before a visit or at the appropriate interval after surgery so the patient can complete it online in the comfort of home.

However, because collecting only via the portal leaves out those who aren’t booting up laptops every night, it is necessary to also collect in clinical settings, with patients using either using tablets or a clinician terminal. (I do this in my exam room daily.) Epic, which we use at Partners, has reasonably convenient functionality on both these fronts.

Interestingly, the notion that older patients can’t use tablets is a myth. “I don’t know how to use a computer,” my patient tells me. “Tablet computer? No way.” But hand him an iPad and he has no problem, and in fact, elderly patients often enjoy it. (Tip: Have a stylus on hand.) Want to reach millennials? Make sure you have text messaging and/or an app (coming soon to Partners).

Make it fast: Don’t ask too many questions and don’t ask too frequently.

There is a natural temptation to collect the wish list of every clinician and researcher involved. Don’t. We learned this when collecting PROMs data relating to pain and mobility of knees. We started out collecting the full KOOS (42 questions) plus the PROMIS-10 Global (10 questions) for knee replacement, and we simply failed. Few of us have patience for 52 questions. Beyond 30 or so questions, patients simply stop answering. Find the most actionable, relevant PROMs for a condition, and ask once per year, perhaps twice, if someone has surgery.

PROMS: Total Knee Replacement - Relief from Knee Pain

Total Knee Replacement: Relief from Knee Pain. Click To Enlarge.

Make it relevant: Look at and use responses in real time to care for the patient.

Perhaps the biggest determinant of whether a patient answers a PROM is whether her doctor looked at her answers the last time. Many of our radiation oncologists use PROMs that ask about side effects and promptly use the information when considering a dose adjustment. Collection rates approach 100% for those providers using the data, not surprisingly. A patient might give you the benefit of the doubt the first time, but ignore the effort she put in at your peril.

This begs the question, how do you get a busy clinician to look at the responses?

How to Engage Clinicians

Early in our program, providers had little interest in PROMs, often dismissing them as a research study. But these days, the response is different. Providers know that PROMs are coming and soon will be required. The question for them now seems to be whether to get some experience now or hold out and be the last iceberg. We have found three ways to convert a clinician from resigned acceptance to an enthusiastic user and champion of PROMs:

Make care easier: Use the PROMs collection platform to reduce administrative burden or meet regulations.

To my dismay, perhaps the most compelling opening argument to providers (including front desk staff and medical assistants) is the opportunity to use the PROMS platform to simultaneously reduce administrative burden. Because a PROMs platform captures data directly from patients, it can be used for pre-visit information entry and mandatory screenings (e.g., smoking status, fall risk, depression, health risk assessments). And it satisfies two-way electronic communication requirements for meaningful use. The clinician can also reference this information in clinical notes, reducing documentation burden.

Make care faster/better: Incorporate PROMs responses in real-time into the EHR to save time and/or provide better clinical care.

Ultimately, what truly engages clinicians is direct care improvement. Incorporating PROMs data into the EHR in real-time can save time and empower clinicians to engage patients in better, more personalized care. For example, every day in my primary care clinic, I see men with symptoms of benign prostatic hypertrophy (BPH). Asking about the seven cardinal symptoms of BPH and their resulting quality of life can absorb precious time. When a patient comes in with his International Prostate Symptom Score already answered, I vault to step 9 of the conversation, the valuable part where we talk about how to help. Instead of spending our time gathering information, we use it to marry my clinical knowledge with the patient’s unique goals and preferences to produce a better outcome. The same is true for patients with Crohn’s disease and COPD.

Make PROMs relevant: Put responses in context with graphical information, decision support, or shared decision-making tools.

PROMs can help patients make informed medical decisions, by clarifying the risks and benefits from the abstract to the numerical or visual.

For example, while some patients with prostate cancer require a life-saving radical prostatectomy, others face a decision of whether to pursue surgery or active surveillance. Some patients think, “Better safe than sorry,” and opt for surgery even when informed of the risks of incontinence and sexual dysfunction. Using the Expanded Prostate Cancer Index Composite for Clinical Practice (EPIC-CP) mentioned earlier, with its detail about incontinence and sexual function after surgery, a patient and his urologist can make a better informed shared decision by viewing these data alongside data about longevity for his specific diagnosis. In another example, the graph shows how we present Partners’ aggregate results for knee replacement to patients considering surgery, to help them understand how much better they might get, how quickly, and how sustainably.

What’s Next for PROMs?

When we began this journey almost five years ago, it was hard to see how a health system that couldn’t invest as much as Partners could succeed with PROMs. Now, more turnkey solutions are being made available within EHRs, and we hope that sharing our experience will help others avoid our early mistakes.

Despite progress, challenges remain for widespread implementation of PROMs collection platforms:

  • Although there is a lack of consensus on which PROM is best for many conditions, there is hope. Groups such as PROMIS plan to provide free, concise, valid PROMs, and the International Consortium for Health Outcomes Measurement (ICHOM) and others are encouraging coalescence around a single set of measures for a particular diagnosis.
  • If PROMs are to be used eventually as a quality measure for reimbursement programs, we must determine how to risk-adjust for clinical and sociodemographic patient characteristics. Data interoperability will also be essential.
  • Finally, as PROMs collection becomes more widespread, we’ll need strategies to avoid overwhelming patients with constant surveys and to reach them conveniently without jeopardizing privacy.

Health care is currently grappling with how to make PROMs an integral part of our care. At Partners, we’ve shown that it is possible when done thoughtfully and relentlessly. Moving forward, the challenge is to use PROMs first to understand, and then to meaningfully transform, what we do to provide the highest value care to our patients.


Acknowledgments: This work is made possible because of the vision, support, and guidance of the leadership of Partners Clinical Affairs, the Center for Population Health, and the Department of Quality, Safety, and Value: Sreekanth Chaguturu, MD, Timothy Ferris, MD, MPH, Gregg Meyer, MD, MPH, and Thomas Sequist, MD, MPH.

We would not have a program without the innovative and tireless work of the Partners PROMs team from whose work these lessons come. Great thanks especially to Director Peter Meyers and to the entire team: Ronnen Cretu, Bernard Joseph, MD, Shawn Martin, Negui Rojas, David St. Amand, Ana Ulchak, Patrick Wright, and Randi Zegman, as well as our past and current partners at our institutions, Adam Licurse, MD, Rachel Clark, MD, Ishani Ganguli, MD, and Josie Elias.

This article was originally published on November 16, 2016. 

New Call for Submissions ­to NEJM Catalyst


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement

Why People Stick With or Abandon Wearable Devices

Embedding fitness trackers in eyeglasses demonstrates that personality and motivational feedback play important roles in activity levels.

Shared Decision Making: Time to Get Personal

Dell Medical School’s musculoskeletal providers are using patient-reported outcome measures to create a more personalized shared decision-making process.

Patient-Reported Outcomes — Are They Living Up to Their Potential?

The patient’s voice can be at the forefront of health care delivery.

Patient Satisfaction in Anesthesiology Matters

A good patient experience is part of a successful anesthetic.

Walking with Our Patients in the Community

Mobile integrated health, a disruptive home-based delivery model, repurposes existing resources and addresses critical care gaps for high-risk, high-cost populations.

Home-to-Home Time — Measuring What Matters to Patients and Payers

Although discharging patients to a post-acute care facility might help hospitals reduce their average length of stay, it’s unclear when post-acute care improves patient outcomes and when it doesn’t.

Improving Communication from the Operating Room: A New Mobile Application to Enhance the Family Experience

How Orlando Health’s new mobile app resulted in a 98% reduced anxiety rate for patients and their families in the waiting room.

Data Graphic: What Are the Most Effective Technologies in Patient Engagement Initiatives?

Members of the NEJM Catalyst Insights Council say that biometric measurement devices such as wireless scales or glucometers are the most effective.

In Your Words: Where Is the Greatest Potential for Technology to Enable Patient Engagement?

Responses from a survey of NEJM Catalyst Insights Council members.

Four Challenges of Launching a Telehealth Program

NewYork-Presbyterian tackled liability, licensure, reimbursement, and workflow to launch their successful telehealth program.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Improving Care and Cutting Costs: Implementation…

Can a formulary system help to prioritize cost-effective lab tests in the same way it…

Patient Incentives

51 Articles

Why People Stick With or Abandon…

Embedding fitness trackers in eyeglasses demonstrates that personality and motivational feedback play important roles in…

Patients As Customers

86 Articles

Better Communication Makes Better Physicians

How Adrienne Boissy and Cleveland Clinic are focusing on physician communication — and getting results.

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now