Patient Engagement
Capturing the Patient Voice

Implementing Patient-Reported Outcome Measures

Article · October 12, 2017

In addition to saving lives, our primary mission in health care is to improve the lives of patients. This means reducing symptoms like chest pain or incontinence, or helping people walk better or open a jar by themselves. But how do you measure symptoms or function besides just asking a patient how she feels? Patient-Reported Outcome Measures (PROMs) are powerful because they use validated questionnaires to turn a symptom into a numerical score. With PROMs, for the first time we can use numbers to describe how much a knee replacement helps someone walk or to quantify the average difference in outcome between a biologic treatment versus conventional pharmacotherapy.

As we seek to provide better value in health care, we will succeed only if we can define what constitutes a good outcome. Despite the dizzying proliferation of clinical performance measures, these often miss the goal:

  • Most capture the process of care, including drawing labs or starting a medication, but fail to capture the very reason that most patients seek care, which is to improve their symptoms.
  • Current outcome measures, including mortality and hospital readmission, are important but are often multifactorial and not significantly under providers’ control.
  • Many exhibit little meaningful variation across providers and, therefore, fail to differentiate between good and exceptional care.
  • Many focus on particular diagnoses, leaving substantial gaps in our ability to measure quality across the full spectrum of care.

PROMs are precisely the missing link in defining a good outcome. They capture quality-of-life issues that are the very reasons that most patients seek care: to address a bothersome symptom, limited function, or ailing mental health. PROMs results vary tremendously among treatment choices, institutions, and providers. And PROMs are the outcome of relevance in nearly every diagnosis, including many in specialties that currently lack good quality measures. By making PROMs an integral part of clinical care, providers can use them to improve an individual patient’s care as well as in aggregate to improve care of a population.

Nevertheless, adoption of PROMS by large health systems has been slow due to several common challenges:

  • Technology barriers: To rapidly administer surveys, calculate scores, and trend results, the data need to be electronic. There is simply no time to manually collect and transcribe handwritten data. This requires an electronic platform that works with the patient portal via Wi-Fi–connected tablets. The platform must also be integrated into the electronic health record (EHR) system so that results flow into the point of care in real time in order to be actionable. And it must work nearly perfectly because neither patients nor providers have the patience for glitches.
  • Operational barriers: Increasing demands on all participants in health care have squelched any appetite for tasks that aren’t required, especially if they take time. Even if the technology works perfectly, patients, clinic staff, and clinicians — busy people already — must work together to make PROMs a reality. Convincing them that the small effort is worthwhile is half the battle. The other half is strategically using PROMs to achieve net time savings by making other activities faster.

Though these barriers loom large, Partners HealthCare has concluded that PROMs are essential to real-time clinical care and to how we measure, compare, and improve care as a system. Our large health system now collects PROMs in 21 specialties and across 64 clinics in orthopedics, urology, psychiatry, and cardiac surgery. We have collected over 200,000 PROMs and are collecting at a rate accelerating quickly past 12,000 per month.

For example, after cardiac catheterization, we collect the Seattle Angina Questionnaire and the Rose Dyspnea Scale to quantify not just how much longer patients live after the procedure, but how much their chest pain has dissipated and how much better they feel in daily life. To determine which subset of patients are most likely to benefit from surgery on their lumbar spine, we collect the PROMIS® domains, measuring functional status, pain intensity and interference, and depression and anxiety. For prostate cancer patients undergoing surgery or radiation, we collect the EPIC-CP to measure symptoms of incontinence and impotence in addition to survival.

These are just a few examples of PROMs in our organization. Less than five years into this journey, we’re beginning to see the promise of how this will help us guide and improve our clinical care.

We certainly have the battle scars you’d expect from such a complex endeavor — asking people to do yet one more thing, to incorporate yet one more piece of technology. Our efforts have taught us which design and operational choices provide the best chances for success. Even when you get everything right, it doesn’t always work, and perhaps the lesson we have learned more than any other is humility. Despite the challenges, PROMs clearly are an integral part of health care’s future. What follows are the lessons we have learned.

How to Engage Patients

Today’s processes ask more of our patients with intake forms, screening questionnaires, patient experience and satisfaction scores, research studies, and more. Increasingly patients begin to doubt the value of the time spent. Without further engagement and education, patients approach PROMs warily. We have found three design principles to be critical for success with patients:

Make it easy: Reach patients where it’s convenient for them, and provide a simple user interface.

Multimodal PROMs collection is essential. The patient portal is the most obvious way to collect PROMs, and it should be the cornerstone of collection. We recommend sending out PROMs before a visit or at the appropriate interval after surgery so the patient can complete it online in the comfort of home.

However, because collecting only via the portal leaves out those who aren’t booting up laptops every night, it is necessary to also collect in clinical settings, with patients using either using tablets or a clinician terminal. (I do this in my exam room daily.) Epic, which we use at Partners, has reasonably convenient functionality on both these fronts.

Interestingly, the notion that older patients can’t use tablets is a myth. “I don’t know how to use a computer,” my patient tells me. “Tablet computer? No way.” But hand him an iPad and he has no problem, and in fact, elderly patients often enjoy it. (Tip: Have a stylus on hand.) Want to reach millennials? Make sure you have text messaging and/or an app (coming soon to Partners).

Make it fast: Don’t ask too many questions and don’t ask too frequently.

There is a natural temptation to collect the wish list of every clinician and researcher involved. Don’t. We learned this when collecting PROMs data relating to pain and mobility of knees. We started out collecting the full KOOS (42 questions) plus the PROMIS-10 Global (10 questions) for knee replacement, and we simply failed. Few of us have patience for 52 questions. Beyond 30 or so questions, patients simply stop answering. Find the most actionable, relevant PROMs for a condition, and ask once per year, perhaps twice, if someone has surgery.

PROMS: Total Knee Replacement - Relief from Knee Pain

Total Knee Replacement: Relief from Knee Pain. Click To Enlarge.

Make it relevant: Look at and use responses in real time to care for the patient.

Perhaps the biggest determinant of whether a patient answers a PROM is whether her doctor looked at her answers the last time. Many of our radiation oncologists use PROMs that ask about side effects and promptly use the information when considering a dose adjustment. Collection rates approach 100% for those providers using the data, not surprisingly. A patient might give you the benefit of the doubt the first time, but ignore the effort she put in at your peril.

This begs the question, how do you get a busy clinician to look at the responses?

How to Engage Clinicians

Early in our program, providers had little interest in PROMs, often dismissing them as a research study. But these days, the response is different. Providers know that PROMs are coming and soon will be required. The question for them now seems to be whether to get some experience now or hold out and be the last iceberg. We have found three ways to convert a clinician from resigned acceptance to an enthusiastic user and champion of PROMs:

Make care easier: Use the PROMs collection platform to reduce administrative burden or meet regulations.

To my dismay, perhaps the most compelling opening argument to providers (including front desk staff and medical assistants) is the opportunity to use the PROMS platform to simultaneously reduce administrative burden. Because a PROMs platform captures data directly from patients, it can be used for pre-visit information entry and mandatory screenings (e.g., smoking status, fall risk, depression, health risk assessments). And it satisfies two-way electronic communication requirements for meaningful use. The clinician can also reference this information in clinical notes, reducing documentation burden.

Make care faster/better: Incorporate PROMs responses in real-time into the EHR to save time and/or provide better clinical care.

Ultimately, what truly engages clinicians is direct care improvement. Incorporating PROMs data into the EHR in real-time can save time and empower clinicians to engage patients in better, more personalized care. For example, every day in my primary care clinic, I see men with symptoms of benign prostatic hypertrophy (BPH). Asking about the seven cardinal symptoms of BPH and their resulting quality of life can absorb precious time. When a patient comes in with his International Prostate Symptom Score already answered, I vault to step 9 of the conversation, the valuable part where we talk about how to help. Instead of spending our time gathering information, we use it to marry my clinical knowledge with the patient’s unique goals and preferences to produce a better outcome. The same is true for patients with Crohn’s disease and COPD.

Make PROMs relevant: Put responses in context with graphical information, decision support, or shared decision-making tools.

PROMs can help patients make informed medical decisions, by clarifying the risks and benefits from the abstract to the numerical or visual.

For example, while some patients with prostate cancer require a life-saving radical prostatectomy, others face a decision of whether to pursue surgery or active surveillance. Some patients think, “Better safe than sorry,” and opt for surgery even when informed of the risks of incontinence and sexual dysfunction. Using the Expanded Prostate Cancer Index Composite for Clinical Practice (EPIC-CP) mentioned earlier, with its detail about incontinence and sexual function after surgery, a patient and his urologist can make a better informed shared decision by viewing these data alongside data about longevity for his specific diagnosis. In another example, the graph shows how we present Partners’ aggregate results for knee replacement to patients considering surgery, to help them understand how much better they might get, how quickly, and how sustainably.

What’s Next for PROMs?

When we began this journey almost five years ago, it was hard to see how a health system that couldn’t invest as much as Partners could succeed with PROMs. Now, more turnkey solutions are being made available within EHRs, and we hope that sharing our experience will help others avoid our early mistakes.

Despite progress, challenges remain for widespread implementation of PROMs collection platforms:

  • Although there is a lack of consensus on which PROM is best for many conditions, there is hope. Groups such as PROMIS plan to provide free, concise, valid PROMs, and the International Consortium for Health Outcomes Measurement (ICHOM) and others are encouraging coalescence around a single set of measures for a particular diagnosis.
  • If PROMs are to be used eventually as a quality measure for reimbursement programs, we must determine how to risk-adjust for clinical and sociodemographic patient characteristics. Data interoperability will also be essential.
  • Finally, as PROMs collection becomes more widespread, we’ll need strategies to avoid overwhelming patients with constant surveys and to reach them conveniently without jeopardizing privacy.

Health care is currently grappling with how to make PROMs an integral part of our care. At Partners, we’ve shown that it is possible when done thoughtfully and relentlessly. Moving forward, the challenge is to use PROMs first to understand, and then to meaningfully transform, what we do to provide the highest value care to our patients.


Acknowledgments: This work is made possible because of the vision, support, and guidance of the leadership of Partners Clinical Affairs, the Center for Population Health, and the Department of Quality, Safety, and Value: Sreekanth Chaguturu, MD, Timothy Ferris, MD, MPH, Gregg Meyer, MD, MPH, and Thomas Sequist, MD, MPH.

We would not have a program without the innovative and tireless work of the Partners PROMs team from whose work these lessons come. Great thanks especially to Director Peter Meyers and to the entire team: Ronnen Cretu, Bernard Joseph, MD, Shawn Martin, Negui Rojas, David St. Amand, Ana Ulchak, Patrick Wright, and Randi Zegman, as well as our past and current partners at our institutions, Adam Licurse, MD, Rachel Clark, MD, Ishani Ganguli, MD, and Josie Elias.

This article was originally published on November 16, 2016. 

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
Jethwani01_pullquote2 patient empowerment

Who Gives Us the Right to “Empower” Patients?

And if we are not truly empowering patients, what exactly should we be striving for?

Patients Welcome IV Self-Care; Physicians Hesitate

Parkland Hospital examines barriers to acceptance of a program that teaches uninsured patients to deliver IV antibiotics at home.

Patient Experience: From Consumer to Patient and All That Comes in Between

The Digital Experience Must Also Be a Human(e) Experience

The most humane experiences happen when we meet patients where they are by designing the types of touch points they want and need.

Hawaii Pacific Health Physician Checklist: Creating a Healing Experience

Transforming the Patient Experience Through the Power of Ritual

Ritual is in fact a primitive checklist: a series of steps that create safety and trust between client and caregiver.

Top Three Most Promising Trends to Capture the Patient Voice

Survey Snapshot: Many Different Channels to Hear the Patient Voice

Commentary from NEJM Catalyst Insights Council members on how to capture the patient voice and make hospital staff hear it.

Empowering Patients Empowers Providers

Why should it be surprising and novel that patient-reported outcomes have such power?

Patient-Reported Outcomes for Shared Decision-Making

Making Patients and Doctors Happier — The Potential of Patient-Reported Outcomes

PRO collection is not only feasible and good for clinical care, but it also may enhance physician satisfaction and prevent burnout.

Patient Voice Roundtable Group

Insights Roundtable Report: Measuring What Matters and Capturing the Patient Voice

From a roundtable discussion and NEJM Catalyst survey, a framework for defining the patient voice and integrating it into care delivery.

A Patient and Family Advisory Council (PFAC) for Quality

A Patient and Family Advisory Council for Quality: Making Its Voice Heard at Memorial Sloan Kettering Cancer Center

Though PFAC models abound, metrics on best practices are lacking. One prominent hospital may have found the answer: let those who are the experts at being patients and caregivers lead the way.

Digital Health: USC Virtual Care Clinic

Digital Technology to Engage Patients: Ensuring Access for All

To ensure that we engage the patient groups who have much to gain from the more flexible health care interactions that digital innovation can provide, we must consider issues of computer literacy, access, and trust.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Patient-Centered Care

277 Articles

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Patient Incentives

75 Articles

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care,…

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now