Patient Engagement

Better Physician Ratings from Discussing PROs with Patients

Case Study · June 20, 2018

The UPMC orthopaedic service line began routine collection of patient-reported outcomes (PROs) from all patients seeking orthopaedic care in 2017. In a survey of patients who had seen a foot and ankle specialist, greater shares of patients who report that their physician discussed their PROs with them reported top-box scores for physician communication and shared decision-making from the Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) survey.

There is a correlation between PROs and patient satisfaction; patients who reported that physicians were discussing their PRO responses during office visits reported significantly better ratings of physician communication and shared decision-making.

Key Takeaways

  1. Patient-reported outcomes can be routinely collected and reported in the electronic health record.

  2. Simply collecting patient-reported outcomes is not sufficient — clinicians must also discuss the responses during visits.

  3. On average, patients who report that their patient-reported outcome responses were discussed during a visit provide higher ratings for doctor communication and shared decision-making.

  4. Physicians may need training to discuss patient-reported outcome results with patients.

The Challenge

As part of the federal Comprehensive Care for Joint Replacement episode-based payment program, UPMC began collecting PROs from patients undergoing elective joint replacement in 2015. These data were used both to fulfill mandatory reporting to the Centers for Medicare and Medicaid Services and to allow clinicians to track improvement in hip and knee function from the patients’ perspective. Because of the operational and financial success of this program, the UPMC orthopaedic service line leadership expanded PRO collection to all patients presenting for care in 35 orthopaedic practices across all orthopaedic specialties in June of 2017.

The use of PROs originated in academic research arenas, but their use has expanded into both clinical care and population health management. Recently, PROs have become a central focus in health care outcomes with a movement toward patient-centeredness as embodied by the Patient-Centered Outcome Research Institute.

The widespread use of electronic health records (EHRs) has reduced the burden of capturing, analyzing, and acting upon PROs. For example, a recent randomized control trial found that patients undergoing chemotherapy for solid tumor cancers who self-reported symptoms electronically to a nursing team were less likely to visit the emergency room, remained on chemotherapy regimens longer, and survived longer than the standardized care group of patients.

PROs have also been shown to improve patient-physician communication and may help individualize treatment for patient-specific concerns or symptoms; these findings are primarily in oncology and behavioral health contexts.

With the expansion of PRO collection, and recognizing the potential to improve patient outcomes, UPMC stakeholders performed a quality improvement project to evaluate the impact that routine electronic collection of PROs has on patient perceptions of their orthopaedic care, using the foot and ankle subspecialty as a case study.

The Goal

Our goal was to better understand the association between PRO clinical use with patient engagement and patient experience in a general orthopaedic population.

The Execution

The UPMC Patient-Reported Outcomes Center creates systems for collecting PRO questionnaires from patients through EpicCare. UPMC began PRO collection through Epic in 2012 with formal creation of the PRO center in 2017, though many of the center functions were informally in place before that time. The Center’s purpose is to improve the management of individual patients, clinical services, and population health using patient-reported outcomes.

The Center’s services include advising on the selection and collection of PROs, managing the integration of PROs into the EHR, and developing methods and reports for clinical use, quality assurance/improvement, and population management. UPMC has been at the national forefront of implementing this functionality with 58 clinical questionnaires in use and 273 active clinic locations live with PRO collection. Patients enter data directly into their EHR by completing questionnaires via the MyUPMC patient portal or an office-based tablet computer.

An electronic PRO implementation has many advantages, including:

  1. patient data flow instantaneously into the EHR, eliminating the need for staff time dedicated to data entry and the potential of data entry error;
  2. questionnaires can be completed in advance of a scheduled visit so the time spent filling out questionnaires in the office is reduced;
  3. providers can trend patient responses to see improvements or declining health during treatment;
  4. these data fulfill requirements from regulatory agencies and payers, such as measures required before and after a procedure or surgery; and
  5. remote administration of questionnaires via the MyUPMC patient portal allows collection of follow-up questionnaires outside of regular office visits at predetermined intervals.

The UPMC orthopaedic service line was an early adopter for the electronic collection of PROs in the spine, foot and ankle, and adult joint reconstruction subspecialties. In 2017, the department proposed to expand its collection and use of PROs to include all general orthopaedic care and its subspecialties, such as sports medicine, hand and upper extremity, orthopaedic trauma, and musculoskeletal oncology. For this study, we focused on patients presenting to foot and ankle specialists.

All patients age 18 and older who saw a foot and ankle specialist between June 2017 and September 2017 with an email recorded in the EHR were sent an invitation to complete an online survey about their orthopaedic office visit. The survey included:

  1. the Altarum Consumer Engagement (ACE) questionnaire, which measures a patient’s commitment to maintaining daily health behaviors, ability to navigate the health care system, and skill at looking up medical information;
  2. two CG-CAHPS survey subsections, which measure patient perceptions of physician communication and shared decision-making;
  3. a question assessing if the patient remembered completing PRO questions (which included a screen shot reminder), and, if they did complete the PRO questions, then a follow-up question; and
  4. asking if their doctor discussed the PRO responses with them.

The Team

The evaluation team comprised the study authors (Medical Director of UPMC Patient-Reported Outcomes; Director of Innovative Quality Care Programs, UPMC; Senior Director of Quality Analytics and Performance, UPMC) in collaboration with the UPMC Chief Quality Officer and key orthopaedic stakeholders (Vice Chairmen of Education, Orthopaedics; Vice Chair of Clinical Outcomes Research, Department of Orthopaedic Surgery).

PROs Associated with Better Top-Box Satisfaction Scores at UPMC

  Click To Enlarge.

The Metrics

Invitations were sent to 4,455 patients who had a visit with one of 63 providers; 558 people completed the survey (response rate 12.5%) of whom 150 reported not completing the PRO measures, 154 reported completing the PRO measures but not discussing them with their doctor, and 254 reported both completing the PRO measures and discussing them with their doctor.

ACE scores that measured the patient’s commitment to health and ability to look up medical information were not statistically different between groups. But ACE scores that measured the patient’s ability to navigate the health care system were statistically significantly different and slightly higher in the group that reported completing PROs and discussing them with their doctor compared to the group that reported completing PROs but not discussing them with their doctor.

The percent of CG-CAHPS scores with top box of physician communication were statistically different between groups. Greater shares of respondents who reported completing PROs and discussing them with their doctor gave top-box scores for provider communication (94%) compared to those who completed but did not discuss PROs (83%) and those who did not complete the PROs (84%). The group not completing PROs and the group who completed PROs but did not discuss them with their doctor were not statistically different.

A similar pattern was seen in CG-CAHPS shared decision-making scores. Greater shares of respondents who reported completing PROs and discussing them with their doctor gave positive scores for shared decision-making (91%) compared to those who completed but did not discuss PROs (82%) and those who did not complete the PROs (81%). The group not completing PROs and the group who completed PROs but did not discuss them with their doctor were not statistically different.

Where to Start

Success requires both the efficient collection of PROs from patients and the ability for physicians to address their responses during the visit. Based on our experience, for efficient collection of PROs, we recommend direct integration into the EHR, the ability to complete questionnaires before a visit, and tablet-based collection in waiting rooms. This enables physicians to immediately access responses and scores in the electronic record.

Next Steps

Future work will focus on educating physicians to explicitly discuss PRO results during visits.

New Call for Submissions ­to NEJM Catalyst

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
DLewis01_pullquote Opening Up to Patient Innovations Like Automated Insulin Delivery

Opening Up to Patient Innovation

Next-gen innovations — by new stakeholders — will need a next-gen regulatory system, and clinicians willing to adapt.

Karen DeSalvo - public health population health social determinants of health expert social services infrastructure

Moving from a Sickness Model of Health Care to One of Well-Being — Part 2

A public health expert discusses the scientific, technological, and financial foundations needed to modernize the social services infrastructure and address social determinants of health.

Karen DeSalvo - public health population health social determinants of health expert social services infrastructure

Moving from a Sickness Model of Health Care to One of Well-Being — Part 1

The expense of the U.S. health care system is so misaligned from what we need as a country — a healthier population. A public health expert discusses how we can move toward a model that addresses social determinants of health and promotes well-being while still growing our economy.

Different Means of Initiating and Sustaining Behavior Change

Survey Snapshot: Social Support, Whether In-Person or Virtual, Is the Key to Sustained Behavior Change

NEJM Catalyst Insights Council members seek ways to create relationships with patients — but the hurdles are high.

Different Means of Initiating and Sustaining Behavior Change

Patient Engagement Survey: What Creates Behavior Change May Not Sustain It

An NEJM Catalyst Insights Council survey finds that human interaction and social support are vital for patient behavior change and employee wellness programs.

How the EpxSubstanceUse Substance Use Disorder Mobile Health Texting Program Works - Preferred Family Healthcare Epharmix Technology Assisted Care Coordination Project Opioid Abuse

Messaging System Helps Caregivers Keep Tabs on Growing Number of Patients with Substance Use Disorder

One behavioral health provider pilots cost-effective outreach to patients with SUD that checks up on their recovery, triages needs, and facilitates communication.

Gary Kaplan Head Shot

Improving Patient Involvement in Care

The CEO of Virginia Mason on facilitating and enhancing patient participation in care.

Chou01_pullquote_Time to Start Using Evidence-Based Approaches to Patient-Provider Communication

Time to Start Using Evidence-Based Approaches to Patient Engagement

Clinicians must acknowledge that there is a dramatic difference between learning a concept about patient-provider communication and learning the actual skills for effective execution.

Nash01_pullquote - A Requiem for Value-Based Care Population Health.

A Requiem for Population Health?

Despite some fits and starts for value-based care, there is abundant evidence that the concept will continue to survive and thrive.

Steps for Deploying Nudge Unit in Health Care

Nudge Units to Improve the Delivery of Health Care

Key information and important choices are constantly being presented in health care. Yet often the frames or default options used are selected without attention to strategic goals.

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

Topics

Patient Incentives

62 Articles

Behavior Change Basics: Simplicity to Self-Compassion

Four principles and practices to keep in mind when trying to help patients lose weight,…

Opening Up to Patient Innovation

Next-gen innovations — by new stakeholders — will need a next-gen regulatory system, and clinicians…

Patient-Centered Care

218 Articles

Survey Snapshot: Seeing Progress in Transparency,…

NEJM Catalyst Insights Council members agree that despite the challenges and caveats, transparency efforts are…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now