Defining “Directionality” in Quality for the Board

Case Study · December 21, 2017

The Board of Directors at Mission Health in Asheville, North Carolina, faced a challenge. First, it wanted to look beyond traditional inpatient quality measures such as mortality, infection rates, and readmissions to a broader, more robust “directional” indicator of quality. At the same time, the Board wanted a clear answer to simple questions related to Mission’s journey to zero preventable harms (e.g., “How are we doing?” and “Are we getting better or worse?”).
In response, Mission’s Quality Team adapted the American Hospital Association/Health Research & Educational Trust’s (AHA/HRET’s) “Harm Across the Board” metric as a way of achieving internal consistency across Mission’s diverse set of seven hospitals, with the indices for the individual hospitals being rolled up into an enterprise-level index that is easily tracked by the Board and has drill-down capabilities for management.

Key Takeaways

  1. Consistent visibility into the directionality of an organization’s core performance metrics, such as quality and safety in health care, is a critical Board need.

  2. To facilitate meaningful Board conversations about directional movements in quality and safety across an enterprise, an enterprise-wide index must be granular enough to explain which component metrics are moving the index up or down.

  3. Data governance (specifically, ensuring uniformity of metric definition, measurement, and timing of reporting) is critical and requires constant vigilance to maintain.

  4. While we can now reliably track inpatient preventable harms, our understanding of preventable harms in the ambulatory care setting is much earlier in development and remains a critically important frontier.

The Challenge

Like all boards, Mission’s governance body struggled with a way to assess the entire organization’s quality and safety performance at a glance. While the Board’s Quality Committee had regularly reviewed detailed quality and safety metrics over the years, there was no way to summarize those metrics across the system and easily see whether the entire organization was moving in the right or wrong direction.

This challenge was exacerbated each time a new hospital with new local metrics joined the system, largely because the metrics most relevant to the system’s flagship 763-bed tertiary center were often different from those most relevant to a critical access or rehabilitation hospital.

The Goal

We sought to create a single index metric — useful at both the system and local levels — that would include all preventable harm metrics relevant to each hospital in an internally consistent manner.

The Execution

In mid-2015, we chose Harm Across the Board, created by AHA/HRET as part of the first Partnership for Patients Hospital Engagement Network initiative, as our roll-up index. This index includes a collection of preventable harm measures and provides hospitals with the ability to select and track only those that are most relevant to their scope of care. We adapted this index to represent the care provided within each Mission Health hospital and began system-level reporting from January 1, 2015, to create a consistent baseline across entities.

Some measures (e.g., readmissions and medication errors with harm) are tracked at every hospital, whereas others (e.g., measures of obstetric harm and surgical site infections) are only tracked at hospitals providing OB or surgical care. Within the index, management can follow the trend for each of these individual metrics to ensure that both attention and resources are directed appropriately. For the board, each hospital’s numerator (the monthly “n” of preventable harms) and denominator (the total number of patient days) are rolled up into a single enterprise-wide metric that is reported as the “total number of preventable harms per 1,000 patient days.” The table shows the measures that are tracked at each hospital.

Harm Across the Board - Preventable Harm Metrics Tracked at Seven Mission Health Hospitals

  Click To Enlarge.

The Team

Mission’s Quality Department led both the design and implementation phases of operationalizing the Harm Across the Board index across the health system, a process that took place over a period of several months in 2015. Working closely with the Chief Quality Officer, Mission’s Clinical Outcomes Coordinator was responsible for ensuring accurate and consistent data governance (e.g., uniformity of metric definition, measurement, and timing of reporting) and data integration. Mission’s COO, CEO, and Chair of the Quality Committee were periodically updated to ensure that the measure would meet the needs of both management and the Board. With successful validation complete, the Board began receiving bimonthly system-level Harm Across the Board updates in FY2016 and monthly updates in FY2017.

Lessons Learned

Creating a directional measure for something as broad as “all preventable harms” can seem daunting. In our experience, we found that it is more important to start tracking something and iteratively improve the measure over time than it is to focus on perfecting the measure and, in so doing, delay tracking anything. With an enterprise index as our ultimate goal, we also found it critically important to first partner with individual hospital leaders to ensure that their hospital-level indices were fully relevant and internally consistent.

This foundational work fostered buy-in and created a shared understanding of the “why” behind our development process at both the local and system levels. Each hospital regularly reports its index to its individual hospital board, tracks changes in component metrics, and reports on trends in their metrics during our monthly System Quality Council. These contributions highlight the importance of “systemness,” foster cross-hospital learning, and ultimately inform the health system Board’s conversation about the performance of the enterprise as a whole.

Finally, during development, we learned that aligning metric definitions and reporting accountability across hospitals was critically important to our ability to reliably track the enterprise index. We were fortunate to have general consistency in terms of how each component metric was tracked within each hospital. Yet it still took several months of work to tighten our data governance processes enough to reliably report an enterprise index.

Next Steps

We recognize that acute and post-acute inpatient care, while important, still only represent a subset of the total care that we deliver across Mission Health. Our teams are working together to find the most relevant ambulatory metrics to add to the current inpatient-only measures, and we plan to begin reporting an ambulatory index to our Board in January 2019. While the literature around what to include in such an ambulatory index is still developing, we are following our own advice and will start by reporting something internally consistent, tracking it regularly, and iteratively improving it over time.


Acknowledgements: The authors thank Pam Brownfield for her tireless work on continuously refining and reporting Harm Across the Board as Mission’s Clinical Outcomes Coordinator.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Leadership

Small Change, Big Consequences — Partial Medicaid Expansions under the ACA

The stakes are high in the debate over partial Medicaid expansions, with far-reaching consequences for patients, state governments, and the federal budget.

The “Stranger Effect” — A Look at Interactions Between Consultants and Care Teams Through the Lens of Social Science

Why do miscommunications between teams and consultants occur, and what can we do about it?

UCLPartners Leadership Competency Framework

Building a New Kind of Leader for an Era of Cooperation

Moving from competition to collaboration requires different leaders, and a different way to evaluate leadership potential.

How to Have a High-Performing Employed Medical Group Without a Hospital Subsidy

It’s a generally accepted view that all hospital-employed physician groups are constitutionally incapable of operating without financial support from the hospital. PinnacleHealth Medical Group found that commonsense tactics can eliminate the need for hospitals to expend precious resources to support employed physicians.

Wanted: Talented, Energetic, Creative People to Work on Difficult, Boring Problems. No Perks.

Let’s admit the inconvenient, boring truth.

Physician Coaching: Clinicians Helping Clinicians on the Things That Matter Most

A physician coach reflects on who gets better, what works, and how to create a learning culture.

Jessica Dudley and Namita Mohta head shots

Professional Fulfillment: Where We Want to Be

How the CMO of Brigham and Women’s Physicians Organization combats burnout at her institution and empowers physicians with leadership skills.

Texas Medical Center Houston Methodist Checklist to Maintain Operations During Hurricane Harvey and Future Flood Events

Lessons in Leadership: How the World’s Largest Medical Center Braced for Hurricane Harvey

Due to learnings from previous hurricanes, five success factors kept the Texas Medical Center operational during one of the worst natural disasters in recent U.S. history.

Lessons in Leadership: How the BayCare Health System Withstood Hurricane Irma

BayCare’s extensive disaster preparations were crucial in weathering the storm.

The VA Is a Quality Leader, Contrary to Many People’s Assumptions

A defense of the Veterans Health Administration, from its former Acting Under Secretary.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Physician Burnout

52 Articles

Survey Snapshot: How Do You Know…

The NEJM Catalyst Insights Council discusses strategies for clinical engagement.

Team Care

111 Articles

What AI Means for Doctors and…

Physicians must hone the “four Cs” — critical thinking, communication, collaboration, and creativity — when…

Leading Teams

171 Articles

What AI Means for Doctors and…

Physicians must hone the “four Cs” — critical thinking, communication, collaboration, and creativity — when…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now