New Marketplace

Hospital-based ACOs Face Challenges in Tracking Performance Indicators

Case Study · August 8, 2018

Given that accountable care organizations (ACOs) have not achieved the degree of cost reductions and quality improvements initially hoped for, we sought to better understand the underlying reasons for their limited success. Our analysis of American Hospital Association (AHA) data on hospital-based ACOs found that many organizations do not have the capability to track and share performance metrics, including financial metrics, and those that do are more likely to have taken on more financial risk. Our findings highlight the challenges of using ACOs as a mechanism to achieve financial cost savings. Bundled payments, which focus on narrower episodes of care, may involve fewer operational complexities and be better structured for financial success.


ACOs and bundled payments are leading strategies to reduce health care costs and improve quality in the United States. Despite their continued growth, including a net increase in the number of ACOs of 11% from Q1 2016 to Q1 2017, the organizations have shown mixed results in achieving cost and quality improvements. To better understand the limited success of hospital-based ACOs, we investigated their ability to track and share performance and financial metrics, and the association between these capabilities and the types of risk contracts ACOs chose to enter.


We analyzed data from the American Hospital Association’s 2015 Survey of Care Systems and Payment (the most recent version of this survey), which tracks the evolution of new delivery and payment models among hospitals. We limited our analysis to general medical and surgical hospitals (n=535), which had a response rate of 42%.

Key Observations

Hospital-based ACOs Are Distributed Across All Hospital Types

Hospital-based ACOs are most common among nonprofit hospitals and academic medical centers, although other types of hospitals, such as government and church-operated hospitals, have ACOs as well.

The ACO model was developed as a way to change the delivery of health care by using financial incentives to push for coordinated care. Our results highlight several areas of concern about ACOs’ ability to reduce health care spending.

Many Hospital-based ACOs Do Not Track and Share Performance Metrics

Fifty-two percent of respondents measure cost and efficiency indicators and share that information with all providers in the ACO; of that group, 57% track patient satisfaction, 63% track financial metrics, 76% track utilization metrics (the services their patients are using and how often), and 84% track clinical quality indicators.

Performance Measures for Hospital-based ACOs

  Click To Enlarge.

When we consider that only about half of respondents (52%) have the ability to track and share performance measures with all providers, we see that only 32% of all hospital-based ACOs are tracking and sharing financial metrics with their providers. Providers cannot know if they are, indeed, providing higher-quality or higher-value care if they are not aware of the financial implications of the care they are providing for their patients. Previous studies have shown that when providers are aware of the costs associated with the care they are providing in real time, they choose less costly options.

Not All ACOs Can Track Patient-Level Services and Costs

As an ACO, a provider organization takes on financial risk by committing to spend a certain amount of money on a specified patient population per year. To responsibly take on this risk, ACOs should know the baseline services their patients are using and how much these services cost. Then, they can make systematic changes to improve care coordination and reduce unnecessary utilization, thereby reducing costs and participating in shared savings.

However, our analysis showed that 59% of ACOs have information systems to track utilization of services and 31% have a “process to conduct ongoing monitoring of services rendered and the cost for those services compared to the revenue received.” Without an understanding of which services their patients are using, much less the cost versus revenue of those services, it is unclear how ACOs can make changes to improve care coordination and avoid unnecessary utilization.

Proportion of Hospital-based ACOs with Capabilities to Manage Risk Effectively

  Click To Enlarge.

Many ACOs do not know until the end of a calendar year which patients are actually attributed to their ACO. This process of retrospective patient attribution may explain why only 41% of hospital-based ACOs are able to verify patient eligibility and benefits. At the same time, electronic health record systems are not necessarily set up to keep track of and report patient utilization, and even fewer track costs to the system. Challenges with health information exchange between providers compound the issue. Some ACOs have started working with multiple stakeholders — including private insurance companies, Medicare, and Medicaid — to improve tracking of information (e.g., via claims data), and this type of information exchange may be a key element for ACOs’ success.

ACOs That Take on More Risk Generally Have Better Data-Tracking Capabilities

ACOs take on varying levels of financial risk, ranging from one-sided risk contracts (where the ACO receives a share of achieved savings below its cost target, but is not penalized for exceeding the target) to global payment models (where ACOs receive a per-member per-month payment to provide all care for their patients). The majority of hospital-based ACOs, 68%, are in one-sided risk models, taking on the least financial risk. We found that a greater percentage of ACOs that take on more financial risk track utilization and financial metrics.

Among ACOs in one-sided risk arrangements, 45% report that they are tracking cost and efficiency indicators, compared to 63% of those in a two-sided risk model, and 79% of those in a global payment model. Similarly, just 28% of ACOs in one-sided contracts are able to track the cost and revenue associated with services rendered compared to 37% in two-sided and 57% in global payment contracts.

Hospital-based ACOs Key Capabilities Are Different Based on Level of Risk

  Click To Enlarge.

Given that many hospital-based ACOs do not have adequate tools to track and manage cost and quality, it is not surprising that those ACOs that have taken on more risk are better-equipped to manage patient-level costs and decisions that impact those costs, such as referral patterns.

Health Care Technology Remains a Barrier to Success

Historically, ACOs tracked performance metrics (clinical quality, patient satisfaction, and financial measures) to report these metrics annually. The Centers for Medicare and Medicaid, for example, requested aggregate information, rather than information at the level of the individual provider; therefore, many systems were set up to report information at the facility level, not the physician level.

However, physicians need individual-level performance metrics and data to know which changes to make in their prescribing and referral patterns. Facility-level data does not give providers actionable information; first, it is not clear which providers are performing well and which need to improve, and second, it allows for a diffusion of responsibility across the system rather than requiring providers or departments to take responsibility for their performance.

ACOs are improving their data analytics capabilities to address this issue, but it is still a challenge. Although EHRs are increasing in adoption and improving capabilities, providers express major concerns about interoperability and health information exchange. These issues are compounded in an ACO, where providers need to know what care their patients are using across potentially multiple health care systems. Many ACOs do not have this capability initially — in fact, 37% of ACOs have a third-party management partner, and of those, 94% have data services provided by that partner.

Are ACOs Taking the Right Approach?

Recent reports continue to show only modest financial successes of ACOs and highlight similar operational and technological challenges to those seen in our results. We believe these challenges remain significant for hospital-based ACOs and it may be easier for hospital systems to implement other types of value-based care — such as bundled payments, which focus on a specific episode of care as the fundamental cost-accounting unit. Rather than attempting to keep track of all care patients receive over time, hospitals can focus on managing defined episodes of care.

Given the specificity of one episode of care with a bundled payment, versus a year of care in an ACO, utilization and cost should be easier to track. Bundled payments are also less daunting for institutions to adopt: Hospitals can decide to contract for bundled payments in departments where they are already performing well. Then they can learn how to succeed with bundled payments in one department and pass on those best practices to others.

A Targeted Approach

ACOs are a broad-stroke model for changing how health care is delivered. Analysis of the AHA survey results highlights the reasons that a broad tool is challenging: It is difficult to get data at the level of granularity required to resonate with individuals, and primary care physicians have limited control over the wide range of care their patients receive over the course of a year. A more targeted tool, like bundled payments, where one defined episode of care can be studied, measured, and perfected, may be a better answer and could even be used within an ACO to distribute capitated payments.

New call for submissions ­to NEJM Catalyst

Now inviting longform articles


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From New Marketplace
Elements of a Sustainable Complex Care Management Contract

Sustainable Financing for Complex Care Management Is Critical to a Value-Driven Health Care System

Care management should be payer-agnostic at its core.

Comparison of Certain Model Features in Blue Cross NC Blue Premier vs Next-Generation ACO Model vs BCBSMA Alternative Quality Contract

Engineering a Rapid Shift to Value-Based Payment in North Carolina: Goals and Challenges for a Commercial ACO Program

We believe North Carolina can be a model for the nation.

Small Molecule Drugs Facing Generic Competition - Orphan and Non-Orphan Drugs - Orphan Drug Act

It’s Time to Reform the Orphan Drug Act

Three proposals for improving the law to reflect 21st-century drug development practices.

Three-Part Pricing of PCSK9 Inhibitors

A New Model for Pricing Drugs of Uncertain Efficacy

Are we paying too much for new drugs before we know how well they work? This innovative pricing model proposes postponing major rewards until efficacy is established — which could help both patients and payers while still paying back investments on the most effective drugs.

what does quality measurement in health care mean

Buzz Survey Report: Addressing the Problems of Quality Measurement

An independent NEJM Catalyst report sponsored by University of Utah Health on patient involvement in quality measurement.

Average HOOS and Average KOOS for patients undergoing hip and knee replacement at CJRI

Building a “Hospital-within-Hospital” Model for Joint Replacements

The Connecticut Joint Replacement Institute has demonstrated that formerly competing independent providers can unite on a common vision to yield drastic improvements in quality, safety, and costs.

Discharge Rates and Follow-Up Internval Dashboard for One Provider at MGH Dermatology

A Successful Pilot to Improve Access by Adjusting Discharge and Follow-Up Rates

Actionable data and modest financial incentives can help motivate clinicians to adjust their behavior around scheduling follow-up appointments.

Cautious Optimism That Value-Based Reimbursement Will Become Primary Revenue Model

Survey Snapshot: What Would Accelerate the Adoption of Value-Based Care?

NEJM Catalyst Insights Council members weigh in on the barriers and path forward to value-based health care.

Strongwater08_pullquote primary care value proposition and disruptive innovation

The Evolution of Primary Care: Embracing Innovation While Protecting the Core Value

Primary care must leverage disruptive innovations to ensure that patients receive first-access, comprehensive, coordinated, continuous care that is woven into a seamlessly integrated system.

Berns01_pullquote nephrologists dialysis facility joint venture conflicts of interest

Dialysis-Facility Joint-Venture Ownership — Hidden Conflicts of Interest

Despite potential benefits, joint ventures between nephrologists and dialysis companies raise legal and ethical concerns because of participants’ conflicts of interest and lack of transparency.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Value Based Care

199 Articles

Curbing Health Care Spending: The Provider’s…

Health care costs have historically grown at about 2% faster than income in the United…

Medicare and Medicaid

122 Articles

A Collaborative Model to Expand Medicaid…

How managing the benefit coverage expansion for the treatment of HCV in New Mexico was…

Vertical Integration and Bold Experimentation

Four points on improving value in health care via vertical integration and aggressive experimentation.

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now