New Marketplace

Hospital-based ACOs Face Challenges in Tracking Performance Indicators

Case Study · August 8, 2018

Given that accountable care organizations (ACOs) have not achieved the degree of cost reductions and quality improvements initially hoped for, we sought to better understand the underlying reasons for their limited success. Our analysis of American Hospital Association (AHA) data on hospital-based ACOs found that many organizations do not have the capability to track and share performance metrics, including financial metrics, and those that do are more likely to have taken on more financial risk. Our findings highlight the challenges of using ACOs as a mechanism to achieve financial cost savings. Bundled payments, which focus on narrower episodes of care, may involve fewer operational complexities and be better structured for financial success.


ACOs and bundled payments are leading strategies to reduce health care costs and improve quality in the United States. Despite their continued growth, including a net increase in the number of ACOs of 11% from Q1 2016 to Q1 2017, the organizations have shown mixed results in achieving cost and quality improvements. To better understand the limited success of hospital-based ACOs, we investigated their ability to track and share performance and financial metrics, and the association between these capabilities and the types of risk contracts ACOs chose to enter.


We analyzed data from the American Hospital Association’s 2015 Survey of Care Systems and Payment (the most recent version of this survey), which tracks the evolution of new delivery and payment models among hospitals. We limited our analysis to general medical and surgical hospitals (n=535), which had a response rate of 42%.

Key Observations

Hospital-based ACOs Are Distributed Across All Hospital Types

Hospital-based ACOs are most common among nonprofit hospitals and academic medical centers, although other types of hospitals, such as government and church-operated hospitals, have ACOs as well.

The ACO model was developed as a way to change the delivery of health care by using financial incentives to push for coordinated care. Our results highlight several areas of concern about ACOs’ ability to reduce health care spending.

Many Hospital-based ACOs Do Not Track and Share Performance Metrics

Fifty-two percent of respondents measure cost and efficiency indicators and share that information with all providers in the ACO; of that group, 57% track patient satisfaction, 63% track financial metrics, 76% track utilization metrics (the services their patients are using and how often), and 84% track clinical quality indicators.

Performance Measures for Hospital-based ACOs

  Click To Enlarge.

When we consider that only about half of respondents (52%) have the ability to track and share performance measures with all providers, we see that only 32% of all hospital-based ACOs are tracking and sharing financial metrics with their providers. Providers cannot know if they are, indeed, providing higher-quality or higher-value care if they are not aware of the financial implications of the care they are providing for their patients. Previous studies have shown that when providers are aware of the costs associated with the care they are providing in real time, they choose less costly options.

Not All ACOs Can Track Patient-Level Services and Costs

As an ACO, a provider organization takes on financial risk by committing to spend a certain amount of money on a specified patient population per year. To responsibly take on this risk, ACOs should know the baseline services their patients are using and how much these services cost. Then, they can make systematic changes to improve care coordination and reduce unnecessary utilization, thereby reducing costs and participating in shared savings.

However, our analysis showed that 59% of ACOs have information systems to track utilization of services and 31% have a “process to conduct ongoing monitoring of services rendered and the cost for those services compared to the revenue received.” Without an understanding of which services their patients are using, much less the cost versus revenue of those services, it is unclear how ACOs can make changes to improve care coordination and avoid unnecessary utilization.

Proportion of Hospital-based ACOs with Capabilities to Manage Risk Effectively

  Click To Enlarge.

Many ACOs do not know until the end of a calendar year which patients are actually attributed to their ACO. This process of retrospective patient attribution may explain why only 41% of hospital-based ACOs are able to verify patient eligibility and benefits. At the same time, electronic health record systems are not necessarily set up to keep track of and report patient utilization, and even fewer track costs to the system. Challenges with health information exchange between providers compound the issue. Some ACOs have started working with multiple stakeholders — including private insurance companies, Medicare, and Medicaid — to improve tracking of information (e.g., via claims data), and this type of information exchange may be a key element for ACOs’ success.

ACOs That Take on More Risk Generally Have Better Data-Tracking Capabilities

ACOs take on varying levels of financial risk, ranging from one-sided risk contracts (where the ACO receives a share of achieved savings below its cost target, but is not penalized for exceeding the target) to global payment models (where ACOs receive a per-member per-month payment to provide all care for their patients). The majority of hospital-based ACOs, 68%, are in one-sided risk models, taking on the least financial risk. We found that a greater percentage of ACOs that take on more financial risk track utilization and financial metrics.

Among ACOs in one-sided risk arrangements, 45% report that they are tracking cost and efficiency indicators, compared to 63% of those in a two-sided risk model, and 79% of those in a global payment model. Similarly, just 28% of ACOs in one-sided contracts are able to track the cost and revenue associated with services rendered compared to 37% in two-sided and 57% in global payment contracts.

Hospital-based ACOs Key Capabilities Are Different Based on Level of Risk

  Click To Enlarge.

Given that many hospital-based ACOs do not have adequate tools to track and manage cost and quality, it is not surprising that those ACOs that have taken on more risk are better-equipped to manage patient-level costs and decisions that impact those costs, such as referral patterns.

Health Care Technology Remains a Barrier to Success

Historically, ACOs tracked performance metrics (clinical quality, patient satisfaction, and financial measures) to report these metrics annually. The Centers for Medicare and Medicaid, for example, requested aggregate information, rather than information at the level of the individual provider; therefore, many systems were set up to report information at the facility level, not the physician level.

However, physicians need individual-level performance metrics and data to know which changes to make in their prescribing and referral patterns. Facility-level data does not give providers actionable information; first, it is not clear which providers are performing well and which need to improve, and second, it allows for a diffusion of responsibility across the system rather than requiring providers or departments to take responsibility for their performance.

ACOs are improving their data analytics capabilities to address this issue, but it is still a challenge. Although EHRs are increasing in adoption and improving capabilities, providers express major concerns about interoperability and health information exchange. These issues are compounded in an ACO, where providers need to know what care their patients are using across potentially multiple health care systems. Many ACOs do not have this capability initially — in fact, 37% of ACOs have a third-party management partner, and of those, 94% have data services provided by that partner.

Are ACOs Taking the Right Approach?

Recent reports continue to show only modest financial successes of ACOs and highlight similar operational and technological challenges to those seen in our results. We believe these challenges remain significant for hospital-based ACOs and it may be easier for hospital systems to implement other types of value-based care — such as bundled payments, which focus on a specific episode of care as the fundamental cost-accounting unit. Rather than attempting to keep track of all care patients receive over time, hospitals can focus on managing defined episodes of care.

Given the specificity of one episode of care with a bundled payment, versus a year of care in an ACO, utilization and cost should be easier to track. Bundled payments are also less daunting for institutions to adopt: Hospitals can decide to contract for bundled payments in departments where they are already performing well. Then they can learn how to succeed with bundled payments in one department and pass on those best practices to others.

A Targeted Approach

ACOs are a broad-stroke model for changing how health care is delivered. Analysis of the AHA survey results highlights the reasons that a broad tool is challenging: It is difficult to get data at the level of granularity required to resonate with individuals, and primary care physicians have limited control over the wide range of care their patients receive over the course of a year. A more targeted tool, like bundled payments, where one defined episode of care can be studied, measured, and perfected, may be a better answer and could even be used within an ACO to distribute capitated payments.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From New Marketplace
Value-Based Pricing Would Lower Drug Costs and Boost Quality

New Marketplace Buzz Survey: Who’s to Blame for Drug Pricing, and How to Fix It

This survey of NEJM Catalyst Insights Council members shows strong opinions about the impact of high prices, who’s to blame, and how to fix the problem.

A Look at the Four Pillars of Primary Care

Pay for Relationship: A Novel Solution to the Primary Care Crisis

What society should and can pay for is care that enables relationships between patients and providers.

Examples of Stages of AI Technology Development and Diffusion

How Artificial Intelligence Is Changing Health Care Delivery

The development of intelligent machines holds great promise for making health care delivery more accurate, efficient, and accessible, but challenges remain for incorporating AI technology into clinical and administrative settings.

Recommendations to Resolve Information Asymmetry at the Strategic Level

Information Asymmetry: The Untapped Value of the Patient

The knowledge and preferences that patients could — and should — share with clinicians would restore balance to point-of-care interactions, leading to better outcomes and enhanced value.

Key Components for Health Care Systems to Address Patient Affordability

The Next Frontier in Reducing Costs of Care: Patient Affordability

To create meaningful point-of-care guidance so that patients can make informed medical and financial decisions, health system leaders and policymakers can develop interventions to address four major components of a proposed patient affordability scale.

Direct-to-Consumer Telemedicine Is the Biggest Coming Threat to Traditional Health Care Organizations

Survey Snapshot: Mega-Mergers and Telemedicine Accelerate Convenient Care Growth

NEJM Catalyst Insights Council members detail how providers are looking to direct-to-consumer telemedicine and partnerships to meet the differing needs of their patient populations.

Opelka01_pullquote - ACS IPU team-based surgical care bundles playbook

Developing a Playbook for IPU-based Surgical Care and New Payment Models

The complexity associated with most surgery lends itself to the integrated practice unit structure, with its focus on the care team and value-based payment.

Convenient Care Has Been Good Overall for the Health Care Industry

New Marketplace Survey: Convenient Care — Opportunity, Threat, or Both?

A survey of the NEJM Catalyst Insights Council shows conflicting views about both the value of convenient care and what respondents’ organizations should do.

Payer-Provider Partnerships Produce Better Quality Outcomes 3 - community health plan - physician partnership

New Research Shows How Payer-Provider Partnerships Can Accelerate Adoption of Evidence-Based Care

Five best practices that are replicable and scalable are facilitating improved clinical and financial outcomes today.

30-Day Mortality Rates at Non-Teaching and Major Teaching Hospitals 2013-2014 - value-based care at academic medical centers

What Value-Based Payment Means for Academic Medical Centers

Academic medical centers must become as dedicated to advancing operational and clinical efficiency as they have been to advancing the science of medicine.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Platforming Health Care to Transform Care…

Health care leaders need to focus less on ownership and control of the delivery process,…

New Marketplace Buzz Survey: Who’s to…

This survey of NEJM Catalyst Insights Council members shows strong opinions about the impact of…

Build vs. Buy: What Should Health…

The consolidation craze continues, but vertical integration has yet to demonstrate real progress toward the…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now