New Marketplace

Hospital-based ACOs Face Challenges in Tracking Performance Indicators

Case Study · August 8, 2018

Given that accountable care organizations (ACOs) have not achieved the degree of cost reductions and quality improvements initially hoped for, we sought to better understand the underlying reasons for their limited success. Our analysis of American Hospital Association (AHA) data on hospital-based ACOs found that many organizations do not have the capability to track and share performance metrics, including financial metrics, and those that do are more likely to have taken on more financial risk. Our findings highlight the challenges of using ACOs as a mechanism to achieve financial cost savings. Bundled payments, which focus on narrower episodes of care, may involve fewer operational complexities and be better structured for financial success.


ACOs and bundled payments are leading strategies to reduce health care costs and improve quality in the United States. Despite their continued growth, including a net increase in the number of ACOs of 11% from Q1 2016 to Q1 2017, the organizations have shown mixed results in achieving cost and quality improvements. To better understand the limited success of hospital-based ACOs, we investigated their ability to track and share performance and financial metrics, and the association between these capabilities and the types of risk contracts ACOs chose to enter.


We analyzed data from the American Hospital Association’s 2015 Survey of Care Systems and Payment (the most recent version of this survey), which tracks the evolution of new delivery and payment models among hospitals. We limited our analysis to general medical and surgical hospitals (n=535), which had a response rate of 42%.

Key Observations

Hospital-based ACOs Are Distributed Across All Hospital Types

Hospital-based ACOs are most common among nonprofit hospitals and academic medical centers, although other types of hospitals, such as government and church-operated hospitals, have ACOs as well.

The ACO model was developed as a way to change the delivery of health care by using financial incentives to push for coordinated care. Our results highlight several areas of concern about ACOs’ ability to reduce health care spending.

Many Hospital-based ACOs Do Not Track and Share Performance Metrics

Fifty-two percent of respondents measure cost and efficiency indicators and share that information with all providers in the ACO; of that group, 57% track patient satisfaction, 63% track financial metrics, 76% track utilization metrics (the services their patients are using and how often), and 84% track clinical quality indicators.

Performance Measures for Hospital-based ACOs

  Click To Enlarge.

When we consider that only about half of respondents (52%) have the ability to track and share performance measures with all providers, we see that only 32% of all hospital-based ACOs are tracking and sharing financial metrics with their providers. Providers cannot know if they are, indeed, providing higher-quality or higher-value care if they are not aware of the financial implications of the care they are providing for their patients. Previous studies have shown that when providers are aware of the costs associated with the care they are providing in real time, they choose less costly options.

Not All ACOs Can Track Patient-Level Services and Costs

As an ACO, a provider organization takes on financial risk by committing to spend a certain amount of money on a specified patient population per year. To responsibly take on this risk, ACOs should know the baseline services their patients are using and how much these services cost. Then, they can make systematic changes to improve care coordination and reduce unnecessary utilization, thereby reducing costs and participating in shared savings.

However, our analysis showed that 59% of ACOs have information systems to track utilization of services and 31% have a “process to conduct ongoing monitoring of services rendered and the cost for those services compared to the revenue received.” Without an understanding of which services their patients are using, much less the cost versus revenue of those services, it is unclear how ACOs can make changes to improve care coordination and avoid unnecessary utilization.

Proportion of Hospital-based ACOs with Capabilities to Manage Risk Effectively

  Click To Enlarge.

Many ACOs do not know until the end of a calendar year which patients are actually attributed to their ACO. This process of retrospective patient attribution may explain why only 41% of hospital-based ACOs are able to verify patient eligibility and benefits. At the same time, electronic health record systems are not necessarily set up to keep track of and report patient utilization, and even fewer track costs to the system. Challenges with health information exchange between providers compound the issue. Some ACOs have started working with multiple stakeholders — including private insurance companies, Medicare, and Medicaid — to improve tracking of information (e.g., via claims data), and this type of information exchange may be a key element for ACOs’ success.

ACOs That Take on More Risk Generally Have Better Data-Tracking Capabilities

ACOs take on varying levels of financial risk, ranging from one-sided risk contracts (where the ACO receives a share of achieved savings below its cost target, but is not penalized for exceeding the target) to global payment models (where ACOs receive a per-member per-month payment to provide all care for their patients). The majority of hospital-based ACOs, 68%, are in one-sided risk models, taking on the least financial risk. We found that a greater percentage of ACOs that take on more financial risk track utilization and financial metrics.

Among ACOs in one-sided risk arrangements, 45% report that they are tracking cost and efficiency indicators, compared to 63% of those in a two-sided risk model, and 79% of those in a global payment model. Similarly, just 28% of ACOs in one-sided contracts are able to track the cost and revenue associated with services rendered compared to 37% in two-sided and 57% in global payment contracts.

Hospital-based ACOs Key Capabilities Are Different Based on Level of Risk

  Click To Enlarge.

Given that many hospital-based ACOs do not have adequate tools to track and manage cost and quality, it is not surprising that those ACOs that have taken on more risk are better-equipped to manage patient-level costs and decisions that impact those costs, such as referral patterns.

Health Care Technology Remains a Barrier to Success

Historically, ACOs tracked performance metrics (clinical quality, patient satisfaction, and financial measures) to report these metrics annually. The Centers for Medicare and Medicaid, for example, requested aggregate information, rather than information at the level of the individual provider; therefore, many systems were set up to report information at the facility level, not the physician level.

However, physicians need individual-level performance metrics and data to know which changes to make in their prescribing and referral patterns. Facility-level data does not give providers actionable information; first, it is not clear which providers are performing well and which need to improve, and second, it allows for a diffusion of responsibility across the system rather than requiring providers or departments to take responsibility for their performance.

ACOs are improving their data analytics capabilities to address this issue, but it is still a challenge. Although EHRs are increasing in adoption and improving capabilities, providers express major concerns about interoperability and health information exchange. These issues are compounded in an ACO, where providers need to know what care their patients are using across potentially multiple health care systems. Many ACOs do not have this capability initially — in fact, 37% of ACOs have a third-party management partner, and of those, 94% have data services provided by that partner.

Are ACOs Taking the Right Approach?

Recent reports continue to show only modest financial successes of ACOs and highlight similar operational and technological challenges to those seen in our results. We believe these challenges remain significant for hospital-based ACOs and it may be easier for hospital systems to implement other types of value-based care — such as bundled payments, which focus on a specific episode of care as the fundamental cost-accounting unit. Rather than attempting to keep track of all care patients receive over time, hospitals can focus on managing defined episodes of care.

Given the specificity of one episode of care with a bundled payment, versus a year of care in an ACO, utilization and cost should be easier to track. Bundled payments are also less daunting for institutions to adopt: Hospitals can decide to contract for bundled payments in departments where they are already performing well. Then they can learn how to succeed with bundled payments in one department and pass on those best practices to others.

A Targeted Approach

ACOs are a broad-stroke model for changing how health care is delivered. Analysis of the AHA survey results highlights the reasons that a broad tool is challenging: It is difficult to get data at the level of granularity required to resonate with individuals, and primary care physicians have limited control over the wide range of care their patients receive over the course of a year. A more targeted tool, like bundled payments, where one defined episode of care can be studied, measured, and perfected, may be a better answer and could even be used within an ACO to distribute capitated payments.

New Call for Submissions ­to NEJM Catalyst


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From New Marketplace
Medicaid Referral to Specialist by the Numbers without eConsults

Can eConsults Save Medicaid?

How eConsults unlock medical expertise for the underserved.

Transparency Can Improve Both Quality and Cost of Care

Survey Snapshot: Seeing Progress in Transparency, but the Quest for Value Continues

NEJM Catalyst Insights Council members agree that despite the challenges and caveats, transparency efforts are essential.

Growth in alternative payment models shared savings shared risk bundled payments population-based payments

The State of the Evidence on Payment Reform

Despite some successes among the many efforts to execute value-oriented payment models, the lack of rigorous independent research studies impedes robust progress.

University of Utah Health Buzz Survey Report Cost of Care and Physician Responsibility Out-of-Pocket Costs

Buzz Survey Report: Cost of Care and Physician Responsibility

An independent NEJM Catalyst report sponsored by University of Utah Health on how practicing clinicians and health care leaders view health care costs.

Donald Berwick Melinda Buntin Patrick Conway Raymond Vara Edward Prewitt head shots IHI forum panel on health care policy

Signal or Noise? Navigating Health Care Policy — Part 2

A discussion from the Institute for Healthcare Improvement National Forum. Part 2 of a two-part series.

Transparency Initiatives Are Maturing But Have Far to Go

New Marketplace Survey: A Slow Path to Transparency for Patients

There is national momentum to empower patients with actionable health care information. But how well is the health care industry faring with this push to improve patient-facing transparency?

Donald Berwick Melinda Buntin Patrick Conway Raymond Vara Edward Prewitt head shots IHI forum panel on health care policy

Signal or Noise? Navigating Health Care Policy — Part 1

A discussion from the Institute for Healthcare Improvement National Forum. Part 1 of a two-part series.

Union Square Family Health Evolving Clinic Schedule - after integrated practice unit changes

Leveraging IPU Principles in Primary Care

How a safety-net clinic reorganized into multidisciplinary teams and restructured clinic flow to improve both efficiency and continuity of patient care.

Goals for Health Care Price Transparency and How to Measure Its Success

Defining the Goals of Health Care Price Transparency: Not Just Shopping Around

What policymakers, providers, and payers need to do to make price transparency successful.

Liljenquist01_pullquote generic drugs

Addressing Generic-Drug Market Failures — The Case for Establishing a Nonprofit Manufacturer

A consortium of hospitals and health plans is establishing a nonprofit generic-drug manufacturer to produce a steady supply of affordable essential drugs and stimulate competition in generic-drug markets.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Hospital-based ACOs Face Challenges in Tracking…

A focus on bundled payments for specific episodes of care — versus the broad changes…

Bundled Payments

51 Articles

Hospital-based ACOs Face Challenges in Tracking…

A focus on bundled payments for specific episodes of care — versus the broad changes…

Survey Snapshot: Seeing Progress in Transparency,…

NEJM Catalyst Insights Council members agree that despite the challenges and caveats, transparency efforts are…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now