New Marketplace

Hospital-based ACOs Face Challenges in Tracking Performance Indicators

Case Study · August 8, 2018

Given that accountable care organizations (ACOs) have not achieved the degree of cost reductions and quality improvements initially hoped for, we sought to better understand the underlying reasons for their limited success. Our analysis of American Hospital Association (AHA) data on hospital-based ACOs found that many organizations do not have the capability to track and share performance metrics, including financial metrics, and those that do are more likely to have taken on more financial risk. Our findings highlight the challenges of using ACOs as a mechanism to achieve financial cost savings. Bundled payments, which focus on narrower episodes of care, may involve fewer operational complexities and be better structured for financial success.


ACOs and bundled payments are leading strategies to reduce health care costs and improve quality in the United States. Despite their continued growth, including a net increase in the number of ACOs of 11% from Q1 2016 to Q1 2017, the organizations have shown mixed results in achieving cost and quality improvements. To better understand the limited success of hospital-based ACOs, we investigated their ability to track and share performance and financial metrics, and the association between these capabilities and the types of risk contracts ACOs chose to enter.


We analyzed data from the American Hospital Association’s 2015 Survey of Care Systems and Payment (the most recent version of this survey), which tracks the evolution of new delivery and payment models among hospitals. We limited our analysis to general medical and surgical hospitals (n=535), which had a response rate of 42%.

Key Observations

Hospital-based ACOs Are Distributed Across All Hospital Types

Hospital-based ACOs are most common among nonprofit hospitals and academic medical centers, although other types of hospitals, such as government and church-operated hospitals, have ACOs as well.

The ACO model was developed as a way to change the delivery of health care by using financial incentives to push for coordinated care. Our results highlight several areas of concern about ACOs’ ability to reduce health care spending.

Many Hospital-based ACOs Do Not Track and Share Performance Metrics

Fifty-two percent of respondents measure cost and efficiency indicators and share that information with all providers in the ACO; of that group, 57% track patient satisfaction, 63% track financial metrics, 76% track utilization metrics (the services their patients are using and how often), and 84% track clinical quality indicators.

Performance Measures for Hospital-based ACOs

  Click To Enlarge.

When we consider that only about half of respondents (52%) have the ability to track and share performance measures with all providers, we see that only 32% of all hospital-based ACOs are tracking and sharing financial metrics with their providers. Providers cannot know if they are, indeed, providing higher-quality or higher-value care if they are not aware of the financial implications of the care they are providing for their patients. Previous studies have shown that when providers are aware of the costs associated with the care they are providing in real time, they choose less costly options.

Not All ACOs Can Track Patient-Level Services and Costs

As an ACO, a provider organization takes on financial risk by committing to spend a certain amount of money on a specified patient population per year. To responsibly take on this risk, ACOs should know the baseline services their patients are using and how much these services cost. Then, they can make systematic changes to improve care coordination and reduce unnecessary utilization, thereby reducing costs and participating in shared savings.

However, our analysis showed that 59% of ACOs have information systems to track utilization of services and 31% have a “process to conduct ongoing monitoring of services rendered and the cost for those services compared to the revenue received.” Without an understanding of which services their patients are using, much less the cost versus revenue of those services, it is unclear how ACOs can make changes to improve care coordination and avoid unnecessary utilization.

Proportion of Hospital-based ACOs with Capabilities to Manage Risk Effectively

  Click To Enlarge.

Many ACOs do not know until the end of a calendar year which patients are actually attributed to their ACO. This process of retrospective patient attribution may explain why only 41% of hospital-based ACOs are able to verify patient eligibility and benefits. At the same time, electronic health record systems are not necessarily set up to keep track of and report patient utilization, and even fewer track costs to the system. Challenges with health information exchange between providers compound the issue. Some ACOs have started working with multiple stakeholders — including private insurance companies, Medicare, and Medicaid — to improve tracking of information (e.g., via claims data), and this type of information exchange may be a key element for ACOs’ success.

ACOs That Take on More Risk Generally Have Better Data-Tracking Capabilities

ACOs take on varying levels of financial risk, ranging from one-sided risk contracts (where the ACO receives a share of achieved savings below its cost target, but is not penalized for exceeding the target) to global payment models (where ACOs receive a per-member per-month payment to provide all care for their patients). The majority of hospital-based ACOs, 68%, are in one-sided risk models, taking on the least financial risk. We found that a greater percentage of ACOs that take on more financial risk track utilization and financial metrics.

Among ACOs in one-sided risk arrangements, 45% report that they are tracking cost and efficiency indicators, compared to 63% of those in a two-sided risk model, and 79% of those in a global payment model. Similarly, just 28% of ACOs in one-sided contracts are able to track the cost and revenue associated with services rendered compared to 37% in two-sided and 57% in global payment contracts.

Hospital-based ACOs Key Capabilities Are Different Based on Level of Risk

  Click To Enlarge.

Given that many hospital-based ACOs do not have adequate tools to track and manage cost and quality, it is not surprising that those ACOs that have taken on more risk are better-equipped to manage patient-level costs and decisions that impact those costs, such as referral patterns.

Health Care Technology Remains a Barrier to Success

Historically, ACOs tracked performance metrics (clinical quality, patient satisfaction, and financial measures) to report these metrics annually. The Centers for Medicare and Medicaid, for example, requested aggregate information, rather than information at the level of the individual provider; therefore, many systems were set up to report information at the facility level, not the physician level.

However, physicians need individual-level performance metrics and data to know which changes to make in their prescribing and referral patterns. Facility-level data does not give providers actionable information; first, it is not clear which providers are performing well and which need to improve, and second, it allows for a diffusion of responsibility across the system rather than requiring providers or departments to take responsibility for their performance.

ACOs are improving their data analytics capabilities to address this issue, but it is still a challenge. Although EHRs are increasing in adoption and improving capabilities, providers express major concerns about interoperability and health information exchange. These issues are compounded in an ACO, where providers need to know what care their patients are using across potentially multiple health care systems. Many ACOs do not have this capability initially — in fact, 37% of ACOs have a third-party management partner, and of those, 94% have data services provided by that partner.

Are ACOs Taking the Right Approach?

Recent reports continue to show only modest financial successes of ACOs and highlight similar operational and technological challenges to those seen in our results. We believe these challenges remain significant for hospital-based ACOs and it may be easier for hospital systems to implement other types of value-based care — such as bundled payments, which focus on a specific episode of care as the fundamental cost-accounting unit. Rather than attempting to keep track of all care patients receive over time, hospitals can focus on managing defined episodes of care.

Given the specificity of one episode of care with a bundled payment, versus a year of care in an ACO, utilization and cost should be easier to track. Bundled payments are also less daunting for institutions to adopt: Hospitals can decide to contract for bundled payments in departments where they are already performing well. Then they can learn how to succeed with bundled payments in one department and pass on those best practices to others.

A Targeted Approach

ACOs are a broad-stroke model for changing how health care is delivered. Analysis of the AHA survey results highlights the reasons that a broad tool is challenging: It is difficult to get data at the level of granularity required to resonate with individuals, and primary care physicians have limited control over the wide range of care their patients receive over the course of a year. A more targeted tool, like bundled payments, where one defined episode of care can be studied, measured, and perfected, may be a better answer and could even be used within an ACO to distribute capitated payments.

New call for submissions ­to NEJM Catalyst

Now inviting longform articles


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From New Marketplace
Washington State Health Care Authority Center of Excellence joint replacement elements and outcomes of program produces strong patient experience scores

Improving Care by Redesigning Payment

A state-run center of excellence uses benefit design to improve outcomes while controlling cost.

Alaigh03_pullquote aco and bundled payment co-exist

ACOs and Bundled Payments: How the Two Can and Should Coexist

The integration of primary care and specialty care physicians through adoption of both payment models will lead to synergistic value.

Pham03_pullquote payment and delivery-system reform

Payment and Delivery-System Reform — The Next Phase

To truly redesign a system, one has to take a holistic approach and move multiple levers in concert, rather than fiddling with individual factors serially and hoping for a coordinated effect.

A Connected Health Data and Service Ecoystem - digital health platform convener

Convening a Digitally Enabled Ecosystem to Address the Chronic Disease Burden of an Underserved Community

Disrupted by technologies, health care needs ecosystem-based solutions for integration and equitable access.

Shachar01_pullquote value-based care fee-for-service fraud and abuse laws

Are Fraud and Abuse Laws Stifling Value-Based Care?

To realize the true potential of value-based care, we must update the underlying assumptions in our fraud and abuse regulatory system.

Porter02_SBUs and IPUs pullquote1

What 21st Century Health Care Should Learn from 20th Century Business

Just as there is no such thing as good marketing or good manufacturing, there is no such thing as good primary care — it completely depends on how that care is tailored to meet the needs of patients with a particular medical condition.

Missouri Show Me Health Plans Cost Calculator for Health Insurance Enrollment screen shot - health insurance education

Facilitating Health Insurance Enrollment in an Uncertain Environment

Lessons learned from five years of open enrollment in Missouri.

Birkmeyer02_pullquote - national physician practices NPPs

Aggregators, Innovators, and the Resurgence of National Physician Practices

A perspective on the growth of NPPs and the ideological divide between for-profit and nonprofit providers.

Medicaid Referral to Specialist by the Numbers without eConsults

Can eConsults Save Medicaid?

How eConsults unlock medical expertise for the underserved.

Transparency Can Improve Both Quality and Cost of Care

Survey Snapshot: Seeing Progress in Transparency, but the Quest for Value Continues

NEJM Catalyst Insights Council members agree that despite the challenges and caveats, transparency efforts are essential.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Payment and Delivery-System Reform — The…

To truly redesign a system, one has to take a holistic approach and move multiple…

Bundled Payments

53 Articles

Improving Care by Redesigning Payment

A state-run center of excellence uses benefit design to improve outcomes while controlling cost.

Improving Care by Redesigning Payment

A state-run center of excellence uses benefit design to improve outcomes while controlling cost.

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now