New Marketplace

The State of the Evidence on Payment Reform

Article · July 18, 2018

Over the past decade, health plans, employers, and other health care purchasers, including states, have rushed to move away from traditional fee-for-service payment toward paying health care providers in a way that rewards them for adhering to certain processes of care, achieving certain outcomes for their patients, and providing a superior patient experience. While the payers’ ability to change provider behavior is likely to be negligible when they provide small bonuses for effective care a year after the fact, a greater focus on financial accountability may encourage providers to promote preventive care and look for ways to cut waste. Models that prompt providers to reach a financial target — such as bundled payments, shared risk, and global budgets — are most likely to do this. Though there is much more to learn, we have early insights into which elements of payment reform seem most promising.

In 2010, when Catalyst for Payment Reform (CPR) was founded, it put out an informal inquiry to three of the largest national commercial health plans to determine what proportion of their payments to doctors and hospitals were tied to performance. At that time, they estimated a range of 1% to 3%. In 2013 and 2014, CPR tracked the implementation of payment reform through a formal survey of commercial health plans representing more than 60% of the nation’s commercial covered population. In addition, in 2016 and 2017, CPR supported the Health Care Payment Learning and Action Network in its National Data Collection Effort.

Across these years of data, CPR finds sharp growth — about half of all commercial payments to doctors and hospitals now flow through value-oriented methods. However, none of these efforts have demonstrated conclusively that this increase in value-oriented payments has led to better, more affordable care.

CPR’s Literature Review

CPR scanned the literature to gauge the state of the evidence of payment reform today. Using a variety of sources, we focused on about 75 articles — both independent, academic studies and self-published announcements — on the performance of payment and delivery reform programs across the United States. To find these studies on PubMed or Google, our search terms included the names of payment or delivery models (e.g., pay-for-performance, shared savings, accountable care organization), well-known programs (e.g., the Alternative Quality Contract, the Medicare Shared Savings Program, PROMETHEUS Bundled Pay), and academic authors known to conduct relevant evaluations (e.g., Michael Chernew, Meredith Rosenthal). We also searched through an inventory of payment and delivery reforms prepared by the Duke-Margolis Evidence Hub. Lastly, we scoured the references in the published articles for additional evaluations of payment and delivery reform programs.

As expected, there are major differences between the academic studies and self-published announcements. The academic studies typically apply a rigor to evaluation that self-published announcements, which often highlight only positive outcomes, do not. Here is what we found regarding the value-oriented payment approaches.


Across the seven evaluations of pay-for-performance (P4P) programs we selected for our review, few showed statistically significant improvements in the quality of care, year over year. For example, in the Integrated Healthcare Association’s (IHA) Value-Based P4P Program, most clinical quality measures and patient experience of care composites stayed stable from 2015 to 2016. But over a longer period, these programs often show larger improvements in quality. Over 4 years in IHA’s program, physician organization performance on diabetes care measures improved — blood pressure control (<140/90 mm Hg) improved by 9.8 percentage points and medical attention to nephropathy improved by 5.6 percentage points.

In addition, a study of the Bridges to Excellence (BTE) program, which was launched in 2002 by a group of large employers in collaboration with several health plans and provider organizations, examined whether providers choosing to be recognized by BTE and therefore eligible for bonus payments performed better on a set of performance measures than providers who did not seek or receive recognition or bonuses. Using data from 2003 to 2006, researchers uncovered statistically significant differences in performance between recognized and non-recognized physicians. In particular, physicians recognized as Physician Office Link providers out-performed non-recognized providers on cervical cancer screening, mammography, and A1C testing. And, physicians recognized as Diabetes Care Link providers out-performed non-recognized providers on all four diabetes process measures assessing quality. None of these evaluations found improvements in the cost of health care.


Nonpayment programs have been shown to reduce the delivery of the service for which payers discontinue payment. We found two initiatives — the South Carolina Birth Outcomes Initiative and Healthy Texas Babies Initiative — focused on reducing early elective deliveries (EEDs), which have been shown to offer no clinical benefit to mothers and babies absent medical necessity. Both state Medicaid agencies stopped paying for EEDs and have since seen reductions. In South Carolina, as of 2016, 76% of all birthing hospitals have a rate of 0% of non-medically necessary EEDs, compared to 9.6% in 2011 — much higher than Leapfrog’s recommended rate of 5%. The rate of EEDs in Texas dropped 2.03 percentage points from 10.63% in 2011.

Growth in alternative payment models shared savings shared risk bundled payments population-based payments

  Click To Enlarge.

Shared Savings

Shared savings is quickly becoming the most prevalent payment reform; our review included more than 30 studies on shared savings programs, many of which were released by the sponsor of the program.

Generally, payers use shared savings arrangements with providers who restructure to form accountable care organizations (ACOs). The number of ACOs has grown significantly since the Affordable Care Act spurred their creation in 2010, with almost 1,000 organizations to date. Based on the articles we reviewed, providers paid through shared savings arrangements improve how they deliver care. The Medicare Shared Savings Program (MSSP), run by CMS, has consistently demonstrated high-quality scores, based on an average across four domains of ACO quality performance. In 2016, the aggregate quality score was 93.4%, up from 91% in the previous year and 86% in 2014. However, MSSP’s quality measures have been critiqued for being easily achieved and not centered on patient outcomes. In a private sector example, Intel Corporation’s Connected Care program, which also uses a shared savings payment model, has seen positive patient experience and satisfaction results and statistically significant improvements in diabetes care and depression screenings, among other clinical measures.

The literature also shows that many shared savings programs generate savings as a result of provider organizations spending below the target budget. However, because Medicare and health plans only give payouts and do not recoup money when ACOs overspend, payers most often end up with net losses — although this is not explicitly stated. While many of the ACOs participating in MSSP generated savings, Medicare saw a net loss of $39 million in 2016, about 0.05% of program costs, because it could not recoup payments from ACOs that overspent.

Shared Risk

While shared risk payment arrangements are not as common as shared savings — representing only about 1% of commercial payments in 2014 and the subject of only 11 of the more than 70 articles we reviewed — the evidence for their impact on the quality of care and health care spending is generally positive. Many payers realize net savings under this model because they are able to recoup costs from ACOs that have spent over the budget.

When ACOs realize savings, most often they stem from lower utilization of health care services, such as fewer hospital admissions and emergency department (ED) visits. For example, the Center for Medicare and Medicaid Innovation’s (CMMI) Pioneer program found that ACO participants reduced rates of hospitalizations and ED visits by 8% and 6% respectively, adding to net savings of about $115 per beneficiary. All but two of the studies we studied examined CMMI’s Pioneer program at the program level (how all Pioneer ACOs performed in aggregate) and at the individual ACO level (the performance of Partners HealthCare’s Pioneer ACO, for example). There are far fewer studies available from the private sector. However, in a review of a commercial health plan, Blue Cross Blue Shield of Massachusetts’ Alternative Quality Contract, researchers attributed reduced spending to decreases in volume of services — approximately 40% of claims savings.

Bundled Payment

Results on bundled or episode-based payment models have been mixed. CMMI’s Bundled Payments for Care Improvement (BPCI) initiatives have produced variable outcomes. However, other initiatives, such as the episode-based payment programs implemented by the Medicaid agencies in Arkansas and Tennessee, have had success in reducing unnecessary utilization and episode costs, as well as improving the quality of care for certain conditions. The Pennsylvania Employees Benefit Trust Fund, working with the Health Care Incentives Improvement Institute (HCI3) in a pilot program for total hip and knee replacements, demonstrated decreases in outpatient costs by $3,524 on average, and UnitedHealthcare’s oncology model led to a reduction of cancer episode costs for five medical groups by a combined $33 million.

The Path Forward

This transition from fee-for-service accomplishes nothing unless these reforms are working and balance the significant investments that providers, health plans, and purchasers are making to support these changes — investments such as integrated electronic health records or new staff for care support teams. Even patients may feel frustrated if these changes put barriers between patients and their providers or in any way disrupt the doctor-patient relationship.

While there is some evidence that certain reforms can work, we need better research to learn which approaches work best in what context. Many of the methodological studies of public or private programs lack an independent, rigorous review. Without meaningful, comprehensive information about the effectiveness of payment reforms, purchasers, health plans, and providers will continue to face challenges in their pursuit of higher-value health care.

New Call for Submissions ­to NEJM Catalyst


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From New Marketplace
Hospital-based ACOs Key Capabilities Are Different Based on Level of Risk

Hospital-based ACOs Face Challenges in Tracking Performance Indicators

A focus on bundled payments for specific episodes of care — versus the broad changes required in an ACO model — may be a more effective method for ACO participants to develop capabilities required to measure utilization, cost, and revenue metrics.

Medicaid Referral to Specialist by the Numbers without eConsults

Can eConsults Save Medicaid?

How eConsults unlock medical expertise for the underserved.

Transparency Can Improve Both Quality and Cost of Care

Survey Snapshot: Seeing Progress in Transparency, but the Quest for Value Continues

NEJM Catalyst Insights Council members agree that despite the challenges and caveats, transparency efforts are essential.

University of Utah Health Buzz Survey Report Cost of Care and Physician Responsibility Out-of-Pocket Costs

Buzz Survey Report: Cost of Care and Physician Responsibility

An independent NEJM Catalyst report sponsored by University of Utah Health on how practicing clinicians and health care leaders view health care costs.

Donald Berwick Melinda Buntin Patrick Conway Raymond Vara Edward Prewitt head shots IHI forum panel on health care policy

Signal or Noise? Navigating Health Care Policy — Part 2

A discussion from the Institute for Healthcare Improvement National Forum. Part 2 of a two-part series.

Transparency Initiatives Are Maturing But Have Far to Go

New Marketplace Survey: A Slow Path to Transparency for Patients

There is national momentum to empower patients with actionable health care information. But how well is the health care industry faring with this push to improve patient-facing transparency?

Donald Berwick Melinda Buntin Patrick Conway Raymond Vara Edward Prewitt head shots IHI forum panel on health care policy

Signal or Noise? Navigating Health Care Policy — Part 1

A discussion from the Institute for Healthcare Improvement National Forum. Part 1 of a two-part series.

Union Square Family Health Evolving Clinic Schedule - after integrated practice unit changes

Leveraging IPU Principles in Primary Care

How a safety-net clinic reorganized into multidisciplinary teams and restructured clinic flow to improve both efficiency and continuity of patient care.

Goals for Health Care Price Transparency and How to Measure Its Success

Defining the Goals of Health Care Price Transparency: Not Just Shopping Around

What policymakers, providers, and payers need to do to make price transparency successful.

Liljenquist01_pullquote generic drugs

Addressing Generic-Drug Market Failures — The Case for Establishing a Nonprofit Manufacturer

A consortium of hospitals and health plans is establishing a nonprofit generic-drug manufacturer to produce a steady supply of affordable essential drugs and stimulate competition in generic-drug markets.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Hospital-based ACOs Face Challenges in Tracking…

A focus on bundled payments for specific episodes of care — versus the broad changes…

Bundled Payments

51 Articles

Hospital-based ACOs Face Challenges in Tracking…

A focus on bundled payments for specific episodes of care — versus the broad changes…

Survey Snapshot: Seeing Progress in Transparency,…

NEJM Catalyst Insights Council members agree that despite the challenges and caveats, transparency efforts are…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now