Patient Engagement

A/B Testing Health Behavior

Article · March 10, 2017

Medicine has been accused of implementing new interventions without a strong underlying evidence base, and that impulse for quick implementation is understandable, says Scott Halpern, Deputy Director for the University of Pennsylvania’s Center for Health Incentives and Behavioral Economics. “We have sick people. We need to do something. We can’t sit idly by,” he says. “But I wonder if that instinct to not just stand there — to do something — might have some unintended consequences, crowding out the potential for greater innovation that has better evidence to support it.” What can we learn from other industries that have grappled with this tension?

A decade ago, there was almost no A/B testing in the corporate sector, notes Harvard economist David Laibson. “The CEO or someone else had a great idea and they implemented it, and then they had anecdotes about how great it was — and that was the way you made progress,” he says, adding, “a lot of that was a huge waste of money.”

This attitude first started changing in Silicon Valley, where companies became experts at treatment and control. Today, they’ll immediately start randomizing a new idea and maybe within a day have enough data to know whether to proceed with it. “We can’t do that in health care, of course, because we don’t have the control and the systems are complicated,” says Laibson. “It’s not that we should aspire to 24-hour experiments, but we should aspire to experiments, to actually randomizing and figuring out what works before we roll things out nationally and embrace them as best practice.”

“If we take the treatment-and-control approach, we’re going to make a lot more progress quickly despite the fact that it’s frustrating,” says Laibson, “because when you commit to running randomized control trials, it slows down the legislation, it slows down the national adoption, because you’ve got to wait a year for those results to come in. But that’s a year that’s worth waiting.”

Those who are working to add more value to the health system may also need some training to help them understand what the best new design might be, adds Dartmouth College marketing expert Punam Keller. She has worked with the Centers for Disease Control and Prevention on HealthCommWorks, which uses an empirical algorithm to help you tailor your messages to your audience’s needs. The website also offers ProofWorks, which is “basically like PhD 101,” says Keller. It includes information on good design, control, random assignment, sample sizes, metrics validation, intervention and testing time, and outcome variables. “Those kinds of things are important if we are to improve the rigor of the data that we have in order to improve our collective understanding,” she says.

Watch the video:

From the NEJM Catalyst event Patient Engagement: Behavioral Strategies for Better Health at the University of Pennsylvania, February 25, 2016.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
PROMs Usage Is Limited but Growing

Patient Engagement Buzz Survey: PROMs Use Is Growing, but Implementation Takes Effort

While clinical outcomes lend themselves to measurement, quantifying how an individual patient experiences symptoms or a loss of function can be more challenging. One promising approach is the use of Patient-Reported Outcome Measures (PROMs).

Taxonomy of the Patient Voice Table

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care, the terms used to describe this goal are proliferating — and muddying the discourse. This taxonomy attempts to classify some of these terms and make some distinctions.

Agarwal01_pullquote handcrafting the patient experience

Handcrafting the Patient Experience

Health care organizations can take cues from consumer-facing companies like Airbnb to creatively insert convenience and surprise into patient encounters.

Screenshot of Tidepool daily diabetes data

A Taxonomy to Engage Patients: Objectives, Design, and Patient Activation

Health information initiatives will succeed only if they focus on patients’ motivation to engage and reflect the type of engagement they seek.

Ghafur01_pullquote -digital health health care consumer patient experience

Engaging Patients Using Digital Technology — Learning from Other Industries

Providers can benefit patients and disrupt health care by learning from the experience of other industries.

Mission Hospital Virtual Sitter - Drawing of the Virtual Bed Zone and Rails

Reducing Inpatient Falls and Injury Rates by Integrating New Technology with Workflow Redesign

How Mission Hospital scaled a virtual sitter pilot and reduced unassisted falls by 44% and fall-related injuries by 40%.

Health Care Providers Should Incentivize Patients

Survey Snapshot: Patient Financial Incentives — There Are No Quick Fixes

The NEJM Catalyst Insights Council agrees that while financial incentives are a common strategy to engage patients in healthy behaviors, they are not necessarily effective.

Support of Family and Friends Is More Effective Than Clinician Support in Realizing Health Goals - From the Patient Engagement Insights Report: Why No Single Health Incentive Works.

Patient Engagement Survey: Why No Single Health Incentive Works

Initiatives to improve patient engagement come in a variety of forms. While insurers, employers, and health care providers are all involved in using financial incentives and penalties for engagement efforts, improvement in health outcomes has been elusive. Achieving that ultimate goal will usually require a combination of financial and social approaches.

Health Systems Attending the Nudge Units in Health Care Symposium - Penn Medicine

Key Insights on Launching a Nudge Unit within a Health Care System

Leaders are finding that making higher-value choices easier through subtle changes to choice architecture can have an outsized impact on medical decision-making.

Barriers to Providing an Oustanding Patient Experience

Buzz Survey Report: Patient Experience

An independent NEJM Catalyst report sponsored by University of Utah Health on barriers to achieving an excellent patient experience.

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

Topics

Patient Incentives

75 Articles

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care,…

Ripe for Disruption: Why and How…

For big tech companies like Amazon, Apple, and Google, the health care sector looks ripe…

Patients As Customers

141 Articles

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now