Patient Engagement

A/B Testing Health Behavior

Article · March 10, 2017

Medicine has been accused of implementing new interventions without a strong underlying evidence base, and that impulse for quick implementation is understandable, says Scott Halpern, Deputy Director for the University of Pennsylvania’s Center for Health Incentives and Behavioral Economics. “We have sick people. We need to do something. We can’t sit idly by,” he says. “But I wonder if that instinct to not just stand there — to do something — might have some unintended consequences, crowding out the potential for greater innovation that has better evidence to support it.” What can we learn from other industries that have grappled with this tension?

A decade ago, there was almost no A/B testing in the corporate sector, notes Harvard economist David Laibson. “The CEO or someone else had a great idea and they implemented it, and then they had anecdotes about how great it was — and that was the way you made progress,” he says, adding, “a lot of that was a huge waste of money.”

This attitude first started changing in Silicon Valley, where companies became experts at treatment and control. Today, they’ll immediately start randomizing a new idea and maybe within a day have enough data to know whether to proceed with it. “We can’t do that in health care, of course, because we don’t have the control and the systems are complicated,” says Laibson. “It’s not that we should aspire to 24-hour experiments, but we should aspire to experiments, to actually randomizing and figuring out what works before we roll things out nationally and embrace them as best practice.”

“If we take the treatment-and-control approach, we’re going to make a lot more progress quickly despite the fact that it’s frustrating,” says Laibson, “because when you commit to running randomized control trials, it slows down the legislation, it slows down the national adoption, because you’ve got to wait a year for those results to come in. But that’s a year that’s worth waiting.”

Those who are working to add more value to the health system may also need some training to help them understand what the best new design might be, adds Dartmouth College marketing expert Punam Keller. She has worked with the Centers for Disease Control and Prevention on HealthCommWorks, which uses an empirical algorithm to help you tailor your messages to your audience’s needs. The website also offers ProofWorks, which is “basically like PhD 101,” says Keller. It includes information on good design, control, random assignment, sample sizes, metrics validation, intervention and testing time, and outcome variables. “Those kinds of things are important if we are to improve the rigor of the data that we have in order to improve our collective understanding,” she says.

Watch the video:

From the NEJM Catalyst event Patient Engagement: Behavioral Strategies for Better Health at the University of Pennsylvania, February 25, 2016.

New Call for Submissions ­to NEJM Catalyst

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
Karen DeSalvo - public health population health social determinants of health expert social services infrastructure

Moving from a Sickness Model of Health Care to One of Well-Being — Part 2

A public health expert discusses the scientific, technological, and financial foundations we need to modernize the social services infrastructure and address social determinants of health.

PROs Associated with Better Top-Box Satisfaction Scores at UPMC

Better Physician Ratings from Discussing PROs with Patients

A UPMC study demonstrates the value of PROs, but finds that physicians need to discuss that information with patients during visits.

Karen DeSalvo - public health population health social determinants of health expert social services infrastructure

Moving from a Sickness Model of Health Care to One of Well-Being — Part 1

The expense of the U.S. health care system is so misaligned from what we need as a country — a healthier population. A public health expert discusses how we can move toward a model that addresses social determinants of health and promotes well-being while still growing our economy.

Different Means of Initiating and Sustaining Behavior Change

Survey Snapshot: Social Support, Whether In-Person or Virtual, Is the Key to Sustained Behavior Change

NEJM Catalyst Insights Council members seek ways to create relationships with patients — but the hurdles are high.

Different Means of Initiating and Sustaining Behavior Change

Patient Engagement Survey: What Creates Behavior Change May Not Sustain It

An NEJM Catalyst Insights Council survey finds that human interaction and social support are vital for patient behavior change and employee wellness programs.

How the EpxSubstanceUse Substance Use Disorder Mobile Health Texting Program Works - Preferred Family Healthcare Epharmix Technology Assisted Care Coordination Project Opioid Abuse

Messaging System Helps Caregivers Keep Tabs on Growing Number of Patients with Substance Use Disorder

One behavioral health provider pilots cost-effective outreach to patients with SUD that checks up on their recovery, triages needs, and facilitates communication.

Gary Kaplan Head Shot

Improving Patient Involvement in Care

The CEO of Virginia Mason on facilitating and enhancing patient participation in care.

Chou01_pullquote_Time to Start Using Evidence-Based Approaches to Patient-Provider Communication

Time to Start Using Evidence-Based Approaches to Patient Engagement

Clinicians must acknowledge that there is a dramatic difference between learning a concept about patient-provider communication and learning the actual skills for effective execution.

Nash01_pullquote - A Requiem for Value-Based Care Population Health.

A Requiem for Population Health?

Despite some fits and starts for value-based care, there is abundant evidence that the concept will continue to survive and thrive.

Steps for Deploying Nudge Unit in Health Care

Nudge Units to Improve the Delivery of Health Care

Key information and important choices are constantly being presented in health care. Yet often the frames or default options used are selected without attention to strategic goals.

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

Topics

Patient Incentives

60 Articles

Survey Snapshot: Social Support, Whether In-Person…

NEJM Catalyst Insights Council members seek ways to create relationships with patients — but the…

Moving from a Sickness Model of…

A public health expert discusses the scientific, technological, and financial foundations we need to modernize…

Patient-Centered Care

211 Articles

Better Physician Ratings from Discussing PROs…

A UPMC study demonstrates the value of PROs, but finds that physicians need to discuss…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now