Patient Engagement

A/B Testing Health Behavior

Article · March 10, 2017

Medicine has been accused of implementing new interventions without a strong underlying evidence base, and that impulse for quick implementation is understandable, says Scott Halpern, Deputy Director for the University of Pennsylvania’s Center for Health Incentives and Behavioral Economics. “We have sick people. We need to do something. We can’t sit idly by,” he says. “But I wonder if that instinct to not just stand there — to do something — might have some unintended consequences, crowding out the potential for greater innovation that has better evidence to support it.” What can we learn from other industries that have grappled with this tension?

A decade ago, there was almost no A/B testing in the corporate sector, notes Harvard economist David Laibson. “The CEO or someone else had a great idea and they implemented it, and then they had anecdotes about how great it was — and that was the way you made progress,” he says, adding, “a lot of that was a huge waste of money.”

This attitude first started changing in Silicon Valley, where companies became experts at treatment and control. Today, they’ll immediately start randomizing a new idea and maybe within a day have enough data to know whether to proceed with it. “We can’t do that in health care, of course, because we don’t have the control and the systems are complicated,” says Laibson. “It’s not that we should aspire to 24-hour experiments, but we should aspire to experiments, to actually randomizing and figuring out what works before we roll things out nationally and embrace them as best practice.”

“If we take the treatment-and-control approach, we’re going to make a lot more progress quickly despite the fact that it’s frustrating,” says Laibson, “because when you commit to running randomized control trials, it slows down the legislation, it slows down the national adoption, because you’ve got to wait a year for those results to come in. But that’s a year that’s worth waiting.”

Those who are working to add more value to the health system may also need some training to help them understand what the best new design might be, adds Dartmouth College marketing expert Punam Keller. She has worked with the Centers for Disease Control and Prevention on HealthCommWorks, which uses an empirical algorithm to help you tailor your messages to your audience’s needs. The website also offers ProofWorks, which is “basically like PhD 101,” says Keller. It includes information on good design, control, random assignment, sample sizes, metrics validation, intervention and testing time, and outcome variables. “Those kinds of things are important if we are to improve the rigor of the data that we have in order to improve our collective understanding,” she says.

Watch the video:

From the NEJM Catalyst event Patient Engagement: Behavioral Strategies for Better Health at the University of Pennsylvania, February 25, 2016.

New Call for Submissions ­to NEJM Catalyst

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
How the EpxSubstanceUse Substance Use Disorder Mobile Health Texting Program Works - Preferred Family Healthcare Epharmix Technology Assisted Care Coordination Project Opioid Abuse

Messaging System Helps Caregivers Keep Tabs on Growing Number of Patients with Substance Use Disorder

One behavioral health provider pilots cost-effective outreach to patients with SUD that checks up on their recovery, triages needs, and facilitates communication.

Gary Kaplan Head Shot

Improving Patient Involvement in Care

The CEO of Virginia Mason on facilitating and enhancing patient participation in care.

Chou01_pullquote_Time to Start Using Evidence-Based Approaches to Patient-Provider Communication

Time to Start Using Evidence-Based Approaches to Patient Engagement

Clinicians must acknowledge that there is a dramatic difference between learning a concept about patient-provider communication and learning the actual skills for effective execution.

Nash01_pullquote - A Requiem for Value-Based Care Population Health.

A Requiem for Population Health?

Despite some fits and starts for value-based care, there is abundant evidence that the concept will continue to survive and thrive.

Steps for Deploying Nudge Unit in Health Care

Nudge Units to Improve the Delivery of Health Care

Key information and important choices are constantly being presented in health care. Yet often the frames or default options used are selected without attention to strategic goals.

Example Cost of Custom-Built PRO Collection Platform

The Cost of Patient-Reported Outcomes in Medicine

Implementation of routine PRO collection is paramount to measuring and maximizing value in health care.

Mentoring, Education, and Engagement: An Employee Training Technique to Improve Patient Experience and Employee Satisfaction

How open communication and one-to-one mentoring generated team spirit and a learning environment.

Face-to-Face Group Sessions Are the Most Useful Mode of Communication in Health Social Networks

Survey Snapshot: Patient Networks Are Important for Outcomes

NEJM Catalyst Insights Council members overwhelmingly agree that social networks can impact chronic diseases and healthy behaviors, but time commitment and funding prevent their prevalence.

Consumerism in Health Care

Will personalized interventions transform how we improve an individual’s health? A frequent-flier expert believes that it will.

Chronic Disease Management and Healthy Behavior Promotion Are the Best Uses of Health Social Networks

Patient Engagement Survey: Social Networks to Improve Patient Health

Social networks — both face-to-face and virtual — have been shown to positively impact behaviors such as weight loss and smoking cessation. Is the health care industry ready to more widely leverage this mechanism for increasing healthy behaviors among patients?

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

Topics

Messaging System Helps Caregivers Keep Tabs…

One behavioral health provider pilots cost-effective outreach to patients with SUD that checks up on…

Patients As Customers

99 Articles

Time to Start Using Evidence-Based Approaches…

Clinicians must acknowledge that there is a dramatic difference between learning a concept about patient-provider…

Patient Incentives

58 Articles

Improving Patient Involvement in Care

The CEO of Virginia Mason on facilitating and enhancing patient participation in care.

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now