Patient Engagement

A/B Testing Health Behavior

Article · March 10, 2017

Medicine has been accused of implementing new interventions without a strong underlying evidence base, and that impulse for quick implementation is understandable, says Scott Halpern, Deputy Director for the University of Pennsylvania’s Center for Health Incentives and Behavioral Economics. “We have sick people. We need to do something. We can’t sit idly by,” he says. “But I wonder if that instinct to not just stand there — to do something — might have some unintended consequences, crowding out the potential for greater innovation that has better evidence to support it.” What can we learn from other industries that have grappled with this tension?

A decade ago, there was almost no A/B testing in the corporate sector, notes Harvard economist David Laibson. “The CEO or someone else had a great idea and they implemented it, and then they had anecdotes about how great it was — and that was the way you made progress,” he says, adding, “a lot of that was a huge waste of money.”

This attitude first started changing in Silicon Valley, where companies became experts at treatment and control. Today, they’ll immediately start randomizing a new idea and maybe within a day have enough data to know whether to proceed with it. “We can’t do that in health care, of course, because we don’t have the control and the systems are complicated,” says Laibson. “It’s not that we should aspire to 24-hour experiments, but we should aspire to experiments, to actually randomizing and figuring out what works before we roll things out nationally and embrace them as best practice.”

“If we take the treatment-and-control approach, we’re going to make a lot more progress quickly despite the fact that it’s frustrating,” says Laibson, “because when you commit to running randomized control trials, it slows down the legislation, it slows down the national adoption, because you’ve got to wait a year for those results to come in. But that’s a year that’s worth waiting.”

Those who are working to add more value to the health system may also need some training to help them understand what the best new design might be, adds Dartmouth College marketing expert Punam Keller. She has worked with the Centers for Disease Control and Prevention on HealthCommWorks, which uses an empirical algorithm to help you tailor your messages to your audience’s needs. The website also offers ProofWorks, which is “basically like PhD 101,” says Keller. It includes information on good design, control, random assignment, sample sizes, metrics validation, intervention and testing time, and outcome variables. “Those kinds of things are important if we are to improve the rigor of the data that we have in order to improve our collective understanding,” she says.

Watch the video:

From the NEJM Catalyst event Patient Engagement: Behavioral Strategies for Better Health at the University of Pennsylvania, February 25, 2016.

New call for submissions ­to NEJM Catalyst

Now inviting longform articles

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
Bilazarian01_pullquote patient engagement both sides of the bed

Patient Engagement from Both Sides of the Bed

When patients and families are included in medical rounds as valued members of the team, the quality of care improves.

Ten Guiding Principles for Patient-Centered Care as Described by Mountain States Health Alliance to consider with patient-centered outcomes research (PCOR)

Complementary Approaches to Patient Engagement in Patient-Centered Outcomes Research

It’s important for providers to understand and apply the key principles foundational to patient-centered outcomes research efforts.

Selected Advantages of Improved Patient Electronic Access to Health Data

Patient-Led Data Sharing — A New Paradigm for Electronic Health Data

The pieces are in place for a truly disruptive shift in how patients can access and use their own clinical data to improve their health.

Collapse of Two Online Health Communities as Result of Removal of Users Starting from Superusers

Online “Superusers” as Allies of the Health Care Workforce

Three proposed steps for integrating peer-driven online health communities with traditional services to improve health outcomes for people with chronic conditions.

The PEACE Project - Patient Experience for Acute Care Elders at Sarasota Memorial Hospital

The PEACE Project

Improving the acute care experience for our eldest patients.

Rabson01_pullquote barriers to patient engagement in primary care practices

What Does It Take to Increase Patient Engagement in Primary Care Settings?

Massachusetts primary care practices are beginning to invest in programs to increase patient engagement, but these strategies can be challenging to implement and remain underutilized and undervalued.

Many Barriers to Engaging Patients in Treatment for Obesity

Survey Snapshot: Approaches to Address Clinician and Societal Roles in Obesity

The obesity problem is getting larger and larger. Some solutions lie beyond the traditional medical setting.

Esch01_pullquote patient activation

OpenNotes, Patient Narratives, and Their Transformative Effects on Patient-Centered Care

The development of standardized tools and techniques are enhancing the ability of providers to interact with patients, but true patient engagement requires a common understanding of its theoretical foundation and an open mind for including patients’ needs and beliefs — in their language — as part of each encounter.

Patients and PCPs Hold Primary Responsibility for Addressing Obesity

Patient Engagement Survey: The Failure of Obesity Efforts and the Collective Nature of Solutions

Patients hold responsibility for their actions, but providers’ approaches to address obesity often fall short as well.

Items from the Shared Decision-Making Process Survey for Elective Surgical Procedures - patient decision aids - decision quality - informed consent

Shared Decision-Making: Staying Focused on the Ultimate Goal

Despite growing acceptance and enthusiasm for patient involvement in their health care decisions, clinicians, at times, lose focus on the ultimate goal of shared decision-making: better health decisions from the informed patient’s perspective.

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

Topics

Patient Incentives

68 Articles

Online “Superusers” as Allies of the…

Three proposed steps for integrating peer-driven online health communities with traditional services to improve health…

Patient-Centered Care

245 Articles

Engaging Stakeholders to Produce Sustainable Change…

How an initiative designed to improve patient outcomes and satisfaction while containing costs led to…

Patients As Customers

123 Articles

Patient Engagement from Both Sides of…

When patients and families are included in medical rounds as valued members of the team,…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now