Patient Engagement

A/B Testing Health Behavior

Article · March 10, 2017

Medicine has been accused of implementing new interventions without a strong underlying evidence base, and that impulse for quick implementation is understandable, says Scott Halpern, Deputy Director for the University of Pennsylvania’s Center for Health Incentives and Behavioral Economics. “We have sick people. We need to do something. We can’t sit idly by,” he says. “But I wonder if that instinct to not just stand there — to do something — might have some unintended consequences, crowding out the potential for greater innovation that has better evidence to support it.” What can we learn from other industries that have grappled with this tension?

A decade ago, there was almost no A/B testing in the corporate sector, notes Harvard economist David Laibson. “The CEO or someone else had a great idea and they implemented it, and then they had anecdotes about how great it was — and that was the way you made progress,” he says, adding, “a lot of that was a huge waste of money.”

This attitude first started changing in Silicon Valley, where companies became experts at treatment and control. Today, they’ll immediately start randomizing a new idea and maybe within a day have enough data to know whether to proceed with it. “We can’t do that in health care, of course, because we don’t have the control and the systems are complicated,” says Laibson. “It’s not that we should aspire to 24-hour experiments, but we should aspire to experiments, to actually randomizing and figuring out what works before we roll things out nationally and embrace them as best practice.”

“If we take the treatment-and-control approach, we’re going to make a lot more progress quickly despite the fact that it’s frustrating,” says Laibson, “because when you commit to running randomized control trials, it slows down the legislation, it slows down the national adoption, because you’ve got to wait a year for those results to come in. But that’s a year that’s worth waiting.”

Those who are working to add more value to the health system may also need some training to help them understand what the best new design might be, adds Dartmouth College marketing expert Punam Keller. She has worked with the Centers for Disease Control and Prevention on HealthCommWorks, which uses an empirical algorithm to help you tailor your messages to your audience’s needs. The website also offers ProofWorks, which is “basically like PhD 101,” says Keller. It includes information on good design, control, random assignment, sample sizes, metrics validation, intervention and testing time, and outcome variables. “Those kinds of things are important if we are to improve the rigor of the data that we have in order to improve our collective understanding,” she says.

Watch the video:

From the NEJM Catalyst event Patient Engagement: Behavioral Strategies for Better Health at the University of Pennsylvania, February 25, 2016.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
Health Care Providers Should Incentivize Patients

Survey Snapshot: Patient Financial Incentives — There Are No Quick Fixes

The NEJM Catalyst Insights Council agrees that while financial incentives are a common strategy to engage patients in healthy behaviors, they are not necessarily effective.

Support of Family and Friends Is More Effective Than Clinician Support in Realizing Health Goals - From the Patient Engagement Insights Report: Why No Single Health Incentive Works.

Patient Engagement Survey: Why No Single Health Incentive Works

Initiatives to improve patient engagement come in a variety of forms. While insurers, employers, and health care providers are all involved in using financial incentives and penalties for engagement efforts, improvement in health outcomes has been elusive. Achieving that ultimate goal will usually require a combination of financial and social approaches.

Health Systems Attending the Nudge Units in Health Care Symposium - Penn Medicine

Key Insights on Launching a Nudge Unit within a Health Care System

Leaders are finding that making higher-value choices easier through subtle changes to choice architecture can have an outsized impact on medical decision-making.

Barriers to Providing an Oustanding Patient Experience

Buzz Survey Report: Patient Experience

An independent NEJM Catalyst report sponsored by University of Utah Health on barriers to achieving an excellent patient experience.

The Patient Engagement Capacity Framework

The Patient Engagement Capacity Model: What Factors Determine a Patient’s Ability to Engage?

Patient engagement assessments often don’t dig deep enough to identify why patients don’t participate in their own health care. We present a new model to help providers pinpoint the reasons for lack of engagement and address them more effectively.

Organizational Mindset Is the Biggest Barrier to Engaging Patients as Consumers

Survey Snapshot: The Patient-Physician Relationship Is Key

Both parties involved in a consumer-facing transaction have access to important information about the product or service — but this isn’t the case with health care.

What High-Need, High-Cost Patients Say About How to Reduce High Utilization of ED and Inpatient Services

High-Need, High-Cost Patients Offer Solutions for Improving Their Care and Reducing Costs

More home health care and after-hours clinics, telemedicine, and home delivery of medications are among top solutions.

Top Physician Pain Points Identified by Chronic Patients

Unmet Needs: Hearing the Challenges of Chronic Patients with Artificial Intelligence

With natural language processing and machine learning, researchers are identifying patient emotional and medical needs that are not being met by clinicians and patient advocacy groups.

Health Care Can Learn a Lot from Other Consumer-Facing Industries - Especially About Customer Service - Consumerization of Health Care

Patient Engagement Survey: Health Care Has a Lot to Learn from Consumer-Friendly Industries

The consumerization of health care continues to reshape the way that patients engage with providers and experience care. Most providers see this fundamental change in the health care model as a necessary response to changing patient demands, and have embraced the need to learn from other industries.

Bilazarian01_pullquote patient engagement both sides of the bed

Patient Engagement from Both Sides of the Bed

When patients and families are included in medical rounds as valued members of the team, the quality of care improves.

Connect

A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

Topics

Patient-Centered Care

262 Articles

Promoting Interoperability: Roles for Commercial Payers

Interoperability is a business imperative for enhancing value in health care, and providers and payers…

Patients As Customers

131 Articles

Buzz Survey Report: Patient Experience

An independent NEJM Catalyst report sponsored by University of Utah Health on barriers to achieving…

Heart Safe Motherhood: Applying Innovation Methodology…

At the Hospital of the University of Pennsylvania, a text message–based blood pressure surveillance program…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now