Patient Engagement

Why Real-World Results Are So Challenging for Digital Health

Article · May 14, 2017

Companies should look for lessons in other industries, create products that are tailored to population subsets, develop synergistic partnerships, and understand the performance characteristics of products in the real world.


Despite years of hype in the field of digital health products — a term that we use here to describe technologies that are designed to have clinical impact on disease — fewer products than expected are being deployed in real-world clinical settings. Many digital health products that demonstrate impressive results in clinical trials often fail to do so in real-world settings.

Why? Much of the success of digital health products is predicated on patient engagement, and clinical trials are among the most engaging environments in health care. Clinical trials involve the use of a variety of tools (e.g., training, close monitoring, payments) to ensure that patients use the technologies appropriately, but few of these tools are used in the real world. In order to cross the chasm from success in clinical trials to success in practice, digital health companies need to focus on patient engagement.

Digital Health Hinges on Engagement

Engagement is so important because many digital health products are designed to achieve behavioral changes for the purpose of preventing or treating chronic diseases. For patients who are at risk for, or are living with, a chronic disease, successful prevention or management requires minute-to-minute, day-to-day changes in decision-making. Patients need to be highly motivated to make behavioral changes, they need to be praised when they follow through, and they need guidance when they slip up. The process needs to be sticky and self-reinforcing in order to maintain patient interest. In a nutshell, patients need to be engaged.

Engagement is important not only for clinical results, but also for the business models of many digital health companies, which frequently incorporate pay-for-performance contracts. Digital health companies need to track data continuously in order to demonstrate their value to the purchaser or user. Products that do not engage patients drag down performance metrics and reduce the viability of the company.

Contrast this with pharmaceuticals: payors pay for drugs before patients take them, and pay-for-performance contracts are the exception; this is part of the reason why nearly 50% of medications for chronic disease are not taken as prescribed.

The Gap Between Clinical Trials and Real-World Results

Successful clinical trials of both digital health products and pharmaceuticals are engaging by definition. In order for a trial to demonstrate significant results — and in order for the results of the trial to be published — the attrition rate for participants needs to be very low. The entire endeavor is designed around ensuring that patients use the product or service appropriately and follow up regularly.

A clinical trial is designed as follows: an artificial scenario is crafted, willing participants volunteer, participants are carefully selected on the basis of optimal criteria, staffers are thoroughly trained, measurements are taken like clockwork, and participants are induced (by means of monetary payments or a barrage of phone calls, emails, and texts) to use the product or service as directed. As a result, the experience of individuals who are involved in a clinical trial typically is much more engaging than that of patients who use the technology in real-world scenarios.

Tools to Drive Engagement

Extending the engagement-boosting practices from a clinical trial to a much larger population of patients in the real world is not feasible because of cost and logistical complexity. As a result, different approaches are needed to facilitate engagement.

  • Maintain engagement-first thinking. Digital health products need to be designed to drive engagement from the ground up. Focusing on an exciting technology or a massive need is not enough. Companies should consider employing principles from behavioral economics, which have been shown to increase engagement and product stickiness by using sophisticated implementations of financial incentives, loyalty points, social structures, and other techniques.

Social media also have been shown to be a powerful force in patient engagement, with applications ranging from open patient support resources such as PatientsLikeMe to the patient-only social media networks used by Omada Health to drive weight loss. Companies also should think about employing strategies from video games and sports. While “gamification,” as this approach is known, has been mostly met with a tepid response from the physician community, don’t forget that Pokémon GO has performed better than many digital health tools at increasing the physical activity levels of its users.

  • Embrace real-world data to drive real-world evidence. Real-world data (RWD) and real-world evidence (RWE) are topics of increasing interest in the medical community and have even garnered the attention of the FDA. While RWD and RWE typically are used to supplement clinical trials, these information sources are increasingly important for digital health companies because they provide critical evidence of user engagement. Digital health companies can design studies to capture RWD and build RWE by crafting scenarios that mirror how products will be used by real patients.

Companies can look to Omada Health’s diabetes prevention study for an example of how to design a study to collect RWD and RWE. Omada recruited participants on Craigslist, participants relayed weight measurements to the company with use of a digital scale equipped with cellular data-transmitting capabilities, and HbA1c measurements were obtained with use of home kits that were mailed to participants. The study was performed in a similar way to how Omada interacts with real patients and thus provided the company with a good understanding of the real-world performance characteristics of its product.

  • Develop product variations. Engagement is not one-size-fits-all. Different tactics will work well for different populations, and digital products may be more successful if the manufacturer is able to develop variations to serve different segments of the market. For example, Propeller Health has developed sensors that can track inhaler use, with the goal of improving outcomes and reducing costs for patients with asthma and chronic obstructive pulmonary disease (COPD). Although patients with both conditions often use inhaled medications, the demographics, etiology, prognosis, and medication regimens for these conditions differ substantially. Product variations specific to the different patient populations using the inhaler will be critical to Propeller’s success in the treatment of these diseases. Companies must think through how to engage each subset of the population that the product is serving.

Making the Leap from Novelty to Necessity

Clinical trials are a critical process in the evolution of digital health products. Evidence is an important means for convincing health care executives to buy these products. However, simply learning that an intervention was successful in a clinical study is not enough. Real-world validation is also important as digital health companies own the validation process from product development to scale. Once clinical validation is achieved, real-world evidence must be gathered to justify success at scale. In order for digital health products to make the leap from novelty to necessity, manufacturers will need to focus on driving patient engagement in real-world settings. Engagement will require taking lessons from other industries, creating products that are tailored to population subsets, developing synergistic partnerships, and understanding the performance characteristics of products in the real world.


This article originally appeared in NEJM Catalyst on February 8, 2017.

New Call for Submissions ­to NEJM Catalyst


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement

Data Graphic: The Two Most Effective Patient Engagement Initiatives

NEJM Insights Council members say the only broadly effective approaches are time spent with patients and shared decision-making.

How Chronic Pain Treatment Falls Short of Patient-Centered Care

Healing the opioid epidemic requires nothing less than a comprehensive theory of patient health.

Patient Engagement versus Patient Experience

The difference between the two terms is muddled but important, says Cleveland Clinic’s Chief Experience Officer.

Evaluating Complex Care Programs: Is It a Zero-Sum Game?

Policymakers see programs for complex patient populations as a way to bend the health care cost curve, but are reduced health care utilization and costs the right measures of success?

Engaging Health Care App Design: Reflections from the Field

Millions of health care apps have perished from user indifference. A better design approach produces apps that users need and want.

Toyin Ajayi Talk Still: The Difference Between Noncompliance and Defiance

The Difference Between Noncompliance and Defiance

How do we align our goals for patient engagement with even the most complex, difficult patients?

Data Graphic: What Are Patient Technology Tools Good For?

NEJM Catalyst Insights Council members weigh on the best and worst uses for patient engagement technology tools.

Patient Engagement Survey: How to Hardwire Engagement into Care Delivery Processes

Technology and social networks can help, but nurses and care teams remain essential, say NEJM Catalyst Insights Council members.

Hardwiring Patient Engagement to Deliver Better Health

Imagine a world in which health care was not “one size fits all,” but customized for each patient individually.

Health Care — A Final Frontier for Design

Design must move beyond narrow projects and encompass complex systems.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Data Graphic: The Two Most Effective…

NEJM Insights Council members say the only broadly effective approaches are time spent with patients…

Patients As Customers

74 Articles

What U.S. Hospitals Can Still Learn…

Cost-cutting lessons are clear, so what stands in our way?

Patient Incentives

46 Articles

Data Graphic: The Two Most Effective…

NEJM Insights Council members say the only broadly effective approaches are time spent with patients…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now