Patient Engagement
Technology to Empower Patients

Why Real-World Results Are So Challenging for Digital Health

Article · July 10, 2017

Companies should look for lessons in other industries, create products that are tailored to population subsets, develop synergistic partnerships, and understand the performance characteristics of products in the real world.


Despite years of hype in the field of digital health products — a term that we use here to describe technologies that are designed to have clinical impact on disease — fewer products than expected are being deployed in real-world clinical settings. Many digital health products that demonstrate impressive results in clinical trials often fail to do so in real-world settings.

Why? Much of the success of digital health products is predicated on patient engagement, and clinical trials are among the most engaging environments in health care. Clinical trials involve the use of a variety of tools (e.g., training, close monitoring, payments) to ensure that patients use the technologies appropriately, but few of these tools are used in the real world. In order to cross the chasm from success in clinical trials to success in practice, digital health companies need to focus on patient engagement.

Digital Health Hinges on Engagement

Engagement is so important because many digital health products are designed to achieve behavioral changes for the purpose of preventing or treating chronic diseases. For patients who are at risk for, or are living with, a chronic disease, successful prevention or management requires minute-to-minute, day-to-day changes in decision-making. Patients need to be highly motivated to make behavioral changes, they need to be praised when they follow through, and they need guidance when they slip up. The process needs to be sticky and self-reinforcing in order to maintain patient interest. In a nutshell, patients need to be engaged.

Engagement is important not only for clinical results, but also for the business models of many digital health companies, which frequently incorporate pay-for-performance contracts. Digital health companies need to track data continuously in order to demonstrate their value to the purchaser or user. Products that do not engage patients drag down performance metrics and reduce the viability of the company.

Contrast this with pharmaceuticals: payors pay for drugs before patients take them, and pay-for-performance contracts are the exception; this is part of the reason why nearly 50% of medications for chronic disease are not taken as prescribed.

The Gap Between Clinical Trials and Real-World Results

Successful clinical trials of both digital health products and pharmaceuticals are engaging by definition. In order for a trial to demonstrate significant results — and in order for the results of the trial to be published — the attrition rate for participants needs to be very low. The entire endeavor is designed around ensuring that patients use the product or service appropriately and follow up regularly.

A clinical trial is designed as follows: an artificial scenario is crafted, willing participants volunteer, participants are carefully selected on the basis of optimal criteria, staffers are thoroughly trained, measurements are taken like clockwork, and participants are induced (by means of monetary payments or a barrage of phone calls, emails, and texts) to use the product or service as directed. As a result, the experience of individuals who are involved in a clinical trial typically is much more engaging than that of patients who use the technology in real-world scenarios.

Tools to Drive Engagement

Extending the engagement-boosting practices from a clinical trial to a much larger population of patients in the real world is not feasible because of cost and logistical complexity. As a result, different approaches are needed to facilitate engagement.

  • Maintain engagement-first thinking. Digital health products need to be designed to drive engagement from the ground up. Focusing on an exciting technology or a massive need is not enough. Companies should consider employing principles from behavioral economics, which have been shown to increase engagement and product stickiness by using sophisticated implementations of financial incentives, loyalty points, social structures, and other techniques.

Social media also have been shown to be a powerful force in patient engagement, with applications ranging from open patient support resources such as PatientsLikeMe to the patient-only social media networks used by Omada Health to drive weight loss. Companies also should think about employing strategies from video games and sports. While “gamification,” as this approach is known, has been mostly met with a tepid response from the physician community, don’t forget that Pokémon GO has performed better than many digital health tools at increasing the physical activity levels of its users.

  • Embrace real-world data to drive real-world evidence. Real-world data (RWD) and real-world evidence (RWE) are topics of increasing interest in the medical community and have even garnered the attention of the FDA. While RWD and RWE typically are used to supplement clinical trials, these information sources are increasingly important for digital health companies because they provide critical evidence of user engagement. Digital health companies can design studies to capture RWD and build RWE by crafting scenarios that mirror how products will be used by real patients.

Companies can look to Omada Health’s diabetes prevention study for an example of how to design a study to collect RWD and RWE. Omada recruited participants on Craigslist, participants relayed weight measurements to the company with use of a digital scale equipped with cellular data-transmitting capabilities, and HbA1c measurements were obtained with use of home kits that were mailed to participants. The study was performed in a similar way to how Omada interacts with real patients and thus provided the company with a good understanding of the real-world performance characteristics of its product.

  • Develop product variations. Engagement is not one-size-fits-all. Different tactics will work well for different populations, and digital products may be more successful if the manufacturer is able to develop variations to serve different segments of the market. For example, Propeller Health has developed sensors that can track inhaler use, with the goal of improving outcomes and reducing costs for patients with asthma and chronic obstructive pulmonary disease (COPD). Although patients with both conditions often use inhaled medications, the demographics, etiology, prognosis, and medication regimens for these conditions differ substantially. Product variations specific to the different patient populations using the inhaler will be critical to Propeller’s success in the treatment of these diseases. Companies must think through how to engage each subset of the population that the product is serving.

Making the Leap from Novelty to Necessity

Clinical trials are a critical process in the evolution of digital health products. Evidence is an important means for convincing health care executives to buy these products. However, simply learning that an intervention was successful in a clinical study is not enough. Real-world validation is also important as digital health companies own the validation process from product development to scale. Once clinical validation is achieved, real-world evidence must be gathered to justify success at scale. In order for digital health products to make the leap from novelty to necessity, manufacturers will need to focus on driving patient engagement in real-world settings. Engagement will require taking lessons from other industries, creating products that are tailored to population subsets, developing synergistic partnerships, and understanding the performance characteristics of products in the real world.


Disclaimer: Omada Health CEO Sean Duffy, an NEJM Catalyst Patient Engagement Thought Leader, had no involvement in the creation of this article.

This article originally appeared in NEJM Catalyst on February 8, 2017.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement

Why People Stick With or Abandon Wearable Devices

Embedding fitness trackers in eyeglasses demonstrates that personality and motivational feedback play important roles in activity levels.

Patient-Reported Outcomes — Are They Living Up to Their Potential?

The patient’s voice can be at the forefront of health care delivery.

Patient Satisfaction in Anesthesiology Matters

A good patient experience is part of a successful anesthetic.

Walking with Our Patients in the Community

Mobile integrated health, a disruptive home-based delivery model, repurposes existing resources and addresses critical care gaps for high-risk, high-cost populations.

Home-to-Home Time — Measuring What Matters to Patients and Payers

Although discharging patients to a post-acute care facility might help hospitals reduce their average length of stay, it’s unclear when post-acute care improves patient outcomes and when it doesn’t.

EASE Orlando Health mobile app patient view

Improving Communication from the Operating Room: A New Mobile Application to Enhance the Family Experience

How Orlando Health’s new mobile app resulted in a 98% reduced anxiety rate for patients and their families in the waiting room.

Data Graphic: What Are the Most Effective Technologies in Patient Engagement Initiatives?

Members of the NEJM Catalyst Insights Council say that biometric measurement devices such as wireless scales or glucometers are the most effective.

In Your Words: Where Is the Greatest Potential for Technology to Enable Patient Engagement?

Responses from a survey of NEJM Catalyst Insights Council members.

Four Challenges of Launching a Telehealth Program

NewYork-Presbyterian tackled liability, licensure, reimbursement, and workflow to launch their successful telehealth program.

If We Can See the Outcome, We Can Change the Outcome

How Penn Medicine’s Chief Innovation Officer parlayed success at one of the nation’s biggest software companies into meaningful outcomes for health care.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Measuring Performance of OpenNotes Initiatives to…

How a New York safety-net health system used data science to identify obstacles to OpenNotes…

Patient Incentives

75 Articles

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care,…

Patients As Customers

141 Articles

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now