Patient Engagement

Unmet Needs: Hearing the Challenges of Chronic Patients with Artificial Intelligence

Article · January 30, 2019

“I wish I had been told that medications were only 30-50% effective before I started in 2010. I also wish I’d heard about possible negative side effects instead of being told It was ‘so safe I’d give it to my mother.’”

This quote is from a post on a public online patient forum for people with osteoporosis. In an age when the Internet has enabled patients to quickly catch up on the latest medical treatment plans and news, patients have also become accustomed to turning to online forums as a resource to share their joys and frustrations with others undergoing the same experience. For patients with acute illnesses, these channels may be a way to find a quick answer from a friendly stranger. For patients with chronic diseases, though, these forums represent a way to build a virtual community online and discuss all aspects of their care with the privacy afforded by online anonymity. Over the past 3 years, Quid, a data and language analysis platform, and UCB, a global biopharmaceutical company, have partnered to analyze these conversations at scale to uncover the collective unmet needs of chronic patients.

Knowing what it is like to live with a chronic disease is impossible unless you have personally experienced it, but the rapid advancements in artificial intelligence now provide the possibility of beginning to understand those difficulties. Using natural language processing and machine learning, we have constructed a data-driven view of unmet patient needs by tapping into more than 500,000 de-identified and anonymous public comments that occurred in openly accessible digital media for patients living with one of ten chronic diseases from 2010 to 2018. These chronic disease areas were chosen to reflect the diversity of online chronic patient communities across a variety of therapeutic areas. The de-identified comments were collected from a variety of some of the largest public forums for chronic diseases, including general ones such as,, and, along with forums like Crohn’s Forum,, and ALZConnected. This article represents our attempt to understand the unmet needs from the collective voices of these patients, the associated pain points in the doctor-patient relationship, and how advocacy groups have thus far addressed these needs.

Patients’ Unmet Needs

By analyzing those patient comments using natural language processing, we identified and prioritized eight specific patient wants or needs that were unmet. Based on the majority of comments and language within each need, these needs were classified into two groups, medical and emotional. Needs that were a mix of medical and emotional were classified based on the predominant language used within the patient comments:

  • Medical needs: needs related to symptoms, treatment specifics, and curing the disease
  • Emotional needs: needs related to knowledge and power to make informed decisions and prepare for daily life, how to manage the disease long term, and community engagement

Interestingly, the collective data shows that six of the top eight unmet needs are more long-term emotional concerns, with the top two both focusing on how to live with the disease (understanding medication side effects and impact on daily life, and coping with living with the condition). Health care providers have long known the importance of addressing quality-of-life issues for chronic patients, and our quantitative analysis now confirms this importance across a variety of disease areas. In other words, patients have come to understand that their conditions cannot be fixed by short-term medical interventions alone and are now actively asking for help in living with the disease — they want some degree of control over how the illness affects their lives. Identifying, understanding, and actively targeting these needs on a granular basis is a critical first step to providing the health care that patients want.

“I understand a flare means psoriasis is worsening. Does it also mean psoriasis is spreading?” Online psoriasis patient forum post

Living with the disease also means understanding the disease. As the figure below demonstrates, two of the top eight needs reference how patients are now seeking to understand their disease more deeply, on theoretical and practical levels. The Internet has made it easy for patients to search any disease and obtain immediate access to information, but because patients lack medical training, there is still a gap in helping them truly understand their conditions in full complexity. As such, it is not enough to inform patients on the exact medical steps that they should take to manage their condition (medication schedule, side effects to watch for, etc.), but rather, patients are asking for help in understanding the reasoning and science behind those medical steps.

  Click To Enlarge.

Increasing Negativity for Some, Hope for Others

“I am having problems that I believe are related to my Crohn’s disease . . . . I am scared to go to the Gastroenterologist, because the medications that I will have to take are frightening.” Crohn’s patient forum post

As we look at the year-over-year analysis of patient comments from the past 10 years, we see that the overall amount of negative and positive comments has remained relatively stable when considering all ten disease states together. However, when we take a closer look at comments from specific disease areas, a more insightful picture emerges: some diseases have seen a rise in negativity, while others have remained stable, and one has actually increased in positivity over the past 10 years.

Patient Forum Post Sentiment Over the Past 10 Years

  Click To Enlarge.

In terms of growing negativity, patient posts from Alzheimer’s, Crohn’s, and epilepsy patients have veered negative. For Alzheimer’s patients, the negativity centers around expectations and testing. For example, patients and caregivers have had increasing concerns over whether neuropsychological testing could objectively test and detect progression of the disease. Others had issues with social interactions and explaining the course of the disease. For Crohn’s patients, access to expensive medications is a common recent theme, with patients finding that continuing coverage even on effective medications to be a challenge. And for epilepsy patients, fears of coping with public seizures and the resulting social fallout is one of their largest concerns.

“There are some people who just do not bend. The ones that do understand and listen, don’t need an explanation? Frustrating — the only cure for that (for me) was to lower my expectations. It was expecting someone to listen and respond that was worse than them not responding.” Alzheimer’s patient forum post

“I am currently on [Drug X] and it is working!! BUT . . . my insurance provider is now saying they will not pay for it.” Crohn’s patient forum post

“I’ve always been outgoing but my fear of having a seizure in public freaks me out!!! [. . .] And that’s what stops me from going out with my friends. They say they understand and it’s ok come out we are here for you but they haven’t got a bloody clue!!!!!” Epilepsy patient forum post

Fortunately, there is hope that some patients are being heard and that health care is improving for them. Within our sample of diabetes conversations, patients are more positively discussing aspects of their health and care. For example, many comments focus on patients understanding the course of their disease better and encouraging each other on weight-loss efforts. One of the key themes of the past 3 years is how patients are giving each other positive dietary advice on which ingredients lead to better control of their glucose. Also, more detailed discussion on glucose measurement and monitoring shows how patients have come to take ownership of this important facet of their care.

“I am happy to report that my a1c was 5.5 today! . . . my weight loss is now 54 pounds. thank you to everyone on this site that has given me encouragement and much needed information on this journey!” Diabetes patient forum post

Taken together, our data suggests that there is no one-size-fits-all-diseases approach to helping chronic patients deal with their quality-of-life issue, and that each disease has its own challenges.

Patient Pain Points

“I had my first doctor’s appointment . . . . he ran a slew of blood tests, did an xray and ordered a ct scan of my head. all of which came back completely normal . . . I told him I need [to be] referred to the next specialist in order to take the next step necessary to find an answer. I KNOW MY BODY AND SOMETHING IS NOT RIGHT!” Parkinson’s patient forum post

How do patients feel about their health care providers? As chronic patients struggle with quality-of-life issues, many have also discussed their issues with their physicians on these online forums.

To gain a better understanding of these issues, we identified the top pain points that patients express when discussing physicians.

Top Physician Pain Points Identified by Chronic Patients

  Click To Enlarge.

All of these pain points are rooted in emotions based on uncertainty and confusion, which leads to feelings of anxiety and distrust. Patients are not only seeking to understand their symptoms and diseases in more detail, but are also asking for help in understanding their test results, why certain treatments are chosen over others, and other questions beyond immediate diagnosis. Many of these comments reveal patients feel as though their physicians do not take the time to effectively explain everything to them, leading to greater feelings of stress, frustration, distrust, and confusion.

“I’m no expert but it seems that there are other stronger medications he could take to help his symptoms but his doctor is difficult to reach and when he does see him, he feels like he is rushed in and out. I think he needs another doctor.” Parkinson’s patient forum post

“This time the doctor just gave me a low dosage for 5 days and it has not done anything. She really does not want to prescribe this due to the bad side effects. Last night I was really depressed and feeling that maybe I have been misdiagnosed.” Psoriasis patient forum post

Solving these issues is not easy, and much of this may be rooted in the demand-driven time constraints faced by most physicians. There needs to be awareness, though, that patients are actively calling for engagement by their physicians to help them understand how to live with their diseases better, through greater knowledge, communication, and emotional support.

A Role for Patient Advocates

Of course, health care providers are not the only players in the health care ecosystem to directly interact with patients. Patient advocates exist to be a voice for patients who would otherwise not be heard. A patient can use an advocacy group for something as simple as finding a local physician or as daunting as negotiating with an insurance company. While these groups provide a variety of services, how well do they align with the unmet needs of chronic patients as discussed above?

To answer this, we analyzed the publicly available services and efforts of 1,200 advocacy organizations and identified seven distinct types of offerings.

Seven Types of Capabilities Offered by Patient Advocacy Organizations

  Click To Enlarge.

We see that the majority focus on clinical research, education, and support. When we cross-match these efforts to our unmet chronic patient needs, the needs are overall supported by advocacy groups, but the distribution of efforts is skewed toward a few needs in particular: developing more and better treatment options (29%) and coping with living with the condition (19%) are well covered. In comparison, understanding medication side effects (8%) and understanding the disease course (8%) have significantly less representation in explicit mentions among initiatives.

Percentage of Patient Advocacy Initiatives That Meet Patient Unmet Needs

  Click To Enlarge.

On the one hand, these results can be rationalized by the argument that within the top three unmet needs, there is a more natural fit for patient advocacy to address the coping need. In addition, some of patients’ unmet needs might be implicitly addressed as part of larger advocacy initiatives. That being said, we do see the opportunity for even greater engagement by advocacy groups to explicitly address some unmet needs such as connecting with other patients for support, educating them on medication and disease understanding, and management of disease progression.

Overall, hearing the collective voice of chronic patients online has demonstrated there is both necessity and opportunity for helping patients better live with their diseases. En masse, patients are asking more and more for all-encompassing treatment including both emotional and medical support. Shared decision-making and patient decision aids are becoming increasingly important to the clinical care experience, and patients will only grow in how much they turn to social media and online resources to seek out those with similar conditions and experiences. It is time for all those in health care — patients, providers, advocates, and others — to respect the importance of both the emotional and the medical elements of the patient experience, and to work together with patients to ensure they can live the lives that they choose.


Special thanks to our contributing authors: Emily Klein, Lucas Buchheim-Jurisson, Faroz Mujir.

Disclosures: ARTIS Ventures is an investor in Quid. UCB is a client of Quid.

New call for submissions

Now accepting submissions for NEJM Catalyst Innovations in Care Delivery, our new peer-reviewed journal


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Patient Engagement
PROMs Usage Is Limited but Growing

Patient Engagement Buzz Survey: PROMs Use Is Growing, but Implementation Takes Effort

While clinical outcomes lend themselves to measurement, quantifying how an individual patient experiences symptoms or a loss of function can be more challenging. One promising approach is the use of Patient-Reported Outcome Measures (PROMs).

Taxonomy of the Patient Voice Table

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care, the terms used to describe this goal are proliferating — and muddying the discourse. This taxonomy attempts to classify some of these terms and make some distinctions.

Agarwal01_pullquote handcrafting the patient experience

Handcrafting the Patient Experience

Health care organizations can take cues from consumer-facing companies like Airbnb to creatively insert convenience and surprise into patient encounters.

Screenshot of Tidepool daily diabetes data

A Taxonomy to Engage Patients: Objectives, Design, and Patient Activation

Health information initiatives will succeed only if they focus on patients’ motivation to engage and reflect the type of engagement they seek.

Ghafur01_pullquote -digital health health care consumer patient experience

Engaging Patients Using Digital Technology — Learning from Other Industries

Providers can benefit patients and disrupt health care by learning from the experience of other industries.

Mission Hospital Virtual Sitter - Drawing of the Virtual Bed Zone and Rails

Reducing Inpatient Falls and Injury Rates by Integrating New Technology with Workflow Redesign

How Mission Hospital scaled a virtual sitter pilot and reduced unassisted falls by 44% and fall-related injuries by 40%.

Health Care Providers Should Incentivize Patients

Survey Snapshot: Patient Financial Incentives — There Are No Quick Fixes

The NEJM Catalyst Insights Council agrees that while financial incentives are a common strategy to engage patients in healthy behaviors, they are not necessarily effective.

Support of Family and Friends Is More Effective Than Clinician Support in Realizing Health Goals - From the Patient Engagement Insights Report: Why No Single Health Incentive Works.

Patient Engagement Survey: Why No Single Health Incentive Works

Initiatives to improve patient engagement come in a variety of forms. While insurers, employers, and health care providers are all involved in using financial incentives and penalties for engagement efforts, improvement in health outcomes has been elusive. Achieving that ultimate goal will usually require a combination of financial and social approaches.

Health Systems Attending the Nudge Units in Health Care Symposium - Penn Medicine

Key Insights on Launching a Nudge Unit within a Health Care System

Leaders are finding that making higher-value choices easier through subtle changes to choice architecture can have an outsized impact on medical decision-making.

Barriers to Providing an Oustanding Patient Experience

Buzz Survey Report: Patient Experience

An independent NEJM Catalyst report sponsored by University of Utah Health on barriers to achieving an excellent patient experience.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Patient-Centered Care

282 Articles

Assessing and Addressing Social Needs in…

Lincoln Community Health Center improved care quality by measuring and responding to upstream social and…

Information Asymmetry: The Untapped Value of…

The knowledge and preferences that patients could — and should — share with clinicians would…

Proactively Catching the Declining Patient

A coordinated effort by UCLA leaders to identify a high-cost population with chronic kidney disease…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now