Care Redesign
Algorithms in Medicine: Identifying Cats vs. Cancer (11:19)

“All around us, it seems like algorithms are doing amazing things. They’re doing human things, and in fact, they’re doing a lot of human things better than humans,” says Ziad Obermeyer, Acting Associate Professor of Health Policy and Management at UC Berkeley.

Algorithms are, for example, predicting the weather, predicting traffic, and driving cars. “And if algorithms can get a driver’s license, it’s not so far-fetched to believe that, one day, they’ll be able to get a medical license, too,” Obermeyer observes. They could predict who will have a heart attack or which patients will generate massive costs, for example, at levels of performance far exceeding human capability.

At some point, will humans no longer be required?

“If we buy even a small part of that vision, there is a lot at stake,” says Obermeyer. He goes on to describe how algorithms work: the first ingredient is finding massive data sets; the second is defining an outcome of interest, called the label. Algorithms then extract signal from data to predict the outcome.

This has worked in lots of other industries. But medicine is different. To illustrate that difference, Obermeyer points to a photo of a cat, noting that we can all agree it is a cat. He then points to an X-ray of a lung and a nodule within it, stating that we can’t easily agree, from that photo, on any specific details about the lung nodule such as calcification.

Obermeyer cat and lung nodule slide

  Click To Enlarge.

Even if we could, those details are not really what providers need to know. The important questions are: Is it going to spread? Will it kill my patient? Can I keep an eye on it, or do I need to remove it?

“We started off thinking we were answering a simple software engineering problem along the lines of, is this a cat? But somehow, along the way, it’s become a much deeper medical problem. Is it cancer? Or, what is cancer?” Obermeyer explains.

You might think, “Sure, you’re telling me the world is complicated,” and “we got some data and predicted an outcome, so let’s just get on with it in medicine.” Many companies, in fact, make billion-dollar bets along that line of thinking.

If you gloss over that truth problem, obtain the data, and predict what you can — maybe train the algorithm to take an image and spit out what a radiologist would say about the image, or have the data predict a cancer diagnosis — the algorithm won’t predict an abstract truth but would presumably still unleash the power of your data. But that’s not necessarily useful to medicine.

Returning to his cat example, Obermeyer tells the audience to pretend we’re in the cat detection business and that we’ve found a database of images that are already labeled “cat” or “no cat.” The people who labeled the cats are preschoolers who weren’t always great at finding cats because they didn’t fully understand what cats were, but they were incentivized with the promise of extra dessert for choosing cats. The problem is, some of the images are hard to call, and we have to take the labelers’ word for it.

Obermeyer cat vs ice cream slide

  Click To Enlarge.

“Picture our algorithm learning about cats from this data set. Are there going to be too many cats or are there going to be not enough cats?” asks Obermeyer. “It’s like a race between human error and human incentives. I honestly don’t know which one you want to bet on, but there’s one thing that’s pretty clear: This is not good for our cat detection business.”

“Imagine a medical system where, every day, we’re confronted by the limits of our understanding, our inability to diagnose cancer, to prevent renal failure, a medical system that’s run by people who are basically good but also make mistakes, and a medical system built on deeply flawed financial incentives that push us to over-diagnose, over-test, upstage, and overcall,” he says. “Also a system that systematically excludes some groups from diagnosis and treatment so that algorithms never have a chance to learn their stories, the story of the cancer that was never diagnosed in the woman who could not afford to take a day off work for her biopsy.”

“If we’re not careful with how we use data from that medical system, we’ll take the enormous power of algorithms, and we’ll use them to automate and scale up our ignorance and our greed,” Obermeyer warns. “That’s why medical problems are so different from software engineering problems. Medicine is different from other fields where algorithms have taken over. Our health care system, and even more so, the human body, are unimaginably, terrifyingly complex, and if we’re honest with ourselves, most of the time, we have absolutely no clue what’s going on.”

“What is sepsis? Why do statins work? Why do some people’s hearts just stop working when they’re in the middle of a basketball game? These are deep, fundamental questions, and there’s no app for these questions,” says Obermeyer.

But that doesn’t mean algorithms have no role to play, he adds. “With care and with effort, and with obsessive attention to detail, we can turn that enormous power of algorithms in to the service of our patients and in the service of a new kind of medical science.”

Instead of asking algorithms to learn from humans and reproduce human judgment — including our errors, biases, and incentives — what if we asked them to learn from nature? Not from insurance claims or electronic health records, but from biological data.

“We need algorithms to learn real outcomes from real data that don’t depend on how much you can pay or what a doctor wrote down about you in her note,” Obermeyer says. These data are the beginning of a new medical science grounded in data and powered by algorithms.

To illustrate this, Obermeyer tells the story of a cardiologist in 1986 Holland, who was called to consult on a 3-year-old boy who’d suffered a cardiac arrest. Unlike many others with his condition, the boy survived thanks to his father’s CPR. However, his older sister had died from the same condition several years before, her father unable to save her. The boy’s electrocardiogram (ECG) showed an unusual pattern; his sister’s ECG, tracked down from a hospital in Poland, looked identical.

Over the next 5 years, the cardiologist collected ECGs from similar cases all over the world, but medical journals were not interested in publishing his work. The cardiologist thought often of giving up, but his brother, also a cardiologist, helped shoulder the burden.

One night, that cardiologist ate dinner with the Editor-in-Chief of the Journal of the American College of Cardiology, who eventually accepted his paper. That chain of events is how we now know Brugada syndrome, and how we’ve saved countless lives by implanting defibrillators before people suffer cardiac arrest.

If we fast-forward 30 years, how might that process look today, in the age of data and algorithms? Thanks to a regional health care system in Sweden, Obermeyer’s lab at UC Berkeley has access to every electrocardiogram ever done in the region. They’ve linked the ECGs to high-quality death certificate data with cause of death and are mining ECG wave forms for the micro-signals that might one day predict and prevent sudden cardiac death.

“All of this with algorithms and at scale and without relying on luck, or the attentiveness and persistence of any one doctor,” says Obermeyer. This is one of many examples of the work his lab and others are doing. “We are looking for partners because we need data, and we need a lot of people to share the work because there is an enormous amount of work to do.”

“That’s why none of us should be worried about our jobs as doctors,” Obermeyer concludes. “It’s not because of what people say: white-collar workers can’t be automated, or government regulations are going to protect us, or we need our soft, human touch in medicine. It’s none of those things. It’s because, if we get this right, we’re going to be a fundamentally new science grounded in data and algorithms.”

“Algorithms aren’t going to put you out of work. They’re going to generate a lot more work for you as we go. I hope you’re ready.”

From the NEJM Catalyst event Provider-Driven Data Analytics to Improve Outcomes, held at Cedars-Sinai Medical Center, January 31, 2019.

More From Care Redesign
Kimberly Dennis and David Newton head shots - SunCloud Mental Health Services

Addressing the Lack of Continuity of Care in Mental Health Services

Co-founders of an integrative outpatient treatment center for mental health and addiction discuss the problematic lack of continuity of care between inpatient and outpatient and physical and mental health services.

Epic OB Hemorrhage PPH Risk Assessment Tool and Alerts at PSJH - Preventing Maternal Death

Improving Maternal Safety Through an EMR Risk Assessment

After a mother died of postpartum hemorrhage, Providence St. Joseph Health made organization-wide changes to prevent future maternal deaths and injuries.

Most Health Care Organizations Have Palliative or End-of-Life Care Programs

Survey Snapshot: Challenging the Resistance to “Palliative”

NEJM Catalyst Insights Council members agree that palliative care is gaining traction, but one of many barriers is getting providers over their resistance to the word “palliative.”

The Assessment of Care Tool - Consisting of Six Visual Analog Scales Corresponding with the IOM Six Dimensions of Perfect Care

Real-Time Pursuit of Outcomes That Matter to Patients

A simple and affordable tool to use at the point of care to drive value creation within clinical microsystems.

Many Patients Who Would Benefit from Palliative - End-of-Life Care Do Not Receive It

Care Redesign Survey: The Power of Palliative Care

Our NEJM Catalyst Insights Council survey on palliative care reveals an interesting dichotomy: While the great majority of organizations have a palliative or end-of-life care program, 60% of patients who would benefit from such services don’t receive them.

Heart Safe Motherhood and Way to Health Two-Way Texting for Blood Pressure Monitoring for Postpartum Women with Preeclampsia

Heart Safe Motherhood: Applying Innovation Methodology for Improved Maternal Outcomes

At the Hospital of the University of Pennsylvania, a text message–based blood pressure surveillance program for postpartum women with preeclampsia improved blood pressure management, reduced readmissions, and increased patient and provider satisfaction.

VHA Whole Health System diagram

Finding the Cause of the Crises: Opioids, Pain, Suicide, Obesity, and Other “Epidemics”

Until we redesign our health care system to address our patients’ personal determinants of health, we will continue to inadequately address our multiple chronic disease crises.

Leff06_pullquote home-based medical care for homebound patients

Using Quality to Shine a Light on Homebound Care

How two thought leaders in the fields of home-based medical care, geriatrics, and palliative medicine advanced a quality-of-care agenda for homebound adults.

Charlotte Yeh head shot - hearing aids hearing loss

“You’re Old Without Hearing Aids”— Addressing the Silent Epidemic of Hearing Loss

Hearing loss isn’t a normal consequence of aging. But it is associated with a higher risk of dementia, depression, and falls. The Chief Medical Officer for AARP Services talks about combating this huge but silent epidemic that impacts all ages.

Dentzer01_pullquote - Stone-Age Policies Stifle Modern Virtual Care Solutions

Stone-Age Policies Stifle Modern Solutions

Health care leaders must advocate for regulatory and reimbursement changes to unlock the potential of innovative technology and care team approaches to Parkinson’s and other suitable conditions.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


How a State Advances Whole-Person Health…

Pennsylvania addresses social determinants of health by bringing together managed care and social services organizations…

Quality Management

178 Articles

Addressing the Lack of Continuity of…

Co-founders of an integrative outpatient treatment center for mental health and addiction discuss the problematic…

Design Thinking

18 Articles

Heart Safe Motherhood: Applying Innovation Methodology…

At the Hospital of the University of Pennsylvania, a text message–based blood pressure surveillance program…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now