Care Redesign
Algorithms in Medicine: Identifying Cats vs. Cancer (11:19)

“All around us, it seems like algorithms are doing amazing things. They’re doing human things, and in fact, they’re doing a lot of human things better than humans,” says Ziad Obermeyer, Acting Associate Professor of Health Policy and Management at UC Berkeley.

Algorithms are, for example, predicting the weather, predicting traffic, and driving cars. “And if algorithms can get a driver’s license, it’s not so far-fetched to believe that, one day, they’ll be able to get a medical license, too,” Obermeyer observes. They could predict who will have a heart attack or which patients will generate massive costs, for example, at levels of performance far exceeding human capability.

At some point, will humans no longer be required?

“If we buy even a small part of that vision, there is a lot at stake,” says Obermeyer. He goes on to describe how algorithms work: the first ingredient is finding massive data sets; the second is defining an outcome of interest, called the label. Algorithms then extract signal from data to predict the outcome.

This has worked in lots of other industries. But medicine is different. To illustrate that difference, Obermeyer points to a photo of a cat, noting that we can all agree it is a cat. He then points to an X-ray of a lung and a nodule within it, stating that we can’t easily agree, from that photo, on any specific details about the lung nodule such as calcification.

Obermeyer cat and lung nodule slide

  Click To Enlarge.

Even if we could, those details are not really what providers need to know. The important questions are: Is it going to spread? Will it kill my patient? Can I keep an eye on it, or do I need to remove it?

“We started off thinking we were answering a simple software engineering problem along the lines of, is this a cat? But somehow, along the way, it’s become a much deeper medical problem. Is it cancer? Or, what is cancer?” Obermeyer explains.

You might think, “Sure, you’re telling me the world is complicated,” and “we got some data and predicted an outcome, so let’s just get on with it in medicine.” Many companies, in fact, make billion-dollar bets along that line of thinking.

If you gloss over that truth problem, obtain the data, and predict what you can — maybe train the algorithm to take an image and spit out what a radiologist would say about the image, or have the data predict a cancer diagnosis — the algorithm won’t predict an abstract truth but would presumably still unleash the power of your data. But that’s not necessarily useful to medicine.

Returning to his cat example, Obermeyer tells the audience to pretend we’re in the cat detection business and that we’ve found a database of images that are already labeled “cat” or “no cat.” The people who labeled the cats are preschoolers who weren’t always great at finding cats because they didn’t fully understand what cats were, but they were incentivized with the promise of extra dessert for choosing cats. The problem is, some of the images are hard to call, and we have to take the labelers’ word for it.

Obermeyer cat vs ice cream slide

  Click To Enlarge.

“Picture our algorithm learning about cats from this data set. Are there going to be too many cats or are there going to be not enough cats?” asks Obermeyer. “It’s like a race between human error and human incentives. I honestly don’t know which one you want to bet on, but there’s one thing that’s pretty clear: This is not good for our cat detection business.”

“Imagine a medical system where, every day, we’re confronted by the limits of our understanding, our inability to diagnose cancer, to prevent renal failure, a medical system that’s run by people who are basically good but also make mistakes, and a medical system built on deeply flawed financial incentives that push us to over-diagnose, over-test, upstage, and overcall,” he says. “Also a system that systematically excludes some groups from diagnosis and treatment so that algorithms never have a chance to learn their stories, the story of the cancer that was never diagnosed in the woman who could not afford to take a day off work for her biopsy.”

“If we’re not careful with how we use data from that medical system, we’ll take the enormous power of algorithms, and we’ll use them to automate and scale up our ignorance and our greed,” Obermeyer warns. “That’s why medical problems are so different from software engineering problems. Medicine is different from other fields where algorithms have taken over. Our health care system, and even more so, the human body, are unimaginably, terrifyingly complex, and if we’re honest with ourselves, most of the time, we have absolutely no clue what’s going on.”

“What is sepsis? Why do statins work? Why do some people’s hearts just stop working when they’re in the middle of a basketball game? These are deep, fundamental questions, and there’s no app for these questions,” says Obermeyer.

But that doesn’t mean algorithms have no role to play, he adds. “With care and with effort, and with obsessive attention to detail, we can turn that enormous power of algorithms in to the service of our patients and in the service of a new kind of medical science.”

Instead of asking algorithms to learn from humans and reproduce human judgment — including our errors, biases, and incentives — what if we asked them to learn from nature? Not from insurance claims or electronic health records, but from biological data.

“We need algorithms to learn real outcomes from real data that don’t depend on how much you can pay or what a doctor wrote down about you in her note,” Obermeyer says. These data are the beginning of a new medical science grounded in data and powered by algorithms.

To illustrate this, Obermeyer tells the story of a cardiologist in 1986 Holland, who was called to consult on a 3-year-old boy who’d suffered a cardiac arrest. Unlike many others with his condition, the boy survived thanks to his father’s CPR. However, his older sister had died from the same condition several years before, her father unable to save her. The boy’s electrocardiogram (ECG) showed an unusual pattern; his sister’s ECG, tracked down from a hospital in Poland, looked identical.

Over the next 5 years, the cardiologist collected ECGs from similar cases all over the world, but medical journals were not interested in publishing his work. The cardiologist thought often of giving up, but his brother, also a cardiologist, helped shoulder the burden.

One night, that cardiologist ate dinner with the Editor-in-Chief of the Journal of the American College of Cardiology, who eventually accepted his paper. That chain of events is how we now know Brugada syndrome, and how we’ve saved countless lives by implanting defibrillators before people suffer cardiac arrest.

If we fast-forward 30 years, how might that process look today, in the age of data and algorithms? Thanks to a regional health care system in Sweden, Obermeyer’s lab at UC Berkeley has access to every electrocardiogram ever done in the region. They’ve linked the ECGs to high-quality death certificate data with cause of death and are mining ECG wave forms for the micro-signals that might one day predict and prevent sudden cardiac death.

“All of this with algorithms and at scale and without relying on luck, or the attentiveness and persistence of any one doctor,” says Obermeyer. This is one of many examples of the work his lab and others are doing. “We are looking for partners because we need data, and we need a lot of people to share the work because there is an enormous amount of work to do.”

“That’s why none of us should be worried about our jobs as doctors,” Obermeyer concludes. “It’s not because of what people say: white-collar workers can’t be automated, or government regulations are going to protect us, or we need our soft, human touch in medicine. It’s none of those things. It’s because, if we get this right, we’re going to be a fundamentally new science grounded in data and algorithms.”

“Algorithms aren’t going to put you out of work. They’re going to generate a lot more work for you as we go. I hope you’re ready.”

From the NEJM Catalyst event Provider-Driven Data Analytics to Improve Outcomes, held at Cedars-Sinai Medical Center, January 31, 2019.

More From Care Redesign
Diagram Illustrating the COPD Care Pathway at Allegheny General Hospital

End-to-End Care for COPD Patients that Improves Outcomes and Lowers Costs

Allegheny General Hospital created a comprehensive solution for patients with chronic obstructive pulmonary disease (COPD) that led to improved clinical outcomes, reduced hospital admissions and readmissions, and a resultant decrease in the total cost of care.

David Blumenthal and Bob Galvin head shots

Ripe for Disruption: Why and How Big Players in the Private Sector Are Taking on Health Care

For big tech companies like Amazon, Apple, and Google, the health care sector looks ripe for disruption. Two executives working in different parts of the health care ecosystem discuss what this means for patients and doctors, including the positives and unintended consequences.

Top challenges facing chronic disease management care - insufficient time and care coordination

Care Redesign Survey: To Improve Chronic Disease Care, Change the Payment Model

Many health care organizations are reasonably effective in treating chronic diseases, but they are limited from doing better by fee-for-service payment, which remains the predominant payment model in the United States. This report serves as a snapshot in time, showing the intent of health care providers to be proactive in treating chronic disease, but limitations in their ability to address population health.

Nirav Shah head shot Stanford - AI in medicine and team care

What AI Means for Doctors and Doctoring

Physicians must hone the “four Cs” — critical thinking, communication, collaboration, and creativity — when leveraging AI as a new partner in their care teams.

End of Life EOL Palliative Care in the ED for Patients with Advanced Cancer - Process Map - MD Anderson Cancer Center

Patient-Centered Care at the End of Life in the ED

How MD Anderson Cancer Center is improving end-of-life care in an unlikely place: the emergency department.

Home-Based Cardiac Rehab - An Overview from Kaiser Permanente Southern California

Saving Lives with Virtual Cardiac Rehabilitation

Collaboration and innovation can improve the performance of cardiac rehabilitation.

Murali01_pullquote Home Recovery Care patient satisfaction

No Place Like Home: Bringing Inpatient Care to the Patient

Providing home-based acute care improves patient satisfaction and care quality while reducing costs.

Rating the Raters - Strengths and Weaknesses Assessment of the Four Public Hospital Quality Rating Systems - 2a

Rating the Raters: An Evaluation of Publicly Reported Hospital Quality Rating Systems

Some promising innovation is taking place among organizations that rate hospital performance, but major systemic change is needed in the field to ensure access to meaningful comparisons through better data and relevant metrics, and to establish integrated oversight through robust audits and peer review.

McKee01_pullquote - the need for coordinated care IPUs for Parkinson's disease

Creating “One-Stop Shop” Care for Parkinson’s

Integrated Practice Units (IPUs) can revolutionize the care of specialty disease conditions, and Parkinson’s disease is a good place to start.

Good Shepherd culturally competent hospice care home visit to the widow of a recently deceased patient

Strangers No More: Culturally Competent Add-On Programs for Diverse Seniors

Creating specialized culturally competent programs to improve patient satisfaction and address the unique health care needs of older immigrants.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Design Thinking

19 Articles

Handcrafting the Patient Experience

Health care organizations can take cues from consumer-facing companies like Airbnb to creatively insert convenience…

Ripe for Disruption: Why and How…

For big tech companies like Amazon, Apple, and Google, the health care sector looks ripe…

Care Redesign Survey: To Improve Chronic…

Many health care organizations are reasonably effective in treating chronic diseases, but they are limited…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now