Care Redesign
Algorithms in Medicine: Identifying Cats vs. Cancer (11:19)

“All around us, it seems like algorithms are doing amazing things. They’re doing human things, and in fact, they’re doing a lot of human things better than humans,” says Ziad Obermeyer, Acting Associate Professor of Health Policy and Management at UC Berkeley.

Algorithms are, for example, predicting the weather, predicting traffic, and driving cars. “And if algorithms can get a driver’s license, it’s not so far-fetched to believe that, one day, they’ll be able to get a medical license, too,” Obermeyer observes. They could predict who will have a heart attack or which patients will generate massive costs, for example, at levels of performance far exceeding human capability.

At some point, will humans no longer be required?

“If we buy even a small part of that vision, there is a lot at stake,” says Obermeyer. He goes on to describe how algorithms work: the first ingredient is finding massive data sets; the second is defining an outcome of interest, called the label. Algorithms then extract signal from data to predict the outcome.

This has worked in lots of other industries. But medicine is different. To illustrate that difference, Obermeyer points to a photo of a cat, noting that we can all agree it is a cat. He then points to an X-ray of a lung and a nodule within it, stating that we can’t easily agree, from that photo, on any specific details about the lung nodule such as calcification.

Obermeyer cat and lung nodule slide

  Click To Enlarge.

Even if we could, those details are not really what providers need to know. The important questions are: Is it going to spread? Will it kill my patient? Can I keep an eye on it, or do I need to remove it?

“We started off thinking we were answering a simple software engineering problem along the lines of, is this a cat? But somehow, along the way, it’s become a much deeper medical problem. Is it cancer? Or, what is cancer?” Obermeyer explains.

You might think, “Sure, you’re telling me the world is complicated,” and “we got some data and predicted an outcome, so let’s just get on with it in medicine.” Many companies, in fact, make billion-dollar bets along that line of thinking.

If you gloss over that truth problem, obtain the data, and predict what you can — maybe train the algorithm to take an image and spit out what a radiologist would say about the image, or have the data predict a cancer diagnosis — the algorithm won’t predict an abstract truth but would presumably still unleash the power of your data. But that’s not necessarily useful to medicine.

Returning to his cat example, Obermeyer tells the audience to pretend we’re in the cat detection business and that we’ve found a database of images that are already labeled “cat” or “no cat.” The people who labeled the cats are preschoolers who weren’t always great at finding cats because they didn’t fully understand what cats were, but they were incentivized with the promise of extra dessert for choosing cats. The problem is, some of the images are hard to call, and we have to take the labelers’ word for it.

Obermeyer cat vs ice cream slide

  Click To Enlarge.

“Picture our algorithm learning about cats from this data set. Are there going to be too many cats or are there going to be not enough cats?” asks Obermeyer. “It’s like a race between human error and human incentives. I honestly don’t know which one you want to bet on, but there’s one thing that’s pretty clear: This is not good for our cat detection business.”

“Imagine a medical system where, every day, we’re confronted by the limits of our understanding, our inability to diagnose cancer, to prevent renal failure, a medical system that’s run by people who are basically good but also make mistakes, and a medical system built on deeply flawed financial incentives that push us to over-diagnose, over-test, upstage, and overcall,” he says. “Also a system that systematically excludes some groups from diagnosis and treatment so that algorithms never have a chance to learn their stories, the story of the cancer that was never diagnosed in the woman who could not afford to take a day off work for her biopsy.”

“If we’re not careful with how we use data from that medical system, we’ll take the enormous power of algorithms, and we’ll use them to automate and scale up our ignorance and our greed,” Obermeyer warns. “That’s why medical problems are so different from software engineering problems. Medicine is different from other fields where algorithms have taken over. Our health care system, and even more so, the human body, are unimaginably, terrifyingly complex, and if we’re honest with ourselves, most of the time, we have absolutely no clue what’s going on.”

“What is sepsis? Why do statins work? Why do some people’s hearts just stop working when they’re in the middle of a basketball game? These are deep, fundamental questions, and there’s no app for these questions,” says Obermeyer.

But that doesn’t mean algorithms have no role to play, he adds. “With care and with effort, and with obsessive attention to detail, we can turn that enormous power of algorithms in to the service of our patients and in the service of a new kind of medical science.”

Instead of asking algorithms to learn from humans and reproduce human judgment — including our errors, biases, and incentives — what if we asked them to learn from nature? Not from insurance claims or electronic health records, but from biological data.

“We need algorithms to learn real outcomes from real data that don’t depend on how much you can pay or what a doctor wrote down about you in her note,” Obermeyer says. These data are the beginning of a new medical science grounded in data and powered by algorithms.

To illustrate this, Obermeyer tells the story of a cardiologist in 1986 Holland, who was called to consult on a 3-year-old boy who’d suffered a cardiac arrest. Unlike many others with his condition, the boy survived thanks to his father’s CPR. However, his older sister had died from the same condition several years before, her father unable to save her. The boy’s electrocardiogram (ECG) showed an unusual pattern; his sister’s ECG, tracked down from a hospital in Poland, looked identical.

Over the next 5 years, the cardiologist collected ECGs from similar cases all over the world, but medical journals were not interested in publishing his work. The cardiologist thought often of giving up, but his brother, also a cardiologist, helped shoulder the burden.

One night, that cardiologist ate dinner with the Editor-in-Chief of the Journal of the American College of Cardiology, who eventually accepted his paper. That chain of events is how we now know Brugada syndrome, and how we’ve saved countless lives by implanting defibrillators before people suffer cardiac arrest.

If we fast-forward 30 years, how might that process look today, in the age of data and algorithms? Thanks to a regional health care system in Sweden, Obermeyer’s lab at UC Berkeley has access to every electrocardiogram ever done in the region. They’ve linked the ECGs to high-quality death certificate data with cause of death and are mining ECG wave forms for the micro-signals that might one day predict and prevent sudden cardiac death.

“All of this with algorithms and at scale and without relying on luck, or the attentiveness and persistence of any one doctor,” says Obermeyer. This is one of many examples of the work his lab and others are doing. “We are looking for partners because we need data, and we need a lot of people to share the work because there is an enormous amount of work to do.”

“That’s why none of us should be worried about our jobs as doctors,” Obermeyer concludes. “It’s not because of what people say: white-collar workers can’t be automated, or government regulations are going to protect us, or we need our soft, human touch in medicine. It’s none of those things. It’s because, if we get this right, we’re going to be a fundamentally new science grounded in data and algorithms.”

“Algorithms aren’t going to put you out of work. They’re going to generate a lot more work for you as we go. I hope you’re ready.”

From the NEJM Catalyst event Provider-Driven Data Analytics to Improve Outcomes, held at Cedars-Sinai Medical Center, January 31, 2019.

More From Care Redesign
Community Resource Referral Type

Assessing and Addressing Social Needs in Primary Care

Lincoln Community Health Center improved care quality by measuring and responding to upstream social and economic risk factors disproportionately affecting low-income households.

Time Driven Activity Based Costing for ECMO

Achieving Value in Highly Complex Acute Care: Lessons from the Delivery of Extra Corporeal Life Support

To improve both the value and outcomes of ECLS, Cedars-Sinai Medical Center created guidelines for ECLS delivery and explored opportunities for more efficient care.

ARISE and SFHN BHVS Collaboration

The “Behavioral Health Vital Signs” Initiative

A safety net system’s trauma-informed approach to integrating interpersonal violence into behavioral health programs in primary care.

OpenNotes Epic Patient Email Cascade Chart February 1, 2018 - August 14, 2018

Measuring Performance of OpenNotes Initiatives to Target Improvement Efforts

How a New York safety-net health system used data science to identify obstacles to OpenNotes use, address technical barriers, and develop strategies for improving clinical note sharing by providers and viewing by patients.

Mapping a Technology Strategy for Bundled Payment Care Using a Value-Driven Framework

Harnessing Emerging Information Technology for Bundled Payment Care Using a Value-Driven Framework

A four-part framework developed by physicians at Partners HealthCare provides a stepwise process for assessing and integrating technologies to effectively use data through a continuous patient experience.

UCLA Health CKD Risk Stratification and Management

Proactively Catching the Declining Patient

A coordinated effort by UCLA leaders to identify a high-cost population with chronic kidney disease and to modify care processes and personnel has led to improved health and reduced utilization.

Telehealth and remote monitoring are little used and ineffective for chronic disease care

Survey Snapshot: Treating Chronic Disease Proactively

Though survey respondents don’t indicate strong use of telehealth and remote monitoring, NEJM Catalyst Insights Council members discuss the ways they’re using these tools to monitor chronic disease, with good results.

Platforming Health Care Operations - Consumer-Driven Health Care - Business-Minded Optimizations

Platforming Health Care to Transform Care Delivery

Health care leaders need to focus less on ownership and control of the delivery process, and more on outcomes, cost efficiency, and customer experience.

Shah05_ integrated systems innovation pullquote

Build vs. Buy: What Should Health Systems Do?

The consolidation craze continues, but vertical integration has yet to demonstrate real progress toward the Triple Aim. Health care leaders would do well to consider innovative approaches that are working in other industries, including the tech-enabled full stack model.

Diagram Illustrating the COPD Care Pathway at Allegheny General Hospital

End-to-End Care for COPD Patients that Improves Outcomes and Lowers Costs

Allegheny General Hospital created a comprehensive solution for patients with chronic obstructive pulmonary disease (COPD) that led to improved clinical outcomes, reduced hospital admissions and readmissions, and a resultant decrease in the total cost of care.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Design Thinking

20 Articles

Taxonomy of the Patient Voice

While health care pursues the important trend of putting patients at the center of care,…

Measuring Performance of OpenNotes Initiatives to…

How a New York safety-net health system used data science to identify obstacles to OpenNotes…

Proactively Catching the Declining Patient

A coordinated effort by UCLA leaders to identify a high-cost population with chronic kidney disease…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now