Care Redesign
Relentless Reinvention

Lost in Thought — The Limits of the Human Mind and the Future of Medicine

Article · October 30, 2017

Interview with Dr. Ziad Obermeyer on how collaboration between doctors and computers will help improve medical care. 


In the good old days, clinicians thought in groups; “rounding,” whether on the wards or in the radiology reading room, was a chance for colleagues to work together on problems too difficult for any single mind to solve.

Today, thinking looks very different: we do it alone, bathed in the blue light of computer screens.

Our knee-jerk reaction is to blame the computer, but the roots of this shift run far deeper. Medical thinking has become vastly more complex, mirroring changes in our patients, our health care system, and medical science. The complexity of medicine now exceeds the capacity of the human mind.

Computers, far from being the problem, are the solution. But using them to manage the complexity of 21st-century medicine will require fundamental changes in the way we think about thinking and in the structure of medical education and research.

It’s ironic that just when clinicians feel that there’s no time in their daily routines for thinking, the need for deep thinking is more urgent than ever. Medical knowledge is expanding rapidly, with a widening array of therapies and diagnostics fueled by advances in immunology, genetics, and systems biology. Patients are older, with more coexisting illnesses and more medications. They see more specialists and undergo more diagnostic testing, which leads to exponential accumulation of electronic health record (EHR) data. Every patient is now a “big data” challenge, with vast amounts of information on past trajectories and current states.

All this information strains our collective ability to think. Medical decision making has become maddeningly complex. Patients and clinicians want simple answers, but we know little about whom to refer for BRCA testing or whom to treat with PCSK9 inhibitors. Common processes that were once straightforward — ruling out pulmonary embolism or managing new atrial fibrillation — now require numerous decisions.

So, it’s not surprising that we get many of these decisions wrong. Most tests come back negative, yet misdiagnosis remains common.1 Patients seeking emergency care are often admitted to the hospital unnecessarily, yet many also die suddenly soon after being sent home.2 Overall, we provide far less benefit to our patients than we hope. These failures contribute to deep dissatisfaction and burnout among doctors and threaten the health care system’s financial sustainability.

If a root cause of our challenges is complexity, the solutions are unlikely to be simple. Asking doctors to work harder or get smarter won’t help. Calls to reduce “unnecessary” care fall flat: we all know how difficult it’s become to identify what care is necessary. Changing incentives is an appealing lever for policymakers, but that alone will not make decisions any easier: we can reward physicians for delivering less care, but the end result may simply be less care, not better care.

The first step toward a solution is acknowledging the profound mismatch between the human mind’s abilities and medicine’s complexity. Long ago, we realized that our inborn sensorium was inadequate for scrutinizing the body’s inner workings — hence, we developed microscopes, stethoscopes, electrocardiograms, and radiographs. Will our inborn cognition alone solve the mysteries of health and disease in a new century? The state of our health care system offers little reason for optimism.

But there is hope. The same computers that today torment us with never-ending checkboxes and forms will tomorrow be able to process and synthesize medical data in ways we could never do ourselves. Already, there are indications that data science can help us with critical problems.

Consider the challenge of reading electrocardiograms. Doctors look for a handful of features to diagnose ischemia or rhythm disturbances — but can we ever truly “read” the waveforms in a 10-second tracing, let alone the multiple-day recording of a Holter monitor? Algorithms, by contrast, can systematically analyze every heartbeat. There are early signs that such analyses can identify subtle microscopic variations linked to sudden cardiac death.3 If validated, such algorithms could help us identify and treat the tens of thousands of Americans who might otherwise drop dead unexpectedly in any given year. And they could guide basic research on the mechanisms of newly discovered predictors.

Collaboration Between Doctors and Computers - Machine Learning

  Click To Enlarge.

Algorithms have also been deployed for an analysis of massive amounts of EHR data whose results suggest that type 2 diabetes has three subtypes, each with its own biologic signature and disease trajectory.4 Knowing which type of patients we’re dealing with can help us deliver treatments to those who benefit most and may help us understand why some patients have complications and others don’t.

There is little doubt that algorithms will transform the thinking underlying medicine. The only question is whether this transformation will be driven by forces from within or outside the field. If medicine wishes to stay in control of its own future, physicians will not only have to embrace algorithms, they will also have to excel at developing and evaluating them, bringing machine-learning methods into the medical domain.

Machine learning has already spurred innovation in fields ranging from astrophysics to ecology. In these disciplines, the expert advice of computer scientists is sought when cutting-edge algorithms are needed for thorny problems, but experts in the field — astrophysicists or ecologists — set the research agenda and lead the day-to-day business of applying machine learning to relevant data.

In medicine, by contrast, clinical records are considered treasure troves of data for researchers from nonclinical disciplines. Physicians are not needed to enroll patients — so they’re consulted only occasionally, perhaps to suggest an interesting outcome to predict. They are far from the intellectual center of the work and rarely engage meaningfully in thinking about how algorithms are developed or what would happen if they were applied clinically.

But ignoring clinical thinking is dangerous. Imagine a highly accurate algorithm that uses EHR data to predict which emergency department patients are at high risk for stroke. It would learn to diagnose stroke by churning through large sets of routinely collected data. Critically, all these data are the product of human decisions: a patient’s decision to seek care, a doctor’s decision to order a test, a diagnostician’s decision to call the condition a stroke. Thus, rather than predicting the biologic phenomenon of cerebral ischemia, the algorithm would predict the chain of human decisions leading to the coding of stroke.

Algorithms that learn from human decisions will also learn human mistakes, such as overtesting and overdiagnosis, failing to notice people who lack access to care, undertesting those who cannot pay, and mirroring race or gender biases. Ignoring these facts will result in automating and even magnifying problems in our current health system.5 Noticing and undoing these problems requires a deep familiarity with clinical decisions and the data they produce — a reality that highlights the importance of viewing algorithms as thinking partners, rather than replacements, for doctors.

Ultimately, machine learning in medicine will be a team sport, like medicine itself. But the team will need some new players: clinicians trained in statistics and computer science, who can contribute meaningfully to algorithm development and evaluation. Today’s medical education system is ill prepared to meet these needs. Undergraduate premedical requirements are absurdly outdated. Medical education does little to train doctors in the data science, statistics, or behavioral science required to develop, evaluate, and apply algorithms in clinical practice.

The integration of data science and medicine is not as far away as it may seem: cell biology and genetics, once also foreign to medicine, are now at the core of medical research, and medical education has made all doctors into informed consumers of these fields. Similar efforts in data science are urgently needed. If we lay the groundwork today, 21st-century clinicians can have the tools they need to process data, make decisions, and master the complexity of 21st-century patients.

From Brigham and Women’s Hospital and Harvard Medical School, Boston (Z.O., T.H.L.), and Press Ganey, Wakefield (T.H.L.) — both in Massachusetts.

1. Institute of Medicine. Improving diagnosis in health care. Washington, DC: National Academies Press, 2015.
2. Obermeyer Z, Cohn B, Wilson M, Jena AB, Cutler DM. Early death after discharge from emergency departments: analysis of national US insurance claims data. BMJ 2017;356:j239-j239. CrossRef | Web of Science | Medline
3. Syed Z, Stultz CM, Scirica BM, Guttag JV. Computationally generated cardiac biomarkers for risk stratification after acute coronary syndrome. Sci Transl Med 2011;3:102ra95-102ra95. CrossRef | Web of Science | Medline
4. Li L, Cheng W-Y, Glicksberg BS, et al. Identification of type 2 diabetes subgroups through topological analysis of patient similarity. Sci Transl Med 2015;7:311ra174-311ra174. CrossRef | Web of Science | Medline
5. Mullainathan S, Obermeyer Z. Does machine learning automate moral hazard and error? Am Econ Rev 2017;107:476-480. CrossRef | Web of Science | Medline

This Perspective article originally appeared in The New England Journal of Medicine.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Care Redesign

What Is Lean Healthcare?

Lean Healthcare is the application of “lean” ideas in healthcare to minimize waste with ongoing process improvement. Learn how to use Lean to improve patient satisfaction and care outcomes while reducing costs.

Zane05_pullquote3 hospital decentralization

Do Hospitals Still Make Sense? The Case for Decentralization of Health Care

The future is here: moving care out of the hospital and into the home and community.

Sociotechnical dimensions used to analyze EHR-related health IT safety concerns

Frustrated with Your EHR? Don’t Blame Your Vendor — Safety Is a Shared Responsibility

Two informatics experts urge individuals and organizations to work together toward safer EHR-enabled patient care.

Relative Health Care System Performance and Spending in 11 High-Income Countries

From Last to First — Could the U.S. Health Care System Become the Best in the World?

The United States could achieve the best-performing health care system in the world by undertaking coordinated efforts that address four challenges.

Addressing the Prescription Opioid Crisis: Advancing Provider Education and Collaborating with All Stakeholders

Providers have a large role to play in tackling the opioid overdose epidemic, but they can’t go it alone.

Root Cause Analysis: Typical Domains of Root Cause: Medical Errors

Does Every Hospital Admission Deserve a Root Cause Analysis?

Will applying the RCA rubric to hospital admissions better help define and manage care?

Uniquely Identified: The Impact of a National Health Index

What does the NHI mean to a New Zealand clinician, researcher, and health care consumer?

The Intersection of Home-Based Primary Care and Home-Based Palliative Care

My Favorite Slide: The Intersection of Home-Based Primary Care and Home-Based Palliative Care

What are the overlapping provider skill sets needed to care for homebound patients?

Simplifying Person-Centered Care with Use of the Personalized Perfect Care (PPC) Bundle

Personalized Perfect Care

The Personalized Perfect Care Bundle: Making quality metrics easier to understand and more patient-centered.

Laboratory Formulary to Facilitate Better Ordering: Outcome of Orders for Restricted Inpatient Tests in 2016

Improving Care and Cutting Costs: Implementation of a Laboratory Formulary to Facilitate Better Laboratory Ordering Practices

Can a formulary system help to prioritize cost-effective lab tests in the same way it has done for prescription drugs?


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Proactively Catching the Declining Patient

A coordinated effort by UCLA leaders to identify a high-cost population with chronic kidney disease…

Coordinated Care

145 Articles

Proactively Catching the Declining Patient

A coordinated effort by UCLA leaders to identify a high-cost population with chronic kidney disease…

Assessing and Addressing Social Needs in…

Lincoln Community Health Center improved care quality by measuring and responding to upstream social and…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now