Using Simulations to Improve Physician Leadership Hiring

Case Study · May 16, 2018

To determine whether candidates for department chairs have leadership skills to match their impressive CVs, our academic medical center incorporated simulated employee encounters into our hiring process. We require that chair candidates participate in simulations to assess their skill at navigating moderately tense, challenging conversations with smart, highly motivated physician employees. Almost 30% of otherwise qualified candidates failed the simulations. Three of the four chairs hired under the process have started their tenure and appear to have the respect of their employees.

Key Takeaways

  1. When searching for health care leaders, simulations of tense leader/physician conversations can reveal leadership behaviors that aren’t measurable through interviews, behavioral interviews, CV reviews, or reference checks.

  2. In the majority of cases, search committee members are confident in their appraisal of the success or failure of a candidate participating in a simulation. Even untrained observers know good (and bad) leadership when they see it.

  3. Given the modest time investment to create and execute a simulation, it’s a worthwhile activity to incorporate into a search process, and it can save the organization from the anguish of being tied to a mediocre (or worse) leader.

The Challenge

In our institution, as in most academic medical organizations, chairs are enormously powerful and in some cases are responsible for more than 450 direct or indirectly reporting employees, including hundreds of physicians. The chair is expected to motivate and inspire a diverse group including smart, ambitious, overworked physicians, and also to hold them accountable to the standards of our organization.

It is a task that presents new challenges to most. Department chairs often get to their positions on the strength of their clinical reputation, academic background, publication record, and letter of interest. Typically, their last hurdle is successfully conversing during group and individual interviews.

However, once in a position of authority, all too often they alienate their talented workforce with excessive authoritarianism, poor listening skills, or an inability to manage conflict. In my role as Vice Dean, I know it’s only a matter of time before I have to calm down a physician who has butted heads with a department chair who probably should never have been put into this type of managerial position: the kind of boss who stands on his or her employee’s foot and simultaneously asks what the problem is.

Coping with the fallout from poor leadership takes everyone’s time: mine, the Chief Medical Officer’s, our risk management staff, and our human resources department. It’s bad for morale and our work environment, and it may drive away the talented physicians, nurses, and other clinicians on whom we depend.

Problematic department chairs generally don’t lie or mislead us during our hiring process. It’s the process that is the problem. We are like a bus company hiring bus drivers that asks applicants to talk about driving a bus, instead of showing how they drive a bus through narrow intersections, back up to a fuel tank, or maneuver into a tight parking spot despite rowdy passengers. We needed a way to measure the competencies demanded by a department chair position that don’t show up in a CV or during a standard interview.

The Goal

Determine whether simulated employee encounters could help us assess the interpersonal leadership skills that truly matter in health care leadership, during the interview process.

The Team

I coordinate all our chair searches. There were four search teams for four positions, and in each one, the search committees, consisting of 8 to 10 faculty and a few chairs, were active participants in the process. Two of us, both full professors who have held administrative positions and were familiar with these physician complaints, served as mock physician faculty members during simulations.

The Execution

After explaining the simulation concept and what we hoped to achieve to the Vice President for Health Affairs/Dean and search committees, we began using the simulations in consecutive searches for the chairs of pediatrics, medicine, family and community medicine, and neurology.

I wrote a scenario and distributed it to the candidates a few weeks before their interview, so that they would have time to consider how they might respond. While having the scenario in advance might seem like getting the answers to the exam, in my experience a department chair usually has some warning that a tense faculty member will be wanting to meet, so letting the candidates know about the simulation mimics real life.

Each candidate came in for a 90-minute group interview with the search committee. Those that passed this stage of the search would be invited back for more in-depth multiday interviews. The simulations were part of this initial interview.

About a third of the way through his or her group interview, each chair candidate was asked to interact with one of our “actors” playing a mildly disgruntled young physician/faculty member. In the scenario, the relatively new faculty member has a mild performance issue but also feels he or she is not getting the support necessary to do his or her job effectively, and is respectfully upset with the chair. Other specialty-appropriate details were included. In the scenario, the simulated physician is not “once-in-a-career-enraged,” but rather moderately irritated. At the same time, the scenario notes some performance issue: for example, that the faculty member has been seeing fewer patients than colleagues and has cancelled some clinics, a “no-no” in our culture.

After the interaction, we asked a few follow-up questions to encourage the candidate to reflect on how it went. We always asked, “Is there anything you would have done differently?” The entire process took about 10 minutes.

The Results

These situations are so common that I expected every candidate with any experience leading physicians would be adroit in executing a simple conversation to address the main points.

I was wrong.

Almost a third of the well-qualified candidates (who had impressive CVs, came from reputable institutions, and had otherwise handled the interviews well) failed these simulations. The behaviors that led to the failures varied but were almost universally viewed as problematic by the search committees who voted. Some candidates immediately came down with too much force, ignoring the physician’s concerns and starting with performance concerns (even though in the scenario these were mild). Others lectured and talked too much, failing to listen to the physician’s concerns, and one even pried into the irrelevant personal life of the physician, wondering about his marriage.

The simulations revealed behaviors and stylistic approaches that were not consistent with our values and would probably be problematic for leadership roles in any institution like ours.

In contrast, the successful candidates listened carefully to the physician, noted strategies they might adopt to address his or her frustrations (including tapping other people who might helpful), and generally had a looser, more curious tone. They also all suggested more regular contact in the near future and many ended by saying, “This is the start of a relationship — we don’t need to fix everything today.” The general tone was open, non-defensive, and nonjudgmental, but also firm. The candidates acknowledged and validated the physician’s feelings and perceptions and demonstrated a willingness to address the issues raised.

Notably, a few of the most successful candidates also expressed some regrets after the scenario. One lamented, “I made a joke too early. He wasn’t ready for that yet and I had to work to recover.” This remark showed insight and a willingness to evaluate interactions and learn from them.

It is interesting that at least in this small sample of candidates, the search committees were usually unanimous in evaluating which candidates had passed the simulation and which had failed. Some failed more dramatically than others, but our search committees turned out to be fairly consistent in how they rated a “successful” encounter.

Most candidates stated enthusiasm for the process. We had not anticipated that candidates might want to show off their people-handling skills, but the simulation does give them an additional opportunity to sell themselves to the committee, especially if they have not led a large clinical enterprise or have other “holes” in their CV.


Eleven out of 37 candidates, or 29%, failed the simulation. We’ve successfully hired four chairs using this process. Three have started and all appear to have the respect of their physician faculty employees. It will take several more searches, as well as tracking the performance of the chairs we hire, to fully evaluate this approach, but we have built it into our process and are planning to use it for all our executive leadership searches going forward.

Where to Start

  • Identify a common people-management challenge presented in the job. In our case, the scenario requires that the leader hold the physician accountable, but also listen. Survey current leaders to identify a common, mildly tense leadership challenge that would be amenable to a conversation.
  • Anticipate some pushback. Most colleagues were supportive and curious, but our search firm worried that having to do a simulation would chase off talented candidates, and a few search committee members worried the simulation would measure acting rather than other skills. (Some may say that the ability to act is in itself a desirable skill for these roles.)
  • Base the process on the skills you’re trying to measure. We had debate about when to provide the scenario to candidates. In the real world, most of our leaders have some notice about challenging conversations that are coming, and can prepare. Our goal wasn’t a “gotcha” experience, but rather a true measure of the leader’s communication style on a good day. In some contexts, it might be preferable simply to notify candidates that a simulation is coming, but not share details until the time.
  • Recruit “actors” who are familiar with the details that make up the basis for the scenario. For example, if the physician is going to complain about a lack of outpatient support, it would be good for the actor to know exactly how many receptionists, licensed practical nurses, and rooms there are available in a typical clinic.

Call for submissions:

Now inviting expert articles, longform articles, and case studies for peer review


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »

More From Leadership
Comparison of EHR Use Measures by Physician Gender

Differences in Ambulatory EHR Use Patterns for Male vs. Female Physicians

UCSF Health found that women providers spent more time in the EHR and documented longer notes on a per-wRVU basis, possibly contributing to greater burnout.

Fiscus01_pullquote - humanizing physician performance review

Humanizing the Annual Physician Performance Review

Transforming the review process from a punitive, deflating experience to a valuable one that strengthens the relationship between physician and organization.

People Believe Strongly That Leadership Can Be Taught

Leadership Survey: Leadership Skills Are Teachable and Vital

Leadership is teachable, and leadership development and training are important, according to our survey on the topic. Yet the same survey reveals that more than half of respondents think their organizations’ efforts to develop and train leaders are lacking in quality and time commitment.

A Preliminary Model of Determinants and Consequences of Unhurried Conversations with Patients

Careful and Kind Care Requires Unhurried Conversations

Health care providers must have time to know their patients in “high definition” to best meet their needs.

From the Commonwealth to Obamacare: Reflections on 10+ Years of Expanding Health Insurance Coverage

The former Executive Director of the Commonwealth Health Insurance Connector — a model for the Affordable Care Act and other state marketplaces — reflects on what worked, what didn’t, and what could be done differently in both Massachusetts and at the federal level.

Time Spent Engaging Directly with 16 Camden RESET Participants or Coordinating Care on Their Behalf

“Putting All the Pieces Back”: Lessons from a Health Care–Led Jail Reentry Pilot

The Camden Coalition’s jail-based reentry program illuminated the necessity and challenges of engaging people with complex health and social needs and helping to transform the systems that serve them.

Sands01_pullquote clinical research partnership for learning health care

Real-World Advice for Generating Real-World Evidence

If envisioned and implemented properly, a partnership between clinical delivery systems and clinical research programs can get us closer to the goal of achieving learning within the care continuum and discovering evidence that is available when it is needed.

The Largest Share of Organizations Do Not Have a Formal Strategy for Clinician Engagement

Leadership Survey: Why Clinicians Are Not Engaged, and What Leaders Must Do About It

Clinician engagement is vital for improving clinical quality and patient satisfaction, as well as the job satisfaction of clinicians themselves. Yet nearly half of health care organizations are not very effective or not at all effective at clinician engagement.

Rowe01_pullquote - clinician well-being - fighting clinician burnout and creating culture of wellness takes all stakeholders

Defending the Term “Burnout”: A Useful Tool in the Quest to Ease Clinician Suffering

Health care leaders must take a preemptive approach to clinician well-being that is supported by all stakeholders and prioritized on an equal footing with essential clinical and financial measures.

Screenshot from the NewYork Quality Care Chronic Condition Dashboard

Success in a Hospital-Integrated Accountable Care Organization

How NewYork Quality Care achieved shared savings — by strengthening collaboration, enhancing care management with telehealth, and transparently sharing performance data.


A weekly email newsletter featuring the latest actionable ideas and practical innovations from NEJM Catalyst.

Learn More »


Leading Teams

174 Articles

Humanizing the Annual Physician Performance Review

Transforming the review process from a punitive, deflating experience to a valuable one that strengthens…

Physician Burnout

54 Articles

Differences in Ambulatory EHR Use Patterns…

UCSF Health found that women providers spent more time in the EHR and documented longer…

Team Care

112 Articles

Humanizing the Annual Physician Performance Review

Transforming the review process from a punitive, deflating experience to a valuable one that strengthens…

Insights Council

Have a voice. Join other health care leaders effecting change, shaping tomorrow.

Apply Now