A staggering 36,000 randomized controlled trials (RCTs) are published each year, on average, and it typically takes about 17 years for findings to reach clinical practice. Proposed changes in frontline care often originate with suggestions from clinicians, but evaluating their merit can be time-consuming and expensive. Focused evidence reviews may yield inconclusive results that don’t lend themselves to clear-cut decisions; pilot studies are vulnerable to poor design, less-than-rigorous evaluation, scant evidence, and other limitations. The result: a logjam of epic proportions that slows the real-world implementation of evidence-based practices. At Kaiser Permanente Southern California (KPSC), we aimed to find a quicker, more effective, more systematic way to break that logjam in a manner that is consistent with the Triple Aim.
An Organizational Imperative
In 2013, KPSC set out to expedite implementation of clinical practices that have a rigorous evidence base, are not used or are underused by KPSC, improve the quality of care, and are likely to be sustainable and cost effective. KPSC’s medical group quality leader, Dr. Michael Kanter, initiated this effort after hundreds of pilot studies and performance improvement activities at KPSC, using PDSA cycles, yielded too few new practices that were implemented widely.
In January 2014, we assigned an analyst on our existing evidence-based medicine team to scan the literature for promising practices that were not in wide use at KPSC. The findings were forwarded to regional chiefs of service, who responded enthusiastically because they had long trusted the team’s evidence reviews to inform clinical decision support and guideline development in their specialty areas. Nevertheless, given that regional chiefs and other stakeholders had limited ability to implement new practices, by June 2014 it was clear that a one-person literature review would be insufficient. A project manager was then hired to help with the implementation effort.
The result was E-SCOPE — Evidence Scanning for Clinical, Operational, and Practice Efficiencies — a system for identifying and rapidly implementing clinical and operational practices that are supported by newly published, high-quality evidence. Although KPSC quality and evidence-based medicine leaders are closely involved in E-SCOPE, its daily operations remain the primary responsibility of the evidence-based team analyst and the project manager.
The Scope and Sequence of E-SCOPE
E-SCOPE is executed in four steps:
1. Conduct quarterly evidence searches. A five-member team — two dedicated staff members (the analyst and the project manager) and three regional quality leaders (the medical group’s regional physician director for quality and clinical analysis, his assistant, and an evidence-based practice leader) — use an internally developed algorithm and custom-search strategy to identify newly published, high-quality RCTs (and systematic reviews of RCTs). After screening more than 1,000 abstracts, the team typically selects about 150 published studies that meet predetermined criteria for improving health care quality, outcomes, affordability, efficiency, or utilization. The criteria were determined by evidence-based medicine consultants trained in approaches such as the Cochrane “risk of bias” tool and the GRADE framework. (Evidence of moderate quality is considered only if it strongly favors benefits over harms.) Each evidence review cycle takes about one month.
2. Decide which evidence-based practices to implement. Studies meeting the selection criteria are distributed to physician and health plan leaders, clinical and operational committee members, task force leaders, area medical directors, representatives from clinical and operational departments, and others. Using their expertise, the stakeholders and regional physician quality leaders discuss the expected benefit of implementing identified practices into clinical or operational pathways. For example, evidence about the benefits of a 16-week behavioral weight loss management program for severely obese patients with psoriasis was sent to the regional chief of dermatology, who discussed it informally with two or three dermatology chiefs of service from KPSC medical centers. These expert dermatologists agreed that promoting weight management classes for severely obese psoriasis patients was an important opportunity. The evidence was then discussed at a quarterly meeting attended by the dermatology chiefs of service from all KPSC medical centers, who gave this practice a green light for implementation. Stakeholders know that such a green light really does mean “go” because the medical group’s physician quality leaders have already given their approval.
The E-SCOPE project manager then assembles a small multidisciplinary team that deploys the practice, using processes tailored to the intervention, setting, patient population, and available resources. For the aforementioned weight loss program, processes included regional outreach to members with psoriasis and a body mass index (BMI) ≥30, as well as office-based alerts and education for providers about referring appropriate patients for weight management. (Outreach and proactive office-based encounters are tools used routinely as part of KPSC Complete Care.)
3. Support implementation of selected practices. Full implementation of an initiative across all KPSC medical centers takes a minimum of several months and relies on existing processes. For the psoriasis initiative, the processes included decision support, weight management classes, and other functions of Complete Care. In deciding which evidence-based practices to implement, stakeholders are encouraged to take over responsibility for implementation, which they do for about 25% of new practices. More often, though, the five-member E-SCOPE team assumes responsibility because stakeholders lack the time. For example, although the regional chief of dermatology championed the psoriasis initiative, neither she nor the medical center chiefs of service had time to implement it. As a result, the E-SCOPE project manager took primary responsibility for preparing provider education slides, conducting education sessions with physicians and nurses, and working with health education leaders to develop referral procedures.
E-SCOPE quality leaders and the implementation team decide on metrics for monitoring ongoing implementation. For instance, metrics for the psoriasis target population include physician referral rates to a KPSC 16-week behavioral weight management program and subsequent patient attendance rates, changes in weight/BMI, and psoriasis medication use. Dermatology departments at medical centers receive monthly performance feedback.
4. Monitor progress. The E-SCOPE team regularly monitors the progress of implementation, typically each quarter. The project manager and analyst request reports from the clinical analytics department on initiative-specific metrics and track them over time. Early monitoring focuses on process metrics, such as the number of regional outreach communications related to weight management for severely obese patients with psoriasis. As implementation proceeds across medical centers and initiatives mature, outcome metrics can be included. If, for instance, a medical center dermatology department’s performance falls short, the regional quality physician leader discusses it with the chief of service and department administrator. As needed, systems issues are investigated and other leaders (e.g., weight loss program managers) are engaged.
E-SCOPE’s Influence on Practice
Use of E-SCOPE for the past 2 years (8 quarters) has accelerated the implementation of 17 evidence-based practices: 10 that KPSC had never used and 7 that were underused. The changes are being deployed across all 14 of KPSC’s medical centers, affecting a wide range of patient populations and clinical conditions. For example, E-SCOPE’s impact includes initial outreach to 17,700 patients with psoriasis about weight and lifestyle management classes, 19,500 patients with benign paroxysmal positional vertigo about the Epley maneuver, and 9,000 patients with stroke about home- and Internet-based exercise programs. Other E-SCOPE projects are in the early stages of implementation.
The interval between study publication and initial implementation ranges from 3 months to 2.5 years (mean, 14.4 months). The practices are all based on high-quality studies that are generalizable to KPSC settings, and we expect them to yield comparable effectiveness, safety, timeliness, and efficiency of care, as had been documented in the underlying research. (Duplicating the original studies’ methods to verify the findings is neither practical nor feasible.) Without E-SCOPE, KPSC probably would not have deployed these evidence-based practices, or at least would not have implemented them so quickly.
The Scope of the Challenges and Opportunities
Despite E-SCOPE’s successes and overall buy-in from the KPSC community, we face challenges because our stakeholders have limited time, often related to their clinical duties but also because some of them work on quality improvement projects outside E-SCOPE. One hurdle, as noted earlier, is finding stakeholders to oversee practice implementation. Indeed, the E-SCOPE project manager and evidence-based medicine analyst (the two dedicated staff members) spend more time on implementation than initially expected, thereby limiting how many new practices can be deployed. In addition, stakeholders may lack time to review studies selected by the E-SCOPE team, which can slow or stall the review and decision-making processes. We have even taken promising practices off the table temporarily because of a lack of stakeholder response. We revisit the topic with stakeholders at least twice more before closing it and moving on to other practices.
About half of the practices identified by E-SCOPE are multidisciplinary, and, unlike drug studies, the majority are not sponsored by industry. For instance, we screen but rarely include drug studies in our E-SCOPE final choices because the KPSC pharmacy department, separately and consistently, reviews evidence in this domain. We also still must explore why KPSC has historically been less likely to implement evidence-based practices from studies without industry support and those that require multidisciplinary implementation. That exploration may help us understand the long delays in implementing newly published evidence-based practices. If industry-funded studies continue to frequently not meet our implementation criteria, most often because they are already implemented, we may consider being more selective about including these studies in the initial screening, thereby making the quarterly search process more efficient.
As we gain more experience with E-SCOPE’s overall strategy, search algorithms, and workflows, we may extend the process to other Kaiser Permanente regions (our Georgia region is just starting to use it). The practices implemented by each region will depend in part on existing practices, resources, and clinical priorities, but a single, centralized quarterly evidence search will help create efficiencies of scale.
Our E-SCOPE experience shows that a systematic approach requiring relatively few staff and other resources can put published evidence into practice within months. Specifics, particularly about how to identify and involve stakeholders, may vary across organizations that wish to try an approach like ours, but the basic process seems broadly applicable. As the amount of new, high-quality evidence published each year continues to increase, why wait to put the best of it into practice?
Acknowledgments: We thank Marguerite Koster, MA, MFT, and Lauren Kadziel, MPH, CHES, for their essential contributions to E-SCOPE.