|Year : 2020 | Volume
| Issue : 1 | Page : 44-50
Assessment of a continuing medical education intervention designed to change physician practice regarding blood transfusion
Evangeline K Arulraja1, Shari A Whicker1, Katherine H Shaver2, Linda M Wells3, A Paul Dallas4, David W Musick4
1 Department of Pediatrics, Virginia Tech Carilion School of Medicine, Roanoke, Virginia, USA
2 Department of Biostatistics, Virginia Tech Carilion School of Medicine, Roanoke, Virginia, USA
3 Department of Continuing Medical Education, Virginia Tech Carilion School of Medicine, Roanoke, Virginia, USA
4 Department of Internal Medicine, Virginia Tech Carilion School of Medicine, Roanoke, Virginia, USA
|Date of Submission||13-Jan-2020|
|Date of Decision||25-Feb-2020|
|Date of Acceptance||04-Mar-2020|
|Date of Web Publication||17-Apr-2020|
David W Musick
Department of Internal Medicine, Virginia Tech Carilion School of Medicine, Roanoke, Virginia
Source of Support: None, Conflict of Interest: None
Background and Objectives: Excessive packed red blood cell (pRBC) transfusions are associated with worse clinical outcomes and unnecessary costs. While multi-faceted continuing medical education (CME) approaches have been shown to be effective methods for changing physician practice, few studies have evaluated this approach as a method for changing blood transfusion practices. Methods: In this prospective cohort study sought to use a multi-faceted CME platform to modify physician transfusion practices. In this prospective cohort study, the authors implemented a multi-faceted CME intervention including didactic presentations, distribution of educational materials, educational posters, and electronic medical record clinical decision support. Primary outcomes were number of pRBC transfusions prior to and after intervention and associated costs. Secondary outcomes included knowledge acquisition, satisfaction, self-reported improvement in knowledge, and intent to change behavior. The intervention targeted physicians from four departments: Surgery, Internal Medicine, Obstetrics and Gynecology, and Emergency Medicine. Results: Fifty-eight physicians participated in the experimental group and seventy-three physicians in the control group. There was a 26% decrease (P <.0001) in pRBC transfusions monthly when comparing the year prior to intervention to post-intervention year. Clinicians reported improved knowledge acquisition regarding transfusion risks and indications (P <.001). Adjusted transfusion practices saved the primary teaching hospital $722,950 following the intervention. Conclusion: This study supports the use of a multi-faceted CME intervention to align clinical practice with evidence-based transfusion guidelines. Future studies should investigate the effectiveness of individual components of multi-faceted CME interventions regarding improved physician knowledge and clinical practice, patient outcomes, and cost-benefit.
Keywords: Blood transfusion practices, continuing medical education, multifaceted continuing medical education
|How to cite this article:|
Arulraja EK, Whicker SA, Shaver KH, Wells LM, Dallas A P, Musick DW. Assessment of a continuing medical education intervention designed to change physician practice regarding blood transfusion. Glob J Transfus Med 2020;5:44-50
|How to cite this URL:|
Arulraja EK, Whicker SA, Shaver KH, Wells LM, Dallas A P, Musick DW. Assessment of a continuing medical education intervention designed to change physician practice regarding blood transfusion. Glob J Transfus Med [serial online] 2020 [cited 2020 Oct 22];5:44-50. Available from: https://www.gjtmonline.com/text.asp?2020/5/1/44/282739
| Introduction|| |
Growing scrutiny of packed red blood cell (pRBC) transfusions began in the 1980s due to concerns about transfusion-related infections such as HIV and hepatitis C. While these complications have been drastically minimized, this reduction has come at a significant cost. For example, pathogen reduction methods, such as nucleic acid testing to screen blood, cost >$1 million per infection prevented. In addition, blood cannot be screened for emerging or unknown infections for which there are no tests, and even when tests are used, they are not always sensitive enough to pick up all contaminants.
Existing literature reveals that pRBC transfusions are associated with a variety of negative clinical outcomes due to their immune-modulating effects causing either immune activation (e.g., transfusion-related acute lung injury) or suppression (e.g., predisposing to [hospital-acquired] infection). Transfusion-related immune tolerance has been shown to predispose patients to an increased risk for multi-organ failure, and there is some recent evidence suggesting an increased risk of tumor recurrence and development of autoimmune disease. Transfusions are associated with an increased risk of nosocomial infections, increased length of hospital stay, and increased mortality.,,,,,,, Unnecessary and/or excessive pRBC transfusions are risky, may lead to negative patient outcomes, and are an inefficient use of resources that can lead to higher costs for both patients and the health-care system.
A growing body of evidence based on randomized, controlled trials recommends a restrictive transfusion strategy, defined as transfusion <8 g/dL for hemodynamically stable adults, even those in critical care, at a hemoglobin concentration of <7 g/dL and using a hemoglobin concentration of <8 g/dL for patients with preexisting cardiovascular disease or those undergoing cardiac or orthopedic surgery, as it has been shown that a lower hemoglobin concentration can be tolerated without adverse clinical outcome., Guidelines also suggest that standard practice should be to initiate transfusion with 1 unit of blood rather than 2 units. In addition, recent evidence suggests that a hemoglobin level should not be used as a final arbiter for transfusion but instead used in conjunction with physiologic symptoms of anemia (difficulty breathing and chest pain) when evaluating the need for transfusion.,,,,,
Despite the general consistency of guidelines and lack of evidence for the efficacy of pRBC transfusion, there is significant variability in pRBC transfusion practice., The challenges in changing physician practice behavior have been well described.,, It has been suggested that change in physician practice behavior can be impacted by organizational factors, individual knowledge and clinical experience, influence of physician opinion leaders, societal expectations, resource availability, and awareness and acceptance of evidence-based practice guidelines., Physician knowledge of and adherence to guideline recommendations for the management of many diseases has been shown to be fair to poor. Specific barriers to the adoption of clinical guidelines include lack of awareness, lack of understanding about how a guideline applies to a physician's practice, lack of consensus due to distrust of guideline creationists, conflict with a physician's current routine or training, belief that following the guideline will not significantly impact patient outcomes, and lack of resources needed to follow guidelines. The evidence suggests that it may take over 15 years for results from publication of a landmark study to reach a 50% utilization rate in clinical practice.
One major approach to influencing physician practice behavior is continuing medical education (CME). Over the past two decades, a variety of interventions have aimed to improve pRBC transfusion practice among physicians with varying success. CME represents the most widely applied approach for keeping physicians up to date with medical advances in the rapidly changing medical field. Yet, <10% of surveyed doctors rely on traditional, didactic CME primarily because passive distribution of information does little to change physician practice behavior.,, CME that includes more interactive content delivery methods and intervention (e.g., small group discussion, problem-based learning, audit, and feedback) has been shown to be more effective.,, CME interventions that employ decision support with best-practice alerts achieved a 24% reduction in pRBC units transfused; these investigators achieved a significant reduction in total number of pRBC units transfused.,,
Aims and Objectives
In addition to addressing physician transfusion practice, the objectives of this study were to determine whether a multifaceted CME intervention would improve the knowledge, competence, and transfusion practice of physicians, as evidenced by a reduction in both the number of transfusions and associated costs. A related objective was to contribute to the CME literature regarding the effectiveness of specific methods used to improve physician knowledge and promote evidence-based practice. For our study, we created a multifaceted intervention with respect to these aforementioned recommendations and used several methods in delivering CME content, in an attempt to educate physicians effectively about blood transfusion practices.
Excessive pRBC transfusions are risky as evidenced by worse patient outcomes and are an unnecessary use of valuable resources that are costly to patients and the health-care system. Despite the well-documented knowledge of these risks, a gap exists between evidence-based medicine and clinical practice; thus, this project asked the question, can a targeted, multifaceted CME-based educational program impact physician transfusion practice?
| Subjects and Methods|| |
Our study was conducted at a 703-bed Level 1 trauma center and tertiary care academic medical center. This study took place between August 1, 2014, and June 30, 2017. The multifaceted CME intervention was targeted at predetermined high-volume transfusing services, specifically the departments of surgery (SR) and its subspecialties of orthopedics, neurosurgery, trauma, vascular, and cardiothoracic; internal medicine (IM) and its subspecialties of pulmonary critical care, hospitalists, gastrointestinal, and cardiology; emergency medicine (EM); and obstetrics and gynecology (OB) and its subspecialty gynecology oncology. [Table 1] reviews the breakdown of physicians by specialty and by control versus experimental group. Individuals in the experimental group were self-selected based on whether they chose to participate in the didactic portion of the intervention or not.
We implemented a multifaceted, 3-year CME intervention to determine if targeted CME could have an impact on physician transfusion practice. The CME intervention included several educational methods implemented sequentially:
- In-person didactic presentations, given at each department's grand rounds sessions or faculty meetings
- Distribution of educational materials, delivered through electronic mail containing a link to online documents, or sent directly to the home addresses of physicians through postal mail
- Placement of educational posters in nursing stations strategically located throughout the teaching hospital [Figure 1] for sample poster]
- A decision support system that modified the electronic medical record system to include a “pop-up” reminder prompting physicians to consider the indications, contraindications, and risks associated with pRBC transfusions. This “hard stop” reminder message also provided a link to current evidence-based guidelines regarding pRBC transfusions and required physicians to provide a short justification for their transfusion order.
To ascertain the effectiveness of our educational intervention, we performed pre- and posttesting of physician knowledge. Posttests were administered on three different occasions: immediately following the didactic presentations or upon completion of the distributed educational materials (immediate posttest), 6 months after the intervention, and 12 months after the intervention. Pre- and posttests were composed of the same eight multiple-choice questions. Physicians were incentivized to complete the posttest by having their names entered into a lottery for a local restaurant gift card.
A Likert scale survey was used to assess physician satisfaction with the different components of the intervention in order to determine which aspects of the intervention were found to be more efficacious and satisfactory than others. The survey also included self-reported intention to change practice behavior because of the intervention.
In the design of these posttests and survey, we evaluated the success of our project using Moore's framework for evaluating CME outcomes with respect to improved competence, knowledge retention, and commitment to practice change. We sought to compare attending physicians' transfusion knowledge base and preintervention transfusion practices to their transfusion practices following the intervention. We also analyzed the costs associated with changes in transfusion practice.
We also collected information regarding participating physicians' transfusion practices using the following variables: ordering provider, ordering provider department, patient outcomes (disposition and length of stay), order date, units of pRBCs ordered, reason for blood order, Medicare Severity-Diagnosis Related Group, principle procedure, and operating room log procedure. For the purposes of this study, we report data solely pertaining to our previously stated hypothesis. We collected the transfusion data at a physician level on a monthly basis for attending physicians in the targeted departments of surgery, IM, EM, and obstetrics and gynecology. We analyzed changes in transfusion practices for pre- and postintervention years for 131 attending physicians in the four previously identified departments.
For transfusion data analyses, we used a generalized linear mixed model with Poisson distribution for repeated measures using count data (number of pRBC units transfused by physicians). We analyzed changes in the absolute number of pRBC transfused and mean monthly pRBC units transfused to compare changes in transfusion practice; these metrics have been used in previous interventions directed at improving physician transfusion practices. We used an alpha level of 0.05 to determine statistical significance. All statistical analyses were performed using statistical analysis software, version 9.3 (SAS Institute Inc., Cary, NC, USA).
Our study involved three data collection periods:
- Preintervention year, August 2014 through June 2015. During this period, we collected data concerning physicians' transfusion practices using the variables previously described
- Intervention year, August 2015 through June 2016. During this period, we implemented the previously described educational interventions and also collected identical data (identical to preintervention year variables) concerning physicians' transfusion practices using the variables previously described
- Postintervention year, August 2016 through June 2017. During this period, we collected knowledge retention and satisfaction survey data.
We also analyzed costs associated with changes in transfusion practices between the preintervention period (year 1) and the postintervention period (year 2). We calculated and compared costs based on the number of transfusions ordered during the preintervention year and the postintervention year using the national average cost of transfusing a unit of leukoreduced pRBCs of $761 per unit.
We used ANOVA to analyze knowledge improvement using the item means for pre- and posttest data. We collected these data at a physician level and aggregated the data for analysis at a departmental level.
The study was deemed to not meet the regulatory definition of human subjects research and was, therefore, exempt from further review by the Institution.
| Results|| |
A total of 131 attending physicians participated in this study. Fifty-eight total physicians participated in the full multifaceted CME intervention and thus composed the experimental group. Seventy-three total physicians opted to not participate in the didactic portion of the CME intervention, and these physicians composed the control group; of note, these physicians were potentially exposed to all other aspects of the multifaceted intervention given the nature of the intervention (e.g., physicians may have attended grand rounds with didactics but chose not to take the pre- or posttests).
There was a decrease of 950 transfusions between pre- and postintervention years among the 131 targeted physicians across all departments [Table 2]. Of the 131 physicians, the 58 who participated in the full multifaceted CME intervention (the experimental group) decreased their transfusions by 20.9% in comparison to the 73 physicians from the control group who reduced their number of transfusions by 17.1%. There was a significant 26% decrease in mean monthly transfusions across all 131 physicians for the targeted departments (P < 0.0001) between preintervention and postintervention years [Figure 2].
|Figure 2: Mean packed red blood cell units transfused monthly in preintervention period and intervention period. There was a 26% significant decrease in the mean number of packed red blood cell units transfused among all targeted departments when comparing preintervention period to intervention period (P < 0.0001). pRBC: Packed red blood cell|
Click here to view
On departmental analysis of the number of transfusions in pre- and postintervention time periods, only the department of surgery increased the total units of pRBC transfusions after implementation of the intervention. The department of IM demonstrated the greatest decrease in pRBC transfusions, decreasing total transfusions by 830 pRBC units [Table 2].
There was a significant improvement of 13.8% in knowledge regarding transfusion risks and indications when comparing the pre- and the immediate posttest (P < 0.001). However, we did not have an adequate number of responses to assess knowledge retention through the delayed posttests at 6- and 12-month intervals. We also did not receive adequate survey responses concerning participant satisfaction or intent to change behavior to conduct meaningful analyses.
The cost analysis [Table 3] involved calculating the difference in number of pRBC units transfused between pre- and postintervention and using the estimated average of $761 per pRBC unit. The IM department demonstrated the greatest cost savings, $631,630 in the 11-month period postintervention. The department of surgery was the only department to demonstrate an increase in costs following the intervention. Among all departments, cost savings were $722,950 when comparing the year of intervention to the year before the intervention.
| Discussion|| |
Our analysis suggests that our multifaceted CME intervention made an impact in overall transfusion practices, one of the important outcomes of a successful CME intervention., We noted that our multifaceted CME intervention produced significant overall results. However, our hypothesis that we would see a significant difference between those physicians who were exposed to the didactic portion and those who were exposed only to the nondidactic portions of the intervention was not supported by the data. Our design involved an educational intervention, an electronic medical record clinical decision support hard stop, and educational posters posted throughout patient care areas. While we were able to clearly discern who participated in the didactic portion of the intervention, we were not able to ascertain who was exposed to the posters and the electronic medical record hard stop. This fact may have contributed to both the significant decrease in overall transfusions and the lack of significant differences in transfusion practice between the experimental and control groups. This fact also meant that performing data analysis at the level of a given individual clinician would have been very difficult to interpret; thus, we made no attempts at such analyses.
In our departmental analysis, the changes in transfusion practice were particularly notable in the department of IM. IM had the greatest reduction in number of pRBC units transfused after implementation of the intervention. A caveat to this finding is that the IM residents received the same multifaceted intervention as the intervention described in this study, in a separate quality improvement residency project that was ongoing during the 2nd year of this study. The influence of the resident intervention may have affected the attending physicians' transfusion practices, and we could not control for this effect. One study found that attending physicians routinely had lower knowledge scores than residents, yet attending physicians exhibited more confidence in their knowledge and influenced residents' transfusion decisions resulting in their ordering of potentially inappropriate transfusions.
Another confounding factor could have been the effect of opinion leaders, defined as trusted physicians viewed as experts who influence members of their own departments. The effect of opinion leaders has been found to be of variable effectiveness but nevertheless must be considered as a potential confounding factor., Certain physicians in the study were already championing restrictive transfusion practices in their respective departments before our CME intervention, and we could not control for this factor.
Furthermore, in our departmental analysis, we found that only the department of surgery increased the number of pRBC units transfused postintervention. This could be due to a multitude of factors, including specific surgical case-mix, lack of willingness to participate in the intervention, or lack of willingness to change behavior. While there were 18 surgeons in the experimental group, we are only able to report aggregated data analyses for surgeons. We speculate that these 18 surgeons could have modified their transfusion practices in adherence to suggested guidelines, but our study design did not allow us to explicitly determine whether this was the case.
Due to the poor response rate for the 6- and 12-month postintervention tests, we were not able to meaningfully assess knowledge retention, satisfaction, and intent to change, which is especially unfortunate as the assessment of motivations and barriers is meaningful in individualizing CME interventions accordingly. Sustainable behavior change is important for the assessment and improvement of CME. These issues both represent directions for further research on this topic.
Another limitation of the study concerns patient case-mix. We were unable to formally analyze patient case-mix, primarily due to a change in the manner in which case-mix data are captured and reported by our hospital system; this change occurred during the data analysis period of our study, thus making it difficult to perform formal case-mix analyses either in the aggregate or by the department. While we intuitively believe that case-mix would not have significantly varied at our academic medical center between the preintervention year and the subsequent postintervention year, we must acknowledge the possibility that changes in case-mix could have impacted our results. In future studies examining blood utilization, changes in transfusion could be measured using pRBC units transfused per 1000 discharges in order to better control for case-mix; this measure is a more direct quality indicator for hospital-wide blood transfusions, and can be corrected for annual changes in health care volumes.
Future directions for transfusion practice improvement research should also incorporate patient outcomes (e.g., hospital-acquired infections, 30-day mortality, and length of stay) into transfusion analysis and associated costs accordingly. In addition to saving valuable resources and reducing costs to the patient and hospital, with respect to decreasing the number of units of pRBC transfused, it is important to calculate other related cost savings that are associated with transfusion practices. For example, transfusions are known to be an independent risk factor for increasing likelihood of acquiring a hospital-acquired infection; examining the increased cost associated with developing such an infection would be useful.
Broader implications of this research include the utility of CME in improving physicians' medical knowledge and promoting evidence-based practice, ultimately leading to improved patient outcomes. This is important in light of the rapidly changing knowledge base of modern medicine, innovations, and standards of care., Our study indicates that institutions making an investment in this type of CME-based research can lead to improved understanding of the impact of targeted CME interventions on institutional costs and that such investments can be stated to “pay for themselves” in terms of cost reductions. In a time when cost control is imperative in health care, the value of this type of study cannot be overstated.
The authors would like to acknowledge Keri K. van Blaricom for her involvement in data collection for this study.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Corwin HL, Theus JW, Cargile CS, Lang NP. Red blood cell transfusion: Impact of an education program and a clinical guideline on transfusion practice. J Hosp Med 2014;9:745-9.
Busch MP, Kleinman SH, Nemo GJ. Current and emerging infectious risks of blood transfusions. JAMA 2003;289:959-62.
Marik PE. The hazards of blood transfusion. Br J Hosp Med 2009;70:12-5.
Carson JL, Carless PA, Hebert PC. Transfusion thresholds and other strategies for guiding allogeneic red blood cell transfusion. Cochrane Database Syst Rev 2012;4:1-69.
Marik PE, Corwin HL. Efficacy of red blood cell transfusion in the critically ill: A systematic review of the literature. Crit Care Med 2008;36:2667-74.
Shander A, Fink A, Javidroozi M, Erhard J, Farmer SL, Corwin H, et al
. Appropriateness of allogeneic red blood cell transfusion: The international consensus conference on transfusion outcomes. Transfus Med Rev 2011;25:232-46.e253.
Vamvakas EC. Establishing causation in transfusion medicine and related tribulations. Transfus Med Rev 2011;25:81-8.
Goodnough LT, Shander A. Patient blood management. Anesthesiology 2012;116:1367-76.
Pattakos G, Koch CG, Brizzio ME, Batizy LH, Sabik JF 3rd
, Blackstone EH, et al
. Outcome of patients who refuse transfusion after cardiac surgery: A natural experiment with severe blood conservation. Arch Intern Med 2012;172:1154-60.
Salpeter SR, Buckley JS, Chatterjee S. Impact of more restrictive blood transfusion strategies on clinical outcomes: A meta-analysis and systematic review. Am J Med 2014;127:124-31.e123.
Carson JL, Guyatt G, Heddle NM, Grossman BJ, Cohn CS, Fung MK, et al
. Clinical practice guidelines from the AABB: Red blood cell transfusion thresholds and storage. JAMA 2016;316:2025-35.
Yazer MH, Triulzi DJ. AABB red blood cell transfusion guidelines: Something for almost everyone. JAMA 2016;316:1984-5.
Shander A, Gross I, Hill S, Javidroozi M, Sledge S; College of American Pathologists, et al
. A new perspective on best transfusion practices. Blood Transfus 2013;11:193-202.
Hébert PC, Wells G, Blajchman MA, Marshall J, Martin C, Pagliarello G, et al
. A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group. N
Engl J Med 1999;340:409-17.
Villanueva C, Colomo A, Bosch A, Concepción M, Hernandez-Gea V, Aracil C, et al
. Transfusion strategies for acute upper gastrointestinal bleeding. N
Engl J Med 2013;368:11-21.
Napolitano LM, Kurek S, Luchette FA, Corwin HL, Barie PS, Tisherman SA, et al
. Clinical practice guideline: Red blood cell transfusion in adult trauma and critical care. Crit Care Med 2009;37:3124-57.
Bennett-Guerrero E, Zhao Y, O'Brien SM, Ferguson TB Jr, Peterson ED, Gammie JS, et al
. Variation in use of blood transfusion in coronary artery bypass graft surgery. JAMA 2010;304:1568-75.
Murphy EL. Health outcomes in recipients of blood transfusion-observational studies versus randomized clinical trials. ISBT Sci Ser 2013;8:172-6.
Saddawi-Konefka D, Schumacher DJ, Baker KH, Charnin JE, Gollwitzer PM. Changing Physician Behavior With Implementation Intentions: Closing the Gap Between Intentions and Actions. Acad Med 2016;91:1211-6.
Grol R, Grimshaw J. From best evidence to best practice: Effective implementation of change in patients' care. Lancet 2003;362:1225-30.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al
. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999;282:1458-65.
Allery LA, Owen PA, Robling MR. Why general practitioners and consultants change their clinical practice: A critical incident study. BMJ 1997;314:870-4.
Bauchner H, Simpson L, Chessare J. Changing physician behaviour. Arch Dis Child 2001;84:459-62.
Nissen SE. Reforming the continuing medical education system. JAMA 2015;313:1813-4.
Mostofian F, Ruban C, Simunovic N, Bhandari M. Changing physician behavior: What works? Am J Manag Care 2015;21:75-84.
Balas EA, Boren SA. Managing clinical knowledge for health care improvement. In: Bemmel J, McCray AT, editors. Yearbook of Medical Informatics 2000: Patient-Centered Systems. Stuttgart, Germany: Schattauer Verlagsgesellschaft; 2000. p. 65-70.
Eisenstaedt RS. Modifying physicians' transfusion practice. Transfus Med Rev 1997;11:27-37.
Goodnough LT, Shieh L, Hadhazy E, Cheng N, Khari P, Maggio P. Improved blood utilization using real-time clinical decision support. Transfusion 2014;54:1358-65.
Hibbs SP, Nielsen ND, Brunskill S, Doree C, Yazer MH, Kaufman RM, et al
. The impact of electronic decision support on transfusion practice: A systematic review. Transfus Med Rev 2015;29:14-23.
Moore DA. Framework for outcomes evaluation in the continuing professional development of physicians. In Davis DA, et al
. The Continuing Professional Development of Physicians: From Research to Practice; 2003. p. 249-74. Chicago, IL., American Medical Association Press.
Tinmouth A, Macdougall L, Fergusson D, Amin M, Graham ID, Hebert PC, et al
. Reducing the amount of blood transfused: A systematic review of behavioral interventions to change physicians' transfusion practices. Arch Intern Med 2005;165:845-52.
Shander A, Hofmann A, Ozawa S, Theusinger OM, Gombotz H, Spahn DR. Activity-based costs of blood transfusions in surgical patients at four hospitals. Transfusion 2010;50:753-65.
Salem-Schatz SR, Avorn J, Soumerai SB. Influence of clinical knowledge, organizational context, and practice style on transfusion decision making. Implications for practice change strategies. JAMA 1990;264:476-83.
Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: An overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ 1998;317:465-8.
Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. Int J Technol Assess Health Care 2005;21:380-5.
Cervero RM, Gaines JK. The impact of CME on physician performance and patient health outcomes: An updated synthesis of systematic reviews. J Contin Educ Health Prof 2015;35:131-8.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]