Joyce MF, Berg S, Bittner EA. Practical strategies for increasing efficiency and effectiveness in critical care education. World J Crit Care Med 2017; 6(1): 1-12
Corresponding Author of This Article
Edward A Bittner, MD, PhD, MSEd, Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, WHT 437, Boston, MA 02114, United States. firstname.lastname@example.org
Checklist of Responsibilities for the Scientific Editor of This Article
This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Practical strategies for increasing efficiency and effectiveness in critical care education
Maurice F Joyce, Sheri Berg, Edward A Bittner
Maurice F Joyce, Sheri Berg, Edward A Bittner, Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, United States
ORCID number: $[AuthorORCIDs]
Author contributions: All authors contributed to this paper with conception and design of the study, literature review and analysis, drafting and critical revision and editing, and final approval of the final version.
Conflict-of-interest statement: No potential conflicts of interest. No financial support.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: Edward A Bittner, MD, PhD, MSEd, Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, WHT 437, Boston, MA 02114, United States. email@example.com
Received: September 1, 2016 Peer-review started: September 5, 2016 First decision: September 29, 2016 Revised: October 30, 2016 Accepted: December 13, 2016 Article in press: December 14, 2016 Published online: February 4, 2017
Technological advances and evolving demands in medical care have led to challenges in ensuring adequate training for providers of critical care. Reliance on the traditional experience-based training model alone is insufficient for ensuring quality and safety in patient care. This article provides a brief overview of the existing educational practice within the critical care environment. Challenges to education within common daily activities of critical care practice are reviewed. Some practical evidence-based educational approaches are then described which can be incorporated into the daily practice of critical care without disrupting workflow or compromising the quality of patient care. It is hoped that such approaches for improving the efficiency and efficacy of critical care education will be integrated into training programs.
Core tip: Evidence-based approaches for improving the efficiency and efficacy of critical care education have been developed and should be integrated into training programs. While a variety of such approaches are described in this paper and elsewhere in the medical education literature they share common characteristics. These include utilizing methods to rapidly identify learner needs, teaching directly to those needs, and providing specific feedback on performance. In addition these approaches emphasize active learning activities and integrate educational experiences from the classroom and clinical settings.
Citation: Joyce MF, Berg S, Bittner EA. Practical strategies for increasing efficiency and effectiveness in critical care education. World J Crit Care Med 2017; 6(1): 1-12
Critical care is a demanding medical specialty in terms of its complexity, the frequency of life threatening situations and the need for rapid decision-making based on incomplete data. The breadth and depth of medical knowledge and technical skill necessary for critical care practice continue to rapidly increase yet the time available for education of trainees has not. Limitations in the duty hours of trainees have reduced clinical exposure and allow less time for traditional methods of education. Increasing clinical volume, administrative responsibilities, and documentation and billing requirements increasingly compete for the time that faculty has available for teaching. It is our mandate as critical care practitioners to educate and ensure that we have competent clinicians able to deliver high quality care to our critically ill patients. It is therefore necessary that we find a solution to the dilemma of providing safe and high-quality care while also providing the necessary education for trainees in clinical settings. Approaches to teaching and learning which account for the exponential growth in medical knowledge, unique learning needs and time constraints of the learners, while adapting to the dynamic and clinically demanding environments of critical care practice are urgently needed[2,3].
In this article, we provide a brief overview of the existing educational practice within the critical care environment. We then discuss challenges to education within common daily activities of critical care practice including bedside care, procedures, handover, and crisis management. Some practical educational approaches are described which can be incorporated into the daily practice of critical care without disrupting workflow or compromising the quality of patient care (Table 1). It is hoped that such approaches will increase the efficiency and efficacy of education that is offered to critical care clinicians, not only during training but throughout their careers.
Table 1 Teaching challenges and strategies for increasing efficiency and effectiveness in critical care education.
Challenges to teaching
Strategies for improvement
Complexity, unpredictability, rapid pace of clinical care limits time available for teaching
Use of effective, time efficient methods to identify learner needs, teaching to those specific needs, and providing feedback
Simultaneously instructing trainees while caring for critically ill patients
Examples: Two-minute observation, one-minute preceptor, activated demonstration and teaching scripts
Wide breadth and depth of knowledge required to care for critically ill patients
Integrate “in-class” experiences with “out-of-class” learning
Varying backgrounds and training levels of the learners
Practicing clinical decision-making in the classroom allows trainees to learn from their mistakes in a safe environment
It is not possible expose trainees to all relevant critical care topics
Trainees need to acquire procedural competence with a number of diagnostic and therapeutic tools
Multifaceted learning strategies with performance assessed and mastery demonstrated away from the clinical setting
Finding the optimal balance between providing procedural opportunities for trainees and ensuring patient safety
Examples: Computer-based learning, task trainers, and simulation to provide conceptual and technical understanding
Observing and then performing procedures in elective settings, before attempting high risk procedures on critically ill patients
Just-in-time training immediately prior to actual performance
Use of adjunct technology (e.g., ultrasound, videolaryngoscopy)
Handovers are complex communication tasks
Develop learning strategies for ensuring information management and collaboration to generate a shared understanding of patients and reduce clinical uncertainty
The process is often error prone and substandard handovers have been linked to adverse events
Critically ill patients are particularly vulnerable to ineffective handovers
Examples: Discussions of approaches to diagnosis and management of specific conditions promotes learning
Limited evidence for a “best” approach
Faculty may have limited experience with new handover processes
Providing feedback on clinical actions taken in the preceding shift
Direct supervision of the handover process by experienced clinicians to ensure that communication of critical patient information is occurring and to answer clinical questions
Supplementing the handover with short educational modules relevant to the patients receiving care
Using handovers to evaluate trainee performance and provide formative feedback
Multidisciplinary team practice
High clinical workloads, finding common time to practice, disruption of clinical activities, and cost
Multidisciplinary training incorporated into the activities of daily practice (in situ simulation) can be inexpensive and less disruptive to staffing
Training specifically designed to improve team dynamics is new for many critical care clinicians
Example: Regular repetition of commonly occurring scenarios can be used to reinforce learning and teamwork
In situ simulation can be used to interrogate departmental and hospital processes in real practice conditions
ICU: Intensive care unit.
The intensive care unit learning practice and educational deficiencies
The intensive care unit (ICU) provides unique opportunities for knowledge and skill acquisition in a dynamic and fast-paced clinical environment. There are opportunities to learn technical skills such as airway management, central line placement and ultrasonography, as well as nontechnical skills such as teamwork, communication, and leadership. Surveys suggest that there is no standardized approach to trainee education within critical care medicine, reflecting highly variable ICU environments and practice patterns[4,5]. Such variation in educational practice is noteworthy as it may affect the quality of trainees’ education through varied exposure to different patient cases, opportunities to perform procedures, experience with different attending physician practice styles and total teaching time. Despite this lack of a standardized structure, many programs use similar traditional clinical teaching methods. Bedside teaching is the most common format for trainee education and a majority of programs also offer didactic lectures and informal teaching sessions. In addition an increasing number of programs include access to an online ‘‘core curriculum’’ of critical care topics. With these didactic approaches, trainees in critical care acquire knowledge and skills through processes of “active” learning by participating in bedside teaching rounds and by directly administering patient care, while “passive” learning occurs through the use of lectures, conferences and journal clubs.
Bedside teaching, often conducted during ICU rounds, is an essential component of critical care education, as it covers clinical assessment, conduct of the physical exam and decision making. In addition the importance of multidisciplinary communication, bedside manners, professionalism, and other essential clinical skills are emphasized. Involving the entire team in bedside rounds also contributes to multidisciplinary team development and improved patient care[7,8]. Educating during bedside care is not a passive activity; rather it requires skill by the critical care provider. Appropriate tailoring of educational topics to trainee needs in relation to current patients provides the trainee with the satisfaction of having learned something directly relevant to patient care, promoting active learning as well as providing a powerful motivational boost and educational reinforcement. “Conference room” teaching typically consists of a combination of standard “core” lectures (e.g., mechanical ventilation, sepsis and shock) and flexible teaching topics based on current relevance to bedside care. While core lectures ensure that trainees are provided a certain amount of fundamental knowledge, flexible educational activities are designed to complement core lectures in order to tailor learning to the specific needs and interests of current team members and are typically initiated in response to issues identified during the bedside rounding.
Critical care has long had an “apprenticeship style” of training in which long hours and “see one-do one-teach one” were the primary means of fostering learning. However, work-hour restrictions, generational differences and increasing external regulations have altered this traditional approach. While these methods of providing critical care education are longstanding, there is mounting evidence that they are no longer sufficient. Many studies have reported suboptimal education of trainees in areas that are fundamental to critical care practice including deficiencies in medical knowledge, procedural skills, handover, communication and crisis management[9-18]. In addition, there is evidence that methods for education of critical care trainees have changed little since the Accreditation Council for Graduate Medical Education (ACGME) instituted duty hour standards and core competencies[1,19]. These deficiencies and an apparent lack of progress in critical care education may have a detrimental effect on patient safety and the quality of care. Perhaps it is because of the apprentice-style educational tradition in critical care that we have been slow to identify and adopt “best practices” of modern education theory for fostering experiential learning. In the age of reduced work hours and increased focus on patient safety, however, we are caught between less experienced clinicians at the bedside and imposed requirements for ensuring clinical competence. To successfully address these challenges requires a different educational experience. The following sections provide approaches for increasing educational efficiency and efficacy during the daily activities of ICU practice. They are founded in educational theory and meant to be readily integrated into existing critical care practice regardless of the size, practice characteristics or economic resources.
Strategies for teaching with limited time
Educators in the ICU environment face the formidable challenge of simultaneously instructing trainees while caring for critically ill patients in a clinical environment where complexity and the knowledge required for decision making is high, time available for teaching is limited, and interruptions are frequent. Due to increased and competing demands on critical care faculty, the time available for clinical teaching appears to be in decline[20,21]. An even greater barrier to teaching than a heavy clinical workload is the misconception that “real teaching” requires an extended formal lecture. With this teaching misconception in mind, clinicians are understandably reluctant to teach because it interferes with patient care. As clinical educators it is important to recognize that every patient interaction has teachable moments. To maximize learning opportunities, educators must be attentive to identifying these moments and then making them pertinent to a learner’s needs. Even small amounts of time focused on teaching can offer important learning opportunities for trainees to acquire new insights and skills. To achieve this efficient and effective teaching approach, a variety of strategies can be successfully employed. These educational strategies share common characteristics including: (1) identifying the learner’s needs; (2) teaching directed to meet those specific needs; and (3) providing performance feedback.
Identifying the learner’s needs saves time by not teaching what the learner already knows or is not ready for. Assessment of the learner’s level of knowledge requires asking good questions as well as the ability to listen and observe. Questions are the educator’s “primary diagnostic tool” to ascertain the learner’s current level of knowledge and experience with similar situations. Questions that precede a patient encounter can help the educator to ascertain the learner’s understanding and experience with the clinical problem at hand-for example, “How do we assess delirium in this patient?”. While questions that follow the learner’s presentation of a patient can guide the educator’s decisions about how and what to teach-for example, “How do you think we should manage this problem?”.
A period of brief observation can be an effective means of assessing the learner’s abilities instead of making inferences based on a patient presentation alone. The “two-minute observation model” is a well described method in which the teacher observes a patient encounter in order to obtain more specific information about the trainee’s learning needs which can be used for providing guidance or feedback. This technique is effective for teaching both history and physical exam skills as well as for teaching communication skills. In advance of the patient encounter, the teacher and learner should agree on which aspect of the interaction will be targeted for the brief observation-such as establishing patient rapport, history taking, physical examination, or discussion with nurse, consultant or family member. As with other learner-centered models, the instructor should set clear expectations, directly observe the learner and provide specific feedback and teaching.
The “one-minute preceptor model” is another focused teaching tool that is easy to implement while engaging in patient care. This method uses a 5 step approach: (1) query the learner about what he/she thinks is going on with the patient; (2) probe for underlying reasoning or alternative explanations; (3) teach a general principle; (4) reinforce what was done well; and (5) correct any errors and make suggestions for improvement. In a mere one minute, the instructor is able to obtain a brief assessment of the trainee, provide an educational pearl, and deliver immediate positive and negative feedback. Research on the one-minute preceptor model suggests that it is an effective and efficient method of engaging learners in high-level case discussions of clinical problems, and its use is associated with strong satisfaction by both learners and teachers[23,24].
“Activated demonstration” is a model in which the learner is asked to observe the clinical teacher performing a skill that is unfamiliar to the learner. After preparing the learner with a preview of the upcoming teaching points, the learner is given a specific assignment to complete while observing, such as “Watch how I perform the laryngoscopy”, and provided expectations in terms of participation. After the demonstration, the teacher “activates” the learner by asking him or her to describe what was observed. A brief discussion of relevant learning points then occurs in which the rationale for the actions is examined and further study may be assigned.
“Teaching scripts” are concise, pre-prepared high-yield lessons that the instructor can teach the learner when the appropriate clinical setting arises. To be most effective the script should be adapted to account for the trainee level, the patient’s clinical circumstances, and the disease process under consideration. Examples of teaching scripts might include “choosing sedation drugs for an intubated patient” or “fluid management in ARDS”. Over time, seasoned clinicians naturally create a portfolio of scripts that they can effortlessly access, but educators at all levels can proactively develop teaching scripts. Limiting the number of learning topics discussed to 2 or 3 per day will increase their significance and the attention paid to each of them. Too many topics can overwhelm the learner, ultimately reducing the educational impact. Finally, it is imperative to briefly review and summarize the important learning topics that were covered and discuss related learning activities. For example, “to review, today we discussed ventilatory management for ARDS. This afternoon, our critical care fellow will share a recent article that is related to our discussion”. This summary reinforces prior learning and encourages evidence-based practice as well as peer-to-peer education.
Feedback is a powerful instructional strategy that can be effectively provided with limited time. The key to feedback is providing specific descriptive comments about a learner’s performance. The “Ask-Tell-Ask” model is a common model for giving feedback. With this approach the teacher first sets the stage for providing feedback by telling the learner, “I would like to give you feedback”. Then, the instructor asks the learner to assess his/her own performance with a question like, “How do you think you did?”. Next, the teacher provides his/her own observations (importantly, positive and corrective), addresses the learner’s self-assessment and provides an action plan for improvement. This approach incorporates the learner’s perspective, avoids judgment and promotes the skill of self-reflection. The timing and location of providing feedback may vary depending on the issue and urgency. On-the-spot feedback based on events occurring at the bedside has the advantage of providing patient-centered in-training evaluations, which are a cornerstone of medical education. In addition trainees highly value feedback related to specific behaviors performed at the bedside, associating high quality teaching with feedback pertaining to specific behaviors such as bedside skills and case presentations. Delaying constructive criticism until a later time might be beneficial in some circumstances to avoid feelings of trainee embarrassment. However it is important to consider that a delay in feedback might also lead to continuation of incorrect and potentially harmful patient care, thus quick context-specific feedback is beneficial in most circumstances with a plan for more extensive discussion in a quiet, “safe” environment at a later time.
Revamping ICU lectures - “flipping” the classroom
Learning within the ICU environment is challenging, not only because of the complexity and rapid pace of patient care but also because of the breadth of knowledge required to care for critically ill patients. A number of critical care organizations have undertaken the task of defining learning objectives for trainees in the critical care setting[33-35]. Given the time constraints associated with clinical practice it is not possible to expose trainees to every topic relevant to critical care. Lectures are a common method of covering a “core curriculum” in critical care yet the efficiency and efficacy of this educational approach is low. It has been shown that learners’ attention decreases after only ten minutes and learners only remember approximately 20% of the transmitted content following a lecture. Consequently, there is need for new educational methods that result in more efficient and effective knowledge transmission than provided in traditional conference room lectures. These new methods should not be limited to the transmission of purely factual knowledge, but should provide the opportunity to apply this knowledge to problem solving in practice.
The “flipped classroom” is a novel instructional paradigm designed to increase learning by integrating in-class experience with out-of-class learning. In this paradigm, learners first gain exposure to new material individually, usually via reading or watching instructional videos. Formal teaching time is then used for learning-centered activities that build on the pre-class work rather than providing traditional lectures. During the formal teaching time, an instructor facilitates trainee-driven discussion of the material via question and answer, discussion, case studies, problem-based learning, and other face-to-face activities. By applying their new knowledge with the guidance of a facilitator, trainees have access to immediate feedback from peers and faculty, which will help them more readily recognize and correct errors in thinking. These “active learning” activities will allow for complex problem solving, peer interaction, and better prepare learners to function independently.
The flipped classroom paradigm is particularly well suited for the ICU learning environment, where acquisition of “core” critical care knowledge is necessary before progressing to the more complex clinical problem solving that is required for patient care. Practicing clinical decision-making in the classroom improves knowledge retention and has the further inherent advantage that the trainees can learn from their mistakes in a safe environment without endangering patients. The flexibility afforded by the flipped classroom allows for learning despite the unpredictability of the ICU environment, as learning materials may be made available to learners regardless of clinical demands or their particular shift schedule. The mechanism used to expose learners to new learning material can vary from simple textbook readings to lecture videos or podcasts. Pre-class assignments can be varied based on differing backgrounds and training levels of the learners. If video-based educational materials are used, they can be paused and replayed, allowing learners to move through the material at their own pace. Using varied formats to present educational content can also support differences in individual learning styles and preferences.
There is no single approach to flipping the classroom in practice. The means of delivering educational content and the ways in which face-to-face activities are used can vary with the subject matter, characteristics of the learners, preferences of the instructor, and available resources. It is essential however that in- and out- of class activities are carefully integrated to optimize the beneficial effects and encourage trainees to be prepared for the in-class activities. Well written objectives that inform the trainees what they are going to learn and how they are going to be assessed should be clearly linked to each individualized learning task.
Some practical tips for flipping the classroom include: (1) learners must be provided resources to acquire factual knowledge prior to the classroom phase. Providing short educational videos, many of which are readily available via Open Educational Resources are effective, provided they are matched to the desired learning objectives. The use of other, non-digital material is, however, equally possible; (2) implemented technology should ideally be easily-accessible and ideally already be familiar to the learners; (3) activities both in the pre-class and classroom phases must be well-structured. Trainees will accept demands for learning more easily when content and time requirements are firmly defined; (4) incentive systems should be implemented to encourage trainees to complete the pre-class activities before the classroom phase. For example, short multiple choice quizzes could be given with correct answers; (5) methods of assessment should be implemented to provide feedback to the trainees on their knowledge acquisition and learning performance achieved through the pre-class and classroom activities; and (6) feedback from trainees is essential to the success of the flipped classroom. Trainees should be encouraged to provide this feedback regularly throughout the learning process including the pre-class activities.
While there is limited literature to date exploring the flipped classroom model in the context of critical care education, evidence of its efficacy from other areas of undergraduate and graduate medical education do appear promising[38,39,41].
Improving procedural training
Critical care trainees must gain procedural competence in a number of technical domains, including vascular access, airway management, bronchoscopy, chest tube placement, and critical care ultrasonography. A fundamental challenge in procedural training is to find the optimal balance between providing educational opportunities for trainees and ensuring safe, efficient patient care. While it can be argued that it is inappropriate to allow an inexperienced trainee to perform a procedure in a high-risk situation, such as in the care of a critically ill patient, it can also be argued that unless trainees are allowed such practice, there will be fewer and fewer clinicians competent to perform life-saving procedures. Since the introduction of the duty-hour limits, concern has arisen that trainees may not be getting as much experience in procedural skills as they once did[42,43]. Given the rapidly changing landscape of critical care practice, with an ever increasing number of diagnostic and therapeutic tools to master, it is necessary that trainees receive high-quality procedural teaching. Although a variety of frameworks for procedural teaching exist in the literature, many training programs continue to rely on an apprenticeship model. The trainers themselves may have varying amounts of expertise with a given procedure which further complicates training. To address these challenges, the literature supports a standardized approach to procedural education with performance assessed and mastery demonstrated away from the clinical setting[44-46]. Multifaceted learning strategies that incorporate computer-based learning, task trainers, and simulation to provide the necessary conceptual and technical understanding of the fundamentals of procedures, followed by observing and then performing procedures on healthy patients in the operating room or other elective situations, have been recommended to facilitate procedural learning before the trainee attempts high risk procedures on critically ill patients[47,48]. Computer-based instruction can provide essential information about a procedure, including its indications, required equipment, and procedural steps. Computer-based learning has been shown to be an effective alternative for providing fundamentals of central line placement, basic ultrasound training and acquisition of knowledge required for difficult airway management[49-52]. After learners receive fundamental information on a procedure, task trainers and simulation can be employed to teach technical skills. Hands-on approaches offer learners physical training in performing procedures and opportunities to rehearse these skills in context without the risk of patient harm. A number of studies have demonstrated that deliberate practice with the use of simulation can improve skills in the clinical environment[53-55]. Use of procedural checklists can be helpful during the technical training to evaluate each step in procedural performance and to appropriately modify behaviors. Adjunct technology can also be utilized to facilitate procedural learning and performance. For example ultrasound use can improve understanding of relevant anatomy and is supported by data demonstrating superiority in overall success and complication reduction for CVL placement, arterial catheter insertion, thoracentesis, and paracentesis. Use of video laryngoscopy, which provides shared views of the airway, improves trainer and trainee collaboration, resulting in more rapid learning curves and increased intubation success rates[55,56].
Even with prior simulation experience, it may be unrealistic to expect trainees to move directly into a dynamic environment such as the ICU and perform procedural skills, especially during crisis situations. Controlled patient encounters that involve performing procedures under elective conditions with supervision by experienced clinicians may help to translate skills that were learned in simulation exercises into the clinical environment in a safe manner. Just-in-time training (JITT) has also been proposed as a training approach to translate learning from the controlled simulation environment into the actual patient setting. With JITT, trainees practice procedural skills and refresh muscle memory immediately prior to performing the procedure on a patient. The JITT concept is based on literature showing that both knowledge and technical skills decay over time and therefore the clinician benefits from training “just-in-time”, moments before the procedure. It has also been described in reducing undesirable outcomes in acute procedures, including CPR skills, endotracheal tube placement, central venous catheter insertion and lumbar puncture[58-60]. In addition JITT has been shown to reduce the time to successful completion of procedures, and may even play a role in long-term retention of procedural skills.
To facilitate JITT all that is needed is a low-fidelity task trainer that is specific to the chosen procedure. Ideally, this task trainer should be a portable model that can be stored and easily accessed in the critical care practice environment. If possible, authentic equipment should be set aside and dedicated for JITT. Prior to beginning the procedure on a patient, the preceptor instructs the learner to perform the procedure on a task trainer as if it were a real patient using a checklist of critical actions. For example, in the case of utilizing JITT for endotracheal intubation, skills that can be practiced include: Proper positioning of the patient; effective bag-valve-mask ventilation techniques; correctly maneuvering a laryngoscope; visualizing the vocal cords; inserting the endotracheal tube through the glottic opening; correctly using airway adjuncts, if needed, as rescue airway devices (e.g., laryngeal mask airway or bougie).
The preceptor monitors the learner’s performance with each skill and provides continuous formative feedback. The learner is encouraged to ask questions throughout the process and the preceptor is provided the opportunity to correct mistakes in real-time and optimize performance which will be immediately transferrable to the actual procedure moments later. When the preceptor is satisfied with the trainee’s demonstrated procedural skills, they can proceed to performing the procedure on the patient. Using this approach, trainee confidence and procedural readiness should improve thereby increasing patient safety.
Facilitating education during handover
Effective handovers allow a team of multiple providers to deliver safe and high quality care by ensuring continuity. Despite its crucial role in ensuring safe and effective patient care, a number of studies have characterized the process as haphazard and error prone and have linked substandard handovers to adverse events. Critically ill patients are particularly vulnerable to ineffective handovers given their complex clinical history and severity of their condition. Patient handover has been identified as a priority to ensure patient safety and the ACGME requires that all training programs monitor handovers. A growing number of studies have proposed educational interventions to improve handovers; however studies that demonstrate improvement in actual patient outcomes based on these interventions are still limited. A recent systematic review found that there were four primary methods for teaching handovers: (1) providing online materials such as videos, texts, and protocols; (2) lectures and group sessions; (3) simulation activities; and (4) role-playing exercises. Common content themes of these educational handover interventions include: (1) information management; (2) team-work, leadership; and communication; and (3) error awareness and professional behavior.
In addition to facilitating continuity of care, handovers can also provide an active learning opportunity[67,68]. Handovers, by nature, are associated with clinical uncertainty, making it important for participants to reduce uncertainty through active dialogue. This active dialogue promotes learning through discussions of approaches to diagnosis and management of specific conditions and may also occur through feedback on clinical actions taken in the preceding shift. For example, if a trainee admits a patient and makes a preliminary diagnosis that was confirmed after their shift, he or she should be provided this feedback, thus affirming his or her diagnostic approach. This post-shift feedback can encourage trainees to reflect on the results of their clinical actions even when they are not present to see them unfold. This feedback approach is especially important given current duty hour restrictions.
In addition to relying on the clinical exchange of information and discussions regarding patient care as a way to promote learning, there are more deliberate methods which can be employed to ensure that learning takes place during handovers. The most obvious of these is direct supervision of the handover process by experienced clinicians (faculty, fellows, etc.) who can provide guidance to trainees. It is important to recognize that while experienced clinicians can supervise the handover, faculty may have limited experience with new handover processes and faculty development may be required before implementation. Supervision of handovers serves as a way to ensure not only that critical patient information is being communicated, but also as a means to answer the clinical questions that arise during the course of the shift, thereby ensuring that all learners have access to clinical teaching. In addition to direct supervision, another approach to enhance the learning process during handovers is to supplement it with short educational modules tailored to a current case or a set of cases that are commonly encountered. With this approach, the handover is linked to practice-based learning and improvement (an ACGME core competency), allowing learners to integrate new knowledge into their clinical practice.
One way for clinicians to optimize the supervision of handovers and associated teaching is to stratify them according to case complexity. Severity of illness, worsening disease trajectory, or incompleteness of the medical history or diagnostic workup, for example, are factors that increase the importance of an effective handover for ensuring care of a vulnerable patient. In contrast, the handover of a stable or otherwise well-characterized patient, even if the handover is performed poorly, is less likely to lead to an adverse event. Factors that increase the risk of an ineffective handover include the degree of familiarity of the clinicians with the patient, the type of handover, and the level of experience of the clinicians involved. At a minimum the handovers for complex, critically ill patients should be supervised until trainees have demonstrated the ability to perform them effectively and consistently. Even after competency has been demonstrated there also may be benefits to continuing some level of handover supervision. From the standpoint of improving patient safety, a skilled observer can reduce handover errors by providing real-time feedback to the participants, thereby contributing to enhanced accuracy and encouraging experiential learning.
Handovers can also be used to evaluate trainee performance and provide formative feedback, as it provides an opportunity to directly observe behaviors related to communication as well as competencies such as professionalism. Evaluations can occur during the handover or as a summary at the end of a rotation. While real-time evaluations have the benefit of providing immediate feedback, summary evaluations at the end of a rotation have the advantage of enabling trainees to assess improvements in handover performance over time and after repeated interactions. Ideally, handover evaluation should be competency-based and linked to specific, observable behaviors. The quality of the handover content can be assessed using questions such as, “was anticipatory guidance provided and easy to interpret?” or “did ‘to-do’ items include a rationale?” Evaluating the receivers of handover content may be more difficult, but observable behaviors could include actions indicating active engagement such as asking questions, taking notes and maintaining eye contact. In addition to monitoring the quality of verbal exchange between senders and receivers of the handover, written documentation to facilitate the handover can be assessed for accuracy and readability. Often, a structured template is used to facilitate the transfer of verbal information during handovers. However, documents that are used to support handovers, whether on paper or generated electronically often contain errors, which most often result from a failure to keep these documents up-to-date. Therefore, examining the accuracy of the information in the document, and making certain that key elements such as medications, allergies and code status are updated, will help to ensure the accuracy of information transmission during handover.
Handovers consist of a series of complex communication tasks and it is critical that trainees acquire the specific skills required to both give and receive them. These skills include developing strategies for information management, managing handover dialogue through active listening, asking questions, and collaborating to generate a shared understanding for optimal exchange of information necessary to guide patient care. The skills required for effective handover communication will improve with greater supervision and feedback from experienced clinicians.
In-situ team based training
The management of critically ill patients requires multidisciplinary teams to work collaboratively. Core elements of team performance (e.g., leadership, adaptability, mutual trust, closed-loop communication) impact the quality and safety of patient care[71-73]. Despite the importance of team performance on patient outcomes, providing training specifically designed to improve team dynamics is a relatively new concept for many medical specialties including critical care. Evidence suggests that to improve multidisciplinary team performance it is necessary to train as a multidisciplinary team. In situ simulation training has been recognized as a technique to improve multidisciplinary team performance[74,75]. Training within the actual critical care environment allows teams to test their effectiveness in a controlled manner and to interrogate departmental and hospital processes in real time and in real locations. In addition, in situ simulation has the advantage that it can be incorporated into the activities of daily practice which is less disruptive to staffing.
Team composition during in situ simulation training should reflect normal working practice including different professions and levels of training. Team members should train in their normal roles and at their own skill level and scope of practice-clinicians should not be expected to perform a skill outside of their scope of practice. While it is possible to teach technical skills using multidisciplinary in situ simulation, it is arguably better suited to teaching nontechnical skills[74,75]. Although it is possible to spend a large amount of money on high fidelity simulation equipment, it is not essential. A basic platform which is adequate for most critical care simulations only requires a vital sign monitor with adjustable parameters, a clinical bed space, and some clinical consumables. There is little evidence that enhanced fidelity creates a better learning environment. In fact, enhanced fidelity may actually detract from the learning environment depending on the learning objectives. In many cases, real people playing the role of a patient (so called “standardized patient actors”) are just as effective and are more realistic especially for scenarios focusing on communication and teamwork. Audiovisual equipment can also be useful for recording the simulation to enhance discussion during debriefing. However, this is not essential. It is feasible to use a phone camera or tablet as a low-cost solution. If video or audio recording is performed, participants must provide consent prior to the session and storage and usage of the electronic media must be controlled.
When initiating an in situ simulation program it is generally best to start with simple scenarios aimed at participants’ readiness level which will invoke challenge rather than frustration or embarrassment. More complicated or complex scenarios can then be introduced once the program has been established. Regular repetition of commonly occurring scenarios such as cardiac arrest, emergency intubation, and sepsis management can be used to reinforce learning and teamwork by applying a “practice until perfect” approach. Complexity may also be added to simple scenarios through the use of embedded participants who play a role that is intended to add cognitive noise or conflict to the scenario.
It is important to spend time preparing the simulation participants prior to the scenario. This “prebriefing” is a time when the facilitator describes the purpose of the simulation, the learning objectives, the process of debriefing, and clarifies expectations. This prebriefing should also include a confidentiality agreement and an explanation of the rules of simulation engagement, including a description of the simulated “patient”, the limitations of the simulation and how they will be overcome, what equipment is available, how drugs and fluids can be “administered,” and safety rules (e.g., the use of a live defibrillator). Adopting a “stop word”, which will immediately terminate the simulation, is also important to ensure participant safety. It is also essential that an environment of trust is created early on, typically during the prebriefing. If participants feel safe and understand how they are expected to participate as a team prior to the session, they will have the maximum opportunity for learning within the time available.
Debriefing is an essential, and arguably the most important, element of simulation because it encourages self-reflection which promotes a deeper level of understanding and thereby increases the likelihood of successful transfer of acquired knowledge and skills to the clinical setting. Debriefing should take place immediately after the simulation especially in the critical care environment where participants must return to clinical responsibilities. When allocating time for an in situ simulation session, it is important to allocate sufficient time for debriefing. As a simple guideline, the debriefing session should be allocated at least the same amount of time as the duration of the simulation scenario itself. Standardized debriefing formats have been suggested to ensure that key components are covered within the limited time frame. The debriefing session should be designed to achieve the learning objectives and tailored to the specific participant and team characteristics. Learning objectives are often specified beforehand, but may also evolve within the simulation. With pre-specified objectives, such as improving particular team behaviors, the debriefing session affords the opportunity to examine how closely participants’ performance approached the goal target, and furthermore, what additional learning is required to bridge the gaps between performance and the target. With evolving objectives, participants may be asked to reflect on the observed evolution of the scenario and to evaluate how the behaviors, attitudes, and choices demonstrated in the simulation relate to real life situations[79,80].
An individual should be designated to facilitate the debriefing process. The facilitator should not “script” the debriefing process but rather should provide sufficient discussion prompts and tools to ensure that participants actively engage in critical analysis, shared reflection and application of the experience to clinical practice. The facilitator is also responsible for ensuring that time and pace is managed effectively. When facilitating a debriefing some simple approaches can be very effective: Start by asking open ended questions such as “how did it go?”. As participants respond, rephrase their responses back to them as skills that are part of the learning objectives. Next ask, “what could you do better?”. When asked this question, the participants will invariably bring up many management areas that you were going to mention. Finally inquire, “what will you do differently next time?”. This will help the trainees focus on making meaningful but simple changes for the next time a similar situation is encountered. The facilitator should close the debriefing by prompting the participants for questions or addressing any specific issues that were not discussed with open ended questions. Debriefing is a time when participants may feel most vulnerable to criticism in front of their peers. This vulnerability may be particularly pronounced with in situ simulation since participants work closely with each other. For this reason, creating a friendly and supportive atmosphere is imperative. In summary, in situ simulation has the potential to improve patient safety by strengthening skills in teamwork and communication that are essential for well-functioning critical care teams.
Technological advances and evolving demands in medical care have led to challenges in ensuring adequate training for providers of critical care. Evidence suggests that reliance on the traditional experience-based model alone is insufficient for ensuring quality and safety in patient care. Evidence-based approaches for improving the efficiency and efficacy of critical care education, have been developed and should be integrated into training programs. While a variety of such approaches are described in this paper they share common characteristics. These include utilizing methods to rapidly identify learner needs, teaching directly to those needs, and providing specific feedback on performance. In addition these approaches emphasize active learning activities and integrate educational experiences from the classroom and clinical settings. Finally such approaches share the advantage that can be incorporated into the daily practice of critical care without substantial cost, workflow disruption or compromise in the quality of patient care. Moving forward, it is imperative that critical care educators keep abreast of emerging educational technologies including personalized learning, mobile technologies and learning analytics. While there is sparse literature describing the benefits and limitations, such technology has the potential to enhance learning and clinical competence within the critical care setting.
Manuscript source: Invited manuscript
Specialty type: Critical care medicine
Country of origin: United States
Peer-review report classification
Grade A (Excellent): 0
Grade B (Very good): B
Grade C (Good): 0
Grade D (Fair): 0
Grade E (Poor): 0
P- Reviewer: Kandil SB S- Editor: Qiu S L- Editor: A E- Editor: Li D
Sabri N, Sun NZ, Cummings BA, Jayaraman D. The Perceived Effect of Duty Hour Restrictions on Learning Opportunities in the Intensive Care Unit.J Grad Med Educ. 2015;7:48-52.
Tainter CR, Wong NL, Bittner EA. Innovative strategies in critical care education.J Crit Care. 2015;30:550-556.
Croley WC, Rothenberg DM. Education of trainees in the intensive care unit.Crit Care Med. 2007;35:S117-S121.
Almoosa KF, Goldenhar LM, Puchalski J, Ying J, Panos RJ. Critical care education during internal medicine residency: a national survey.J Grad Med Educ. 2010;2:555-561.
Barrett H, Bion JF. An international survey of training in adult intensive care medicine.Intensive Care Med. 2005;31:553-561.
Peters M, Ten Cate O. Bedside teaching in medical education: a literature review.Perspect Med Educ. 2014;3:76-88.
Kim MM, Barnato AE, Angus DC, Fleisher LA, Kahn JM. The effect of multidisciplinary care teams on intensive care unit mortality.Arch Intern Med. 2010;170:369-376.
Lane D, Ferri M, Lemaire J, McLaughlin K, Stelfox HT. A systematic review of evidence-informed practices for patient care rounds in the ICU*.Crit Care Med. 2013;41:2015-2029.
Cox CE, Carson SS, Ely EW, Govert JA, Garrett JM, Brower RG, Morris DG, Abraham E, Donnabella V, Spevetz A. Effectiveness of medical resident education in mechanical ventilation.Am J Respir Crit Care Med. 2003;167:32-38.
Wilcox SR, Seigel TA, Strout TD, Schneider JI, Mitchell PM, Marcolini EG, Cocchi MN, Smithline HA, Lutfy-Clayton L, Mullen M. Emergency medicine residents’ knowledge of mechanical ventilation.J Emerg Med. 2015;48:481-491.
Mueller UW, Potter JM. Polymerization of human transcortin in plasma.J Steroid Biochem. 1984;20:1261-1266.
Boots RJ, Egerton W, McKeering H, Winter H. They just don’t get enough! Variable intern experience in bedside procedural skills.Intern Med J. 2009;39:222-227.
Marshall JC, Kwong W, Kommaraju K, Burns KE. Determinants of Citation Impact in Large Clinical Trials in Critical Care: The Role of Investigator-Led Clinical Trials Groups.Crit Care Med. 2016;44:663-670.
Cleland JA, Ross S, Miller SC, Patey R. “There is a chain of Chinese whispers”: empirical data support the call to formally teach handover to prequalification doctors.Qual Saf Health Care. 2009;18:267-271.
Sawatsky AP, Mikhael JR, Punatar AD, Nassar AA, Agrwal N. The effects of deliberate practice and feedback to teach standardized handoff communication on the knowledge, attitudes, and practices of first-year residents.Teach Learn Med. 2013;25:279-284.
McCallister JW, Gustin JL, Wells-Di Gregorio S, Way DP, Mastronarde JG. Communication skills training curriculum for pulmonary and critical care fellows.Ann Am Thorac Soc. 2015;12:520-525.
Hope AA, Hsieh SJ, Howes JM, Keene AB, Fausto JA, Pinto PA, Gong MN. Let’s Talk Critical. Development and Evaluation of a Communication Skills Training Program for Critical Care Fellows.Ann Am Thorac Soc. 2015;12:505-511.
Hayes CW, Rhee A, Detsky ME, Leblanc VR, Wax RS. Residents feel unprepared and unsupervised as leaders of cardiac arrest teams in teaching hospitals: a survey of internal medicine residents.Crit Care Med. 2007;35:1668-1672.
Chudgar SM, Cox CE, Que LG, Andolsek K, Knudsen NW, Clay AS. Current teaching and evaluation methods in critical care medicine: has the Accreditation Council for Graduate Medical Education affected how we practice and teach in the intensive care unit?Crit Care Med. 2009;37:49-60.
Irby DM, Wilkerson L. Teaching when time is limited.BMJ. 2008;336:384-387.
Neher JO, Gordon KC, Meyer B, Stevens N. A five-step “microskills” model of clinical teaching.J Am Board Fam Pract. 1992;5:419-424.
Aagaard E, Teherani A, Irby DM. Effectiveness of the one-minute preceptor model for diagnosing the patient and the learner: proof of concept.Acad Med. 2004;79:42-49.
Farrell SE, Hopson LR, Wolff M, Hemphill RR, Santen SA. What’s the Evidence: A Review of the One-Minute Preceptor Model of Clinical Teaching and Implications for Teaching in the Emergency Department.J Emerg Med. 2016;51:278-283.
Wilkerson L, Sarkin RT. Arrows in the Quiver: evaluation of a workshop on ambulatory teaching.Acad Med. 1998;73:S67-S69.
McGee S. A piece of my mind. Bedside teaching rounds reconsidered.JAMA. 2014;311:1971-1972.
Gonzalo JD, Heist BS, Duffy BL, Dyrbye L, Fagan MJ, Ferenchick G, Harrell H, Hemmer PA, Kernan WN, Kogan JR. The art of bedside rounds: a multi-center qualitative study of strategies used by experienced bedside teachers.J Gen Intern Med. 2013;28:412-420.
Bhave M, Brzezinski M. Teaching in the ICU: A Comprehensive Review.ICU Director. 2013;4:270-278.
Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education.Acad Emerg Med. 2008;15:1010-1016.
Green GM, Chen EH. Top 10 ideas to improve your bedside teaching in a busy emergency department.Emerg Med J. 2015;32:76-77.
Watling CJ, Lingard L. Toward meaningful evaluation of medical trainees: the influence of participants’ perceptions of the process.Adv Health Sci Educ Theory Pract. 2012;17:183-194.
Gonzalo JD, Heist BS, Duffy BL, Dyrbye L, Fagan MJ, Ferenchick G, Harrell H, Hemmer PA, Kernan WN, Kogan JR. Content and timing of feedback and reflection: a multi-center qualitative study of experienced bedside teachers.BMC Med Educ. 2014;14:212.
Bion JF, Barrett H. Development of core competencies for an international training programme in intensive care medicine.Intensive Care Med. 2006;32:1371-1383.
Buckley JD, Addrizzo-Harris DJ, Clay AS, Curtis JR, Kotloff RM, Lorin SM, Murin S, Sessler CN, Rogers PL, Rosen MJ. Multisociety task force recommendations of competencies in Pulmonary and Critical Care Medicine.Am J Respir Crit Care Med. 2009;180:290-295.
Fessler HE, Addrizzo-Harris D, Beck JM, Buckley JD, Pastores SM, Piquette CA, Rowley JA, Spevetz A. Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: executive summary from the Multi-Society Working Group.Crit Care Med. 2014;42:2290-2291.
Hartley J, Cameron A. Some Observations on the Efficiency of Lecturing.Educ Rev. 1967;20:30-37.
Prober CG, Khan S. Medical education reimagined: a call to action.Acad Med. 2013;88:1407-1410.
Tainter CR, Wong NL, Cudemus-Deseda GA, Bittner EA. The “Flipped Classroom” Model for Teaching in the Intensive Care Unit: Rationale, Practical Considerations, and an Example of Successful Implementation.J Intensive Care Med. 2016; Epub ahead of print.
Tolks D, Schäfer C, Raupach T, Kruse L, Sarikas A, Gerhardt-Szép S, Kllauer G, Lemos M, Fischer MR, Eichner B. An Introduction to the Inverted/Flipped Classroom Model in Education and Advanced Training in Medicine and in the Healthcare Professions.GMS J Med Educ. 2016;33:Doc46.
Kleinpell R, Ely EW, Williams G, Liolios A, Ward N, Tisherman SA. Web-based resources for critical care education.Crit Care Med. 2011;39:541-553.
Young TP, Bailey CJ, Guptill M, Thorp AW, Thomas TL. The flipped classroom: a modality for mixed asynchronous and synchronous learning in a residency program.West J Emerg Med. 2014;15:938-944.
Dehmer JJ, Amos KD, Farrell TM, Meyer AA, Newton WP, Meyers MO. Competence and confidence with basic procedural skills: the experience and opinions of fourth-year medical students at a single institution.Acad Med. 2013;88:682-687.
Promes SB, Chudgar SM, Grochowski CO, Shayne P, Isenhour J, Glickman SW, Cairns CB. Gaps in procedural experience and competency in medical school graduates.Acad Emerg Med. 2009;16 Suppl 2:S58-S62.
Nestel D, Groom J, Eikeland-Husebø S, O’Donnell JM. Simulation for learning and teaching procedural skills: the state of the science.Simul Healthc. 2011;6 Suppl:S10-S13.
Huang GC, McSparron JI, Balk EM, Richards JB, Smith CC, Whelan JS, Newman LR, Smetana GW. Procedural instruction in invasive bedside procedures: a systematic review and meta-analysis of effective teaching approaches.BMJ Qual Saf. 2016;25:281-294.
Lammers RL, Davenport M, Korley F, Griswold-Theodorson S, Fitch MT, Narang AT, Evans LV, Gross A, Rodriguez E, Dodge KL. Teaching and assessing procedural skills using simulation: metrics and methodology.Acad Emerg Med. 2008;15:1079-1087.
McSparron JI, Michaud GC, Gordan PL, Channick CL, Wahidi MM, Yarmus LB, Feller-Kopman DJ, Makani SS, Koenig SJ, Mayo PH. Simulation for Skills-based Education in Pulmonary and Critical Care Medicine.Ann Am Thorac Soc. 2015;12:579-586.
Chenkin J, Lee S, Huynh T, Bandiera G. Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training.Acad Emerg Med. 2008;15:949-954.
Xiao Y, Seagull FJ, Bochicchio GV, Guzzo JL, Dutton RP, Sisley A, Joshi M, Standiford HC, Hebden JN, Mackenzie CF. Video-based training increases sterile-technique compliance during central venous catheter insertion.Crit Care Med. 2007;35:1302-1306.
Platz E, Liteplo A, Hurwitz S, Hwang J. Are live instructors replaceable? Computer vs. classroom lectures for EFAST training.J Emerg Med. 2011;40:534-538.
Bello G, Pennisi MA, Maviglia R, Maggiore SM, Bocci MG, Montini L, Antonelli M. Online vs live methods for teaching difficult airway management to anesthesiology residents.Intensive Care Med. 2005;31:547-552.
Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit.Crit Care Med. 2009;37:2697-2701.
Huang GC, Newman LR, Schwartzstein RM, Clardy PF, Feller-Kopman D, Irish JT, Smith CC. Procedural competence in internal medicine residents: validity of a central venous catheter insertion assessment instrument.Acad Med. 2009;84:1127-1134.
De Jong A, Molinari N, Conseil M, Coisel Y, Pouzeratte Y, Belafia F, Jung B, Chanques G, Jaber S. Video laryngoscopy versus direct laryngoscopy for orotracheal intubation in the intensive care unit: a systematic review and meta-analysis.Intensive Care Med. 2014;40:629-639.
Kory P, Guevarra K, Mathew JP, Hegde A, Mayo PH. The impact of video laryngoscopy use during urgent endotracheal intubation in the critically ill.Anesth Analg. 2013;117:144-149.
Weaver SJ, Newman-Toker DE, Rosen MA. Reducing cognitive skill decay and diagnostic error: theory-based practices for continuing education in health care.J Contin Educ Health Prof. 2012;32:269-278.
Cheng A, Brown LL, Duff JP, Davidson J, Overly F, Tofil NM, Peterson DT, White ML, Bhanji F, Bank I. Improving cardiopulmonary resuscitation with a CPR feedback device and refresher simulations (CPR CARES Study): a randomized clinical trial.JAMA Pediatr. 2015;169:137-144.
Nishisaki A, Donoghue AJ, Colborn S, Watson C, Meyer A, Brown CA, Helfaer MA, Walls RM, Nadkarni VM. Effect of just-in-time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit.Anesthesiology. 2010;113:214-223.
Kessler D, Pusic M, Chang TP, Fein DM, Grossman D, Mehta R, White M, Jang J, Whitfill T, Auerbach M. Impact of Just-in-Time and Just-in-Place Simulation on Intern Success With Infant Lumbar Puncture.Pediatrics. 2015;135:e1237-e1246.
Braga MS, Tyler MD, Rhoads JM, Cacchio MP, Auerbach M, Nishisaki A, Larson RJ. Effect of just-in-time simulation training on provider performance and patient outcomes for clinical procedures: a systematic review.BMJ Simulation and Technology Enhanced Learning. 2015;1:94-100.
Lane-Fall MB, Collard ML, Turnbull AE, Halpern SD, Shea JA. ICU Attending Handoff Practices: Results From a National Survey of Academic Intensivists.Crit Care Med. 2016;44:690-698.
Colvin MO, Eisen LA, Gong MN. Improving the Patient Handoff Process in the Intensive Care Unit: Keys to Reducing Errors and Improving Outcomes.Semin Respir Crit Care Med. 2016;37:96-106.
Wohlauer MV, Arora VM, Horwitz LI, Bass EJ, Mahar SE, Philibert I. The patient handoff: a comprehensive curricular blueprint for resident education to improve continuity of care.Acad Med. 2012;87:411-418.
Bhabra G, Mackeith S, Monteiro P, Pothier DD. An experimental comparison of handover methods.Ann R Coll Surg Engl. 2007;89:298-300.
Reader TW, Flin R, Mearns K, Cuthbertson BH. Developing a team performance framework for the intensive care unit.Crit Care Med. 2009;37:1787-1793.
Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature.Acta Anaesthesiol Scand. 2009;53:143-151.
Reader TW, Cuthbertson BH. Teamwork and team training in the ICU: where do the similarities with aviation end?Crit Care. 2011;15:313.
Patterson MD, Geis GL, Falcone RA, LeMaster T, Wears RL. In situ simulation: detection of safety threats and teamwork training in a high risk emergency department.BMJ Qual Saf. 2013;22:468-477.
Spurr J, Gatward J, Joshi N, Carley SD. Top 10 (+1) tips to get started with in situ simulation in emergency and critical care departments.Emerg Med J. 2016;33:514-516.
Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning.Med Educ. 2012;46:636-647.
Rudolph JW, Raemer DB, Simon R. Establishing a safe container for learning in simulation: the role of the presimulation briefing.Simul Healthc. 2014;9:339-349.
Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More Than One Way to Debrief: A Critical Review of Healthcare Simulation Debriefing Methods.Simul Healthc. 2016;11:209-217.
Patterson MD, Blike GT, Nadkarni VM. EditorsIn: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. SourceAdvances in Patient Safety: New Directions and Alternative Approaches.
Rockville (MD): Agency for Healthcare Research and Quality (US); In Situ Simulation: Challenges and Results; 2008;.
Spector JM. Emerging Educational Technologies and Research Directions.Educ Technol Soc. 2013;16:21-30.