1
|
Metz A, Jensen TM, Dicharry L, Farley AB. Building trusting relationships in teams to support evidence use and implementation in human services: feasibility and acceptability of a training and coaching approach. FRONTIERS IN HEALTH SERVICES 2024; 4:1353741. [PMID: 39726895 PMCID: PMC11670258 DOI: 10.3389/frhs.2024.1353741] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Accepted: 11/20/2024] [Indexed: 12/28/2024]
Abstract
Background Professionals who provide implementation support in human service systems describe relationships as being critical to support evidence use; however, developing trusting relationships are not strongly featured in implementation science literature. The aims of this study were to (a) assess the feasibility and acceptability of a theory-driven training and coaching approach for building trusting relationships among members of an implementation team who were supporting the implementation of an evidence-informed program in a public child welfare system in the United States and (b) gauge the initial efficacy of the approach in terms of the development of trusting relationships and subsequent implementation outcomes. Methods Consistent with a convergent mixed-methods approach, we collected both quantitative and qualitative data to address our research questions. Quantitative methods included an adapted version of the Trusting Relationship Questionnaire, a seven-item measure of psychological safety, and items designed to measure acceptability of the training. Qualitative data were collected through semi-structured interviews with participants. Results Sixteen individuals participated in the program, consisting of a kick-off training event, five monthly training modules, and five monthly coaching sessions with implementation team leads. Session attendance rates and self-reported satisfaction highlight the general feasibility and acceptability of the training and coaching approach. On average, participants also reported significant increases over time in their perceptions that they were trusted by their team. Results from in-depth interviews further indicated the efficacy of the program in terms of cultivating trust among team members and promoting several elements that were theorized to link trusting relationships to implementation outcomes. Discussion Findings suggest the training and coaching approach for trust building was acceptable and feasible. Additionally, results indicate the value of the approach in building trust among implementation partners to increase commitment to implementation efforts, promote a culture of learning and psychological safety, and increase participants' sense of capability and motivation for supporting implementation.
Collapse
Affiliation(s)
- Allison Metz
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Todd M. Jensen
- School of Education, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Lacy Dicharry
- College of Human Sciences and Education, Louisiana State University, Baton Rouge, LA, United States
| | - Amanda B. Farley
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| |
Collapse
|
2
|
McKay HA, Kennedy SG, Macdonald HM, Naylor PJ, Lubans DR. The Secret Sauce? Taking the Mystery Out of Scaling-Up School-Based Physical Activity Interventions. J Phys Act Health 2024; 21:731-740. [PMID: 38936808 DOI: 10.1123/jpah.2024-0274] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Accepted: 04/16/2024] [Indexed: 06/29/2024]
Abstract
Over the last 4 decades, physical activity researchers have invested heavily in determining "what works" to promote healthy behaviors in schools. Single and multicomponent school-based interventions that target physical education, active transportation, and/or classroom activity breaks effectively increased physical activity among children and youth. Yet, few of these interventions are ever scaled-up and implemented under real-world conditions and in diverse populations. To achieve population-level health benefits, there is a need to design school-based health-promoting interventions for scalability and to consider key aspects of the scale-up process. In this opinion piece, we aim to identify challenges and advance knowledge and action toward scaling-up school-based physical activity interventions. We highlight the key roles of planning for scale-up at the outset, scale-up pathways, trust among partners and program support, program adaptation, evaluation of scale-up, and barriers and facilitators to scaling-up. We draw upon our experience scaling-up effective school-based interventions and provide a solid foundation from which others can work toward bridging the implementation-to-scale-up gap.
Collapse
Affiliation(s)
- Heather A McKay
- Active Aging Research Team, University of British Columbia, Vancouver, BC, Canada
- Department of Family Practice, University of British Columbia, Vancouver, BC, Canada
| | - Sarah G Kennedy
- School of Health Sciences, Western Sydney University, Penrith, NSW, Australia
| | - Heather M Macdonald
- Active Aging Research Team, University of British Columbia, Vancouver, BC, Canada
- Department of Family Practice, University of British Columbia, Vancouver, BC, Canada
| | - Patti-Jean Naylor
- School of Exercise Science, Physical and Health Education, Faculty of Education, University of Victoria, Victoria, BC, Canada
| | - David R Lubans
- Centre for Active Living and Learning, College of Human and Social Futures, University of Newcastle, Callaghan, NSW, Australia
- Active Living Research Program, Hunter Medical Research Institute, New Lambton Heights, NSW, Australia
- Faculty of Sport and Health Sciences, University of Jyväskylä, Jyväskylä, Finland
| |
Collapse
|
3
|
Triplett NS, AlRasheed R, Johnson C, McCabe CJ, Pullmann MD, Dorsey S. Supervisory Alliance as a Moderator of the Effects of Behavioral Rehearsal on TF-CBT Fidelity: Results from a Randomized Trial of Supervision Strategies. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2024; 51:254-267. [PMID: 38157131 PMCID: PMC11162559 DOI: 10.1007/s10488-023-01334-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/13/2023] [Indexed: 01/03/2024]
Abstract
Exposure is an important element of treatment for many evidence-based treatments but can be challenging to implement. Supervision strategies to support exposure delivery may be an important tool to facilitate the use of exposure techniques; however, they must be considered and used in the context of the supervisory alliance. The present study examined relations between supervisory alliance and fidelity to the trauma narrative (TN; i.e., imaginal exposure) component of Trauma-Focused Cognitive Behavioral Therapy. We also examined how supervisory alliance moderated the effect of behavioral rehearsal use in supervision on TN fidelity. We analyzed data from a randomized controlled trial, in which forty-two supervisors and their clinicians (N = 124) from 28 Washington State community-based mental health offices participated. Clinicians were randomized to receive one of two supervision conditions-symptom and fidelity monitoring (SFM) or SFM with behavioral rehearsal (SFM + BR). Supervisory alliance alone did not predict delivery (i.e., occurrence) or extensiveness of delivery of the trauma narrative. Client-focused supervisory alliance moderated the effectiveness of behavioral rehearsal-as client-focused alliance increased, the odds of delivering the TN also increased significantly. Future research should further investigate how to appropriately match supervision techniques with supervisory dyads and explore the interplay of alliance with supervision techniques a supervisor might employ.
Collapse
Affiliation(s)
- Noah S Triplett
- Department of Psychology, University of Washington, Box 351525, 119A Guthrie Hall, Seattle, WA, 98195, USA.
| | - Rashed AlRasheed
- Department of Psychology, University of Washington, Box 351525, 119A Guthrie Hall, Seattle, WA, 98195, USA
| | - Clara Johnson
- Department of Psychology, University of Washington, Box 351525, 119A Guthrie Hall, Seattle, WA, 98195, USA
| | - Connor J McCabe
- Department of Psychiatry and Behavioral Medicine, University of Washington, Box 356560, 1959 NE Pacific Street, Seattle, WA, 98195, USA
| | - Michael D Pullmann
- UW School Mental Health Assessment, Research, and Training (SMART) Center, University of Washington, 6200 74th Street, Building 29, Seattle, WA, 98115, USA
| | - Shannon Dorsey
- Department of Psychology, University of Washington, Box 351525, 119A Guthrie Hall, Seattle, WA, 98195, USA
| |
Collapse
|
4
|
Kopelovich SL, Blank J, McCain C, Hughes M, Strachan E. Applying the Project ECHO Model to Support Implementation and Sustainment of Cognitive Behavioral Therapy for Psychosis. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2023; 44:00005141-990000000-00086. [PMID: 37389485 PMCID: PMC11107895 DOI: 10.1097/ceh.0000000000000511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/01/2023]
Abstract
INTRODUCTION Project Extension for Community Healthcare Outcomes (ECHO) is a teleconsultation model that leverages technology to sustain specialized interventions in underresourced settings. We present the application of the ECHO model to longitudinal training and consultation for community behavioral health providers learning to deliver cognitive behavioral therapy for psychosis, an evidence-based psychotherapy for individuals with psychotic disorders that has poorly penetrated the US mental health system. METHODS We analyzed within-group change over practitioners' 6-month ECHO participation cycle using the Expanded Outcomes Framework. We evaluated outcomes associated with participation, satisfaction, knowledge acquisition, performance, patient symptom severity, and functional impairment. RESULTS In the first 3 years, the cognitive behavioral therapy for psychosis ECHO Clinics supported 150 providers from 12 community agencies. Forty percent did not complete the 6-month ECHO calendar, most commonly due to separation from their agency. Participants reported high degrees of satisfaction. Declarative and procedural knowledge increased over the 6-month period. Of the 24 providers who received a fidelity review, 87.5% met or exceeded the competency benchmark within the 6-month period. Clinical outcomes reflected reductions in hallucinations, negative symptoms, depression, mania, and functional impairment, but no reductions were detected in delusions, disorganized speech, or abnormal psychomotor behavior. DISCUSSION ECHO Clinics offer a mode of providing continuous access to expert instruction, peer-to-peer consultation, and case-based learning that other workforce training models lack. Our evaluation suggests that the ECHO model supports continuous professional development for practitioners, most of whom had indicated inadequate preparation for their role. We observed improved learner and select patient outcomes.
Collapse
Affiliation(s)
- Sarah L. Kopelovich
- Department of Psychiatry and Behavioral Sciences University of Washington, Seattle, WA
| | - Jennifer Blank
- Department of Psychiatry and Behavioral Sciences University of Washington, Seattle, WA
| | - Chris McCain
- Department of Psychiatry and Behavioral Sciences University of Washington, Seattle, WA
| | - MacKenzie Hughes
- Department of Psychiatry and Behavioral Sciences University of Washington, Seattle, WA
| | - Eric Strachan
- Department of Psychiatry and Behavioral Sciences University of Washington, Seattle, WA
| |
Collapse
|
5
|
Patel ZS, Casline E, Jensen-Doss A. A thematic analysis of broker consultation in a TF-CBT community based learning collaborative. JOURNAL OF CHILD & ADOLESCENT TRAUMA 2023; 16:197-208. [PMID: 37234825 PMCID: PMC10205920 DOI: 10.1007/s40653-022-00472-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/04/2022] [Indexed: 05/28/2023]
Abstract
Background Consultation is an important implementation strategy to increase competence, knowledge, and fidelity to evidence-based practice in community settings. However, the literature has primarily focused on consultation for clinical providers, and less is known about consultation for "broker" professionals, or those who identify and refer children to mental health services. Given their critical role in connecting youth to evidence-based treatment, investigating broker knowledge and use of evidence-based screening and referral are needed. Objective To address this gap, the current study examines the content of consultation provided to broker professionals.
Collapse
Affiliation(s)
- Zabin S. Patel
- Department of Psychology Coral Gables, University of Miami, Miami, United States 33124 FL
| | - Elizabeth Casline
- Department of Psychology Coral Gables, University of Miami, Miami, United States 33124 FL
| | - Amanda Jensen-Doss
- Department of Psychology Coral Gables, University of Miami, Miami, United States 33124 FL
| |
Collapse
|
6
|
Marriott BR, Walker MR, Howard J, Puspitasari A, Scott K, Albright K, Lewis CC. Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle? ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2023; 50:366-378. [PMID: 36542316 PMCID: PMC10201798 DOI: 10.1007/s10488-022-01244-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 12/03/2022] [Indexed: 12/24/2022]
Abstract
PURPOSE Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address. METHODS The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session. RESULTS Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care. CONCLUSION Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.
Collapse
Affiliation(s)
- Brigid R Marriott
- Department of Psychiatry, Indiana University School of Medicine, 410 W. 10th St, 46202, Indianapolis, IN, USA.
| | - Madison R Walker
- University of North Carolina, 135 Dauer Dr, 27599, Chapel Hill, NC, USA
| | | | - Ajeng Puspitasari
- Rogers Behavioral Health, 576 Bielenberg Dr Ste 180, 55125, Woodbury, MN, USA
| | - Kelli Scott
- Brown University, 121 S. Main Street, 02903, Providence, RI, USA
| | - Karen Albright
- Division of General Internal Medicine, University of Colorado School of Medicine, 12631 East 17th Ave, 80045, Aurora, CO, USA
| | - Cara C Lewis
- MacColl Center for Healthcare Innovation, Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, 98101, Seattle, WA, USA
| |
Collapse
|
7
|
Torales J, Vilallba-Arias J, Bogado JA, O'Higgins M, Almirón-Santacruz J, Ruiz Díaz N, García O, Amarilla-Salvioni D, Castaldelli-Maia JM, Ventriglio A, Barrios I. Satisfaction with Telepsychiatry during the COVID-19 pandemic: Patients' and psychiatrists' report from a University Hospital. Int J Soc Psychiatry 2023; 69:156-160. [PMID: 34991382 PMCID: PMC9936176 DOI: 10.1177/00207640211070762] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
BACKGROUND The COVID-19 pandemic has led to important changes in the approach to patients worldwide. Different agencies have proposed and implemented telemedicine-based care services in order to ensure access to health care for all people. AIM The aim of this study was to determine the satisfaction of patients using the Telepsychiatry service offered by the Department of Psychiatry of the Hospital de Clínicas (National University of Asunción, Paraguay). METHODS A cross sectional and descriptive study has been conducted. Participants were recruited through a phone-based survey. Satisfaction with Telepsychiatry has been measured with an adapted version of a satisfaction survey in Teleneurology. As a complement, psychiatrists from the Hospital de Clínicas were also interviewed about their rate of satisfaction with Telepsychiatry. RESULTS A total of 530 patients were included, 51.3% of whom were women. The consultation satisfaction scale ranged between 2.15 and 4.30 with a mean score of 3.02 ± 0.32. Cronbach's alpha for the scale was .897, indicating a good internal consistency. Patients' satisfaction was higher for the perception of Telepsychiatry and lower for the doctor-patient relationship. Of the physicians, 87.5% were satisfied with the Telepsychiatry service. CONCLUSION The satisfaction overall score indicates patients' general satisfaction with the quality of care in Telepsychiatry, mainly regarding the perception of health care. and lower satisfaction with the doctor-patient relationship. Nine out of 10 psychiatrists felt satisfied with the Telepsychiatry service and considered that the degree of patient's satisfaction was acceptable during the Telepsychiatry consultation.
Collapse
Affiliation(s)
- Julio Torales
- Department of Medical Psychology, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay.,Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay
| | - Jorge Vilallba-Arias
- Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay.,Department of Psychiatry (Santa Rosa Campus), School of Medical Sciences, National University of Asunción, Santa Rosa del Aguaray, Paraguay
| | - José Andrés Bogado
- Department of Psychiatry (Santa Rosa Campus), School of Medical Sciences, National University of Asunción, Santa Rosa del Aguaray, Paraguay
| | - Marcelo O'Higgins
- Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay
| | - José Almirón-Santacruz
- Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay
| | - Noelia Ruiz Díaz
- Department of Medical Psychology, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay.,Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay
| | - Oscar García
- Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay
| | - Diego Amarilla-Salvioni
- Department of Psychiatry, School of Medical Sciences, National University of Asunción, San Lorenzo, Paraguay
| | - João Mauricio Castaldelli-Maia
- Department of Neuroscience, Fundação do ABC, Santo André, Brazil.,Department of Psychiatry, University of São Paulo, São Paulo, Brazil
| | - Antonio Ventriglio
- Department of Clinical and Experimental Medicine, University of Foggia, Foggia, Italy
| | - Iván Barrios
- Department of Statistics (Santa Rosa Campus), School of Medical Sciences, National University of Asunción, Santa Rosa del Aguaray, Paraguay
| |
Collapse
|
8
|
Metz A, Kainz K, Boaz A. Intervening for sustainable change: Tailoring strategies to align with values and principles of communities. FRONTIERS IN HEALTH SERVICES 2023; 2:959386. [PMID: 36925846 PMCID: PMC10012724 DOI: 10.3389/frhs.2022.959386] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 12/05/2022] [Indexed: 01/19/2023]
Abstract
This paper presents a rationale for tailoring implementation strategies within a values-driven implementation approach. Values-driven implementation seeks to organize implementers around clarifying statements of their shared values in ways that harmonize implementation dynamics related to individual and group mental models, relationships among implementers, and the implementation climate. The proposed approach to tailoring strategies is informed by systems theory and emphasizes the need to focus on both tangible events and behaviors, as well deeper patterns, structures, relationships, and mental models, in order to increase the likelihood of sustaining implementation efforts and improving outcomes for people and communities. We offer for consideration three specific sets of context determinants that are under-represented in the implementation literature and that emerge as especially relevant within a systems approach to identifying and successfully tailoring implementation strategies in the implementation setting including relationships, mental models, and implementation climate.
Collapse
Affiliation(s)
- Allison Metz
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Kirsten Kainz
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Annette Boaz
- Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, England
| |
Collapse
|
9
|
Metz A, Jensen T, Farley A, Boaz A, Bartley L, Villodas M. Building trusting relationships to support implementation: A proposed theoretical model. FRONTIERS IN HEALTH SERVICES 2022; 2:894599. [PMID: 36925800 PMCID: PMC10012819 DOI: 10.3389/frhs.2022.894599] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 08/08/2022] [Indexed: 11/13/2022]
Abstract
This paper presents a theory of change that articulates (a) proposed strategies for building trust among implementation stakeholders and (b) the theoretical linkages between trusting relationships and implementation outcomes. The theory of change describes how trusting relationships cultivate increases in motivation, capability, and opportunity for supporting implementation among implementation stakeholders, with implications for commitment and resilience for sustained implementation, and ultimately, positive implementation outcomes. Recommendations related to the measurement of key constructs in the theory of change are provided. The paper highlights how the development of a testable causal model on trusting relationships and implementation outcomes can provide a bridge between implementation research and implementation practice.
Collapse
Affiliation(s)
- Allison Metz
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Todd Jensen
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Amanda Farley
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Annette Boaz
- London School of Hygiene and Tropical Medicine, London, United Kingdom
| | - Leah Bartley
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Melissa Villodas
- School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| |
Collapse
|
10
|
Lyon AR, Liu FF, Connors EH, King KM, Coifman JI, Cook H, McRee E, Ludwig K, Law A, Dorsey S, McCauley E. How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care. Implement Sci Commun 2022; 3:79. [PMID: 35869500 PMCID: PMC9306246 DOI: 10.1186/s43058-022-00325-y] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Accepted: 07/10/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Initial training and ongoing post-training consultation (i.e., ongoing support following training, provided by an expert) are among the most common implementation strategies used to change clinician practice. However, extant research has not experimentally investigated the optimal dosages of consultation necessary to produce desired outcomes. Moreover, the degree to which training and consultation engage theoretical implementation mechanisms-such as provider knowledge, skills, and attitudes-is not well understood. This study examined the effects of a brief online training and varying dosages of post-training consultation (BOLT+PTC) on implementation mechanisms and outcomes for measurement-based care (MBC) practices delivered in the context of education sector mental health services. METHODS A national sample of 75 clinicians who provide mental health interventions to children and adolescents in schools were randomly assigned to BOLT+PTC or control (services as usual). Those in BOLT+PTC were further randomized to 2-, 4-, or 8-week consultation conditions. Self-reported MBC knowledge, skills, attitudes, and use (including standardized assessment, individualized assessment, and assessment-informed treatment modification) were collected for 32 weeks. Multilevel models were used to examine main effects of BOLT+PTC versus control on MBC use at the end of consultation and over time, as well as comparisons among PTC dosage conditions and theorized mechanisms (skills, attitudes, knowledge). RESULTS There was a significant linear effect of BOLT+PTC over time on standardized assessment use (b = .02, p < .01), and a significant quadratic effect of BOLT+PTC over time on individualized assessment use (b = .04, p < .001), but no significant effect on treatment modification. BOLT + any level of PTC resulted in higher MBC knowledge and larger growth in MBC skill over the intervention period as compared to control. PTC dosage levels were inconsistently predictive of outcomes, providing no clear evidence for added benefit of higher PTC dosage. CONCLUSIONS Online training and consultation in MBC had effects on standardized and individualized assessment use among clinicians as compared to services as usual with no consistent benefit detected for increased consultation dosage. Continued research investigating optimal dosages and mechanisms of these established implementation strategies is needed to ensure training and consultation resources are deployed efficiently to impact clinician practices. TRIAL REGISTRATION ClinicalTrials.gov NCT05041517 . Retrospectively registered on 10 September 2021.
Collapse
Affiliation(s)
- Aaron R. Lyon
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| | - Freda F. Liu
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| | - Elizabeth H. Connors
- grid.47100.320000000419368710Department of Psychiatry, Yale University, 389 Whitney Avenue, Office 106, New Haven, CT 06511 USA
| | - Kevin M. King
- grid.34477.330000000122986657Department of Psychology, University of Washington, Guthrie Hall, Box 351525, Seattle, WA 98195 USA
| | - Jessica I. Coifman
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| | - Heather Cook
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| | - Erin McRee
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| | - Kristy Ludwig
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| | - Amy Law
- grid.34477.330000000122986657Graduate Medical Education, University of Washington, Learning Gateway, Box 358220, Seattle, WA 98109 USA
| | - Shannon Dorsey
- grid.34477.330000000122986657Department of Psychology, University of Washington, Guthrie Hall, Box 351525, Seattle, WA 98195 USA
| | - Elizabeth McCauley
- grid.34477.330000000122986657Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA 98115 USA
| |
Collapse
|
11
|
Quetsch LB, Herschell AD, Kolko D, Liebsack BK, Carroll RA. Testing a community developed training protocol for an evidence-based treatment. EVALUATION AND PROGRAM PLANNING 2022; 92:102055. [PMID: 35227960 PMCID: PMC11443478 DOI: 10.1016/j.evalprogplan.2022.102055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Revised: 01/20/2022] [Accepted: 02/11/2022] [Indexed: 06/14/2023]
Abstract
The implementation of evidence-based treatments (EBTs) in community behavioral health settings is a recommended practice, yet training experienced by community-based clinicians may require novel and creative training methods. The current study focused on creating a training protocol for Alternative for Families: a Cognitive-Behavioral Therapy from both evidence-based foundations and community-based agency feedback to promote better EBT integration into agencies. Twenty-four clinicians from three agencies were trained using a community-informed training protocol. Outcomes for clinician-reports of organizational functioning, self-reports of skill and knowledge, and observational single-subject data of clinician skills were assessed. Minimal improvements were found for clinician skills across self-report and observations. More research on tailoring trainings to meet needs of community agencies and clinicians should be explored to determine best practice in wide-scale implementation efforts.
Collapse
Affiliation(s)
- Lauren B Quetsch
- University of Arkansas, Department of Psychological Science, USA
| | - Amy D Herschell
- Community Care Behavioral Health Organization, UPMC Insurance Services Division, USA; University of Pittsburgh School of Medicine, UPMC Western Psychiatric Hospital, USA.
| | - David Kolko
- University of Pittsburgh School of Medicine, UPMC Western Psychiatric Hospital, USA
| | | | - Regina A Carroll
- University of Nebraska Medical Center, Munroe-Meyer Institute, USA
| |
Collapse
|
12
|
Knowledge and attitudes of Implementation Support Practitioners-Findings from a systematic integrative review. PLoS One 2022; 17:e0267533. [PMID: 35544529 PMCID: PMC9094539 DOI: 10.1371/journal.pone.0267533] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Accepted: 04/10/2022] [Indexed: 01/08/2023] Open
Abstract
BACKGROUND It requires thoughtful planning and work to successfully apply and sustain research-supported interventions like healthcare treatments, social support, or preventive programs in practice. Implementation support practitioners (ISPs) such as facilitators, technical assistance providers, knowledge brokers, coaches or consultants may be involved to actively support the implementation process. This article presents knowledge and attitudes ISPs bring to their work. METHODS Building on a previously developed program logic, a systematic integrative review was conducted. Literature was sourced by searching nine electronic data bases, organizational websites, and by launching a call for publications among selected experts and social media. Article screening was performed independently by two researchers, and data from included studies were extracted by members of the research team and quality-assured by the lead researcher. The quality of included RCTs was assessed based on a framework by Hodder and colleagues. Thematic Analysis was used to capture information on knowledge and attitudes of ISPs across the included studies. Euler diagrams and heatmaps were used to present the results. RESULTS Results are based on 79 included studies. ISPs reportedly displayed knowledge about the clinical practice they work with, implementation / improvement practice, the local context, supporting change processes, and facilitating evidence-based practice in general. In particular, knowledge about the intervention to be implemented and its target population, specific improvement / implementation methods and approaches, organizational structures and sensitivities, training, and characteristics of (good) research was described in the literature. Seven themes describing ISPs' attitudes were identified: 1) professional, 2) motivated / motivating / encouraging / empowering, 3) empathetic / respectful / sensitive, 4) collaborative / inclusive, 5) authentic, 6) creative / flexible / innovative / adaptive, and 7) frank / direct / honest. Pertaining to a professional attitude, being responsive and focused were the most prevalent indicators across included publications. CONCLUSION The wide range and complexity of knowledge and attitudes found in the literature calls for a comprehensive and systematic approach to collaboratively develop a professional role for ISPs across disciplines. Embedding the ISP role in different health and social welfare settings will enhance implementation capacities considerably.
Collapse
|
13
|
Skar AMS, Braathu N, Peters N, Bækkelund H, Endsjø M, Babaii A, Borge RH, Wentzel-Larsen T, Ehrhart MG, Sklar M, Brown CH, Aarons GA, Egeland KM. A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate. BMC Health Serv Res 2022; 22:298. [PMID: 35246135 PMCID: PMC8895588 DOI: 10.1186/s12913-022-07539-9] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 01/24/2022] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate. METHODS A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate. RESULTS After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI. CONCLUSIONS The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation. TRIAL REGISTRATION Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .
Collapse
Affiliation(s)
- Ane-Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway.
| | - Nora Braathu
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Nadina Peters
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Harald Bækkelund
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Mathilde Endsjø
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Aida Babaii
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Randi Hovden Borge
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Tore Wentzel-Larsen
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway ,Centre for Child and Adolescent Mental Health, Eastern and Southern Norway, Gullhaugveien 1, 0484 Oslo, Norway
| | - Mark G. Ehrhart
- grid.170430.10000 0001 2159 2859Department of Psychology, University of Central Florida, 4111 Pictor Lane, Orlando, FL 32816-1390 USA
| | - Marisa Sklar
- grid.266100.30000 0001 2107 4242Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812 USA ,grid.267102.00000000104485736Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123 USA
| | - C. Hendricks Brown
- grid.16753.360000 0001 2299 3507Feinberg School of Medicine, Northwestern University, 750 North Lake Shore Drive, Chicago, IL 60611 USA
| | - Gregory A. Aarons
- grid.266100.30000 0001 2107 4242Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812 USA ,grid.267102.00000000104485736Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123 USA
| | - Karina M. Egeland
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| |
Collapse
|
14
|
Caron EB, Lind TA, Dozier M. Strategies that Promote Therapist Engagement in Active and Experiential Learning: Micro-Level Sequential Analysis. CLINICAL SUPERVISOR 2021; 40:112-133. [PMID: 34248258 DOI: 10.1080/07325223.2020.1870023] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Therapists' active learning increases treatment fidelity, but research is needed on supervisory strategies to engage therapists in active learning. This study used sequential analysis to examine consultant behaviors associated with increased and decreased probability of eliciting therapists' active learning. The study included 162 consultation sessions from 27 community therapists implementing Attachment and Biobehavioral Catch-up. Consultants' client discussion, information provision, and modeling were associated with reduced likelihood of active learning. Consultants' questions, engagement in active learning strategies, use of video, and silence were associated with greater likelihood of therapist active learning. These findings inform supervisors' attempts to encourage active learning.
Collapse
Affiliation(s)
- E B Caron
- Department of Psychological Science, Fitchburg State University
| | - Teresa A Lind
- Department of Child and Family Development, San Diego State University.,Child and Adolescent Research Center (CASRC)
| | - Mary Dozier
- Department of Psychological and Brain Sciences, University of Delaware
| |
Collapse
|
15
|
Trivasse H, Webb TL, Waller G. A meta-analysis of the effects of training clinicians in exposure therapy on knowledge, attitudes, intentions, and behavior. Clin Psychol Rev 2020; 80:101887. [DOI: 10.1016/j.cpr.2020.101887] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Revised: 05/24/2020] [Accepted: 06/30/2020] [Indexed: 10/23/2022]
|
16
|
Lochman JE, Boxmeyer CL, Kassing FL, Powell NP, Stromeyer SL. Cognitive Behavioral Intervention for Youth at Risk for Conduct Problems: Future Directions. JOURNAL OF CLINICAL CHILD AND ADOLESCENT PSYCHOLOGY : THE OFFICIAL JOURNAL FOR THE SOCIETY OF CLINICAL CHILD AND ADOLESCENT PSYCHOLOGY, AMERICAN PSYCHOLOGICAL ASSOCIATION, DIVISION 53 2019; 48:799-810. [PMID: 30892949 PMCID: PMC6710135 DOI: 10.1080/15374416.2019.1567349] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
This article briefly overviews the history of cognitive behavioral intervention (CBI) for children displaying early signs, or actual diagnoses, of conduct disorders. A series of randomized control trials have identified evidence-based CBI programs for children with these behavior problems at various developmental stages from preschool through adolescence. Although it is critically important for the field to disseminate these existing programs as developed, we argue that it is important to also move beyond the existing evidence-based programs. Research should continue to test new comprehensive, multicomponent interventions, fueled by our evolving understanding of active mechanisms that contribute to children's externalizing behavior problems. The future of research in this area can also benefit from a focus on four central issues. First, research can address how single interventions can have meaningful impact on a range of transdiagnostic outcomes because the intervention mechanisms may affect those various outcomes. Second, rooted in implementation science, we are beginning to understand better how evidence-based programs can be disseminated in the real world, examining key issues such as the adequacy of training approaches and the role of therapist and organizational characteristics. Third, a major focus of research can be on how to optimize intervention outcomes, including a focus on microtrials, on tailoring of interventions, on examining rigorously how interventions are delivered, and on the integration of technology and of other approaches such as mindfulness training into CBI. Fourth, research can explore how the therapeutic relationship and the therapists' characteristics can play substantial roles in effective CBI with conduct problem children.
Collapse
Affiliation(s)
- John E. Lochman
- The University of Alabama, 348 Gorddon Palmer Hall, Tuscaloosa, Alabama 35487
| | | | | | - Nicole P. Powell
- The University of Alabama, 348 Gorddon Palmer Hall, Tuscaloosa, Alabama 35487
| | - Sara L. Stromeyer
- The University of Alabama, 348 Gorddon Palmer Hall, Tuscaloosa, Alabama 35487
| |
Collapse
|
17
|
Hanson RF, Saunders BE, Ralston E, Moreland AD, Peer SO, Fitzgerald MM. Statewide implementation of child trauma-focused practices using the community-based learning collaborative model. Psychol Serv 2018; 16:170-181. [PMID: 30550316 DOI: 10.1037/ser0000319] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
A significant number of youth in the United States experience traumatic events that substantially increase the risk of physical and behavioral health problems across the life span. This public health concern warrants concerted efforts to promote trauma-informed, evidence-based practices that facilitate recovery. Although youth-focused trauma-specific treatments exist, determining effective ways to disseminate and implement these services-so that they are available, accessible, and sustainable-poses an ongoing challenge. This paper describes a comprehensive model for such implementation, the community-based learning collaborative (CBLC), developed as part of Project BEST, a four-phase statewide initiative to promote trauma-focused practices. The CBLC augments the learning collaborative model by including clinical and nonclinical (i.e., broker) professionals from multiple service organizations within a targeted community. CBLCs aim to build capacity for sustained implementation of trauma-focused practices by promoting interprofessional collaboration among those involved in the coordination and provision of these services. This paper describes the iterative development of the CBLC by examining participant completion data across the three completed phases of Project BEST (N = 13 CBLCs; 1,190 participants). Additionally, data from Project BEST's third phase (N = 6 CBLCs; 639 participants) were used to evaluate changes in the frequency of specific practices, pre- to post-CBLC, and post-CBLC perceived utility of CBLC components. High participant completion rates, significant increases in reported trauma-focused practices, and positive ratings of the CBLC's utility provisionally support the feasibility and efficacy of the model's final iteration. Implications for implementation and CBLC improvements are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Collapse
|
18
|
Choy-Brown M, Stanhope V. The Availability of Supervision in Routine Mental Health Care. CLINICAL SOCIAL WORK JOURNAL 2018; 46:271-280. [PMID: 30906079 PMCID: PMC6426317 DOI: 10.1007/s10615-018-0687-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Clinical supervision is an embedded resource for practice quality in community mental health organizations. Supervision has been found to increase provider competence and decrease stress. In addition, supervision has been associated with service user outcomes including decreased depressive symptoms. However, little is known about the availability and nature of supervision in real world settings. The primary aims of this study were to identify available supervision and the extent to which contextual factors are related to that availability. The data source for this study was a multi-state and multi-site (N=14) NIMH-funded trial survey of providers (N=273). Supervision was measured by hours per week (quantity) and by utilization of best practice activities (content). Univariate, chi-square, independent samples t-tests, and ANOVA analyses were used to assess supervision content and quantity and to examine subgroup differences. Participants reported an average of 2.17 hours of supervision per week and 28.6% of participants endorsed best practice content. Supervision quantity varied significantly across sites (p<.05) and program type (p<.05) while content did not. Individual role within the organization had a significant relationship with reported supervision content (p<.001). In these settings, organizations are exercising discretion in how to utilize supervision within the available time. Supervision time also varied by program type, increasing with the intensity of services. Findings demonstrate that reports of availability vary according to position within the organization and the intensity of services within a given program type. Implications for workforce development, access to quality services, and implementation of evidence-based practices are discussed.
Collapse
|
19
|
Lucid L, Meza R, Pullmann MD, Jungbluth N, Deblinger E, Dorsey S. Supervision in Community Mental Health: Understanding Intensity of EBT Focus. Behav Ther 2018; 49:481-493. [PMID: 29937252 PMCID: PMC6020167 DOI: 10.1016/j.beth.2017.12.007] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2017] [Revised: 12/13/2017] [Accepted: 12/13/2017] [Indexed: 11/29/2022]
Abstract
The goal of the present study was to examine clinician, supervisor, and organizational factors that are associated with the intensity of evidence-based treatment (EBT) focus in workplace-based clinical supervision of a specific EBT, Trauma-Focused Cognitive Behavioral Therapy (TF-CBT). Supervisors (n = 56) and clinicians (n = 207) from mental health organizations across Washington State completed online self-report questionnaires. Multilevel modeling (MLM) analyses were used to examine the relative influence of nested clinician and supervisor factors on the intensity of EBT focus in supervision. We found that 33% of the variance in clinician report of EBT supervision intensity clustered at the supervisor level and implementation climate was the only significant factor associated with EBT supervision intensity. While individual clinician and supervisor factors may play a role in EBT coverage in supervision, our results suggest that an implementation climate that supports EBT may be the most critical factor for improving intensity of EBT coverage. Thus, implementation efforts that address the extent to which EBTs are expected, rewarded, and supported within an organization may be needed to support greater coverage of EBT during workplace-based supervision.
Collapse
Affiliation(s)
| | - Rosemary Meza
- Department of Psychology, University of Washington, Guthrie Hall, Box 351525, Seattle 98195, USA
| | - Michael D. Pullmann
- Psychiatry and Behavioral Sciences, University of Washington School of Medicine, 2815 Eastlake Avenue East, Suite 200, Seattle, WA 98102, USA
| | - Nathaniel Jungbluth
- Department of Psychology, University of Washington, Guthrie Hall, Box 351525, Seattle 98195, USA
| | - Esther Deblinger
- School of Osteopathic Medicine, Rowan University, 42 E. Laurel Road, Suite 1100, Stratford, NJ 08084, USA
| | - Shannon Dorsey
- Department of Psychology, University of Washington, Guthrie Hall, Box 351525, Seattle 98195, USA
| |
Collapse
|
20
|
Barnett M, Brookman-Frazee L, Regan J, Saifan D, Stadnick N, Lau A. How Intervention and Implementation Characteristics Relate to Community Therapists' Attitudes Toward Evidence-Based Practices: A Mixed Methods Study. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2018; 44:824-837. [PMID: 28236076 DOI: 10.1007/s10488-017-0795-0] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
Therapists' perceptions toward evidence-based practices (EBPs) are important in implementation efforts, however little is known about characteristics of EBPs associated with more positive attitudes. This mixed-methods study examined how intervention and implementation characteristics of six EBPs related to therapist attitudes. Quantitative analysis of 793 cross-sectional surveys revealed that therapists endorsed more positive attitudes toward EBPs with (1) prescribed session content and order and (2) required consultation. Associations between these intervention and implementation characteristics and attitudes were not moderated by therapist experience or emotional exhaustion. Qualitative analyses complemented quantitative findings, indicating that "structure" was appealing for interventions and that therapists felt supported by consultation.
Collapse
Affiliation(s)
- Miya Barnett
- Department of Counseling, Clinical, & School Psychology, Gevirtz Graduate School of Education, University of California, Santa Barbara, Santa Barbara, CA, 93106-9490, USA.
| | - Lauren Brookman-Frazee
- Department of Psychiatry, University of California, San Diego, San Diego, CA, USA.,Child and Adolescents Service Research Center, San Diego, CA, USA
| | - Jennifer Regan
- Hathaway-Sycamores, Child and Family Services, Pasadena, CA, USA
| | - Dana Saifan
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Nicole Stadnick
- Department of Psychiatry, University of California, San Diego, San Diego, CA, USA.,Child and Adolescents Service Research Center, San Diego, CA, USA
| | - Anna Lau
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| |
Collapse
|
21
|
Barac R, Kimber M, Johnson S, Barwick M. The effectiveness of consultation for clinicians learning to deliver motivational interviewing with fidelity. ACTA ACUST UNITED AC 2018; 15:510-533. [PMID: 29883279 DOI: 10.1080/23761407.2018.1480988] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
Abstract
Despite the emerging literature documenting gains in clinician competence following consultation, little empirical work has examined consultation as an implementation strategy. To this end, the present study examined consultation in the context of implementing motivational interviewing in four community child and youth mental health organizations. We used qualitative methods with a dual goal: to describe the consultation process and to explore trainees' perspectives on consultation. Participants included 22 clinicians and 9 supervisors who received monthly, group, phone-based consultation for seven months following training in motivational interviewing. Analyses showed that consultation was perceived as effective because it helped to "keep motivational interviewing alive," fulfilled a profound learning function through collaboration and connection with others, and served as protected time for reflection on practice change. Our findings contribute to a body of knowledge about consultation elements that appear to be effective when implementing research-supported interventions in child and youth mental health.
Collapse
Affiliation(s)
- Raluca Barac
- a Child Health Evaluative Sciences, Research Institute , Hospital for Sick Children , Toronto , Canada.,b Department of Psychology , Memorial University of Newfoundland , St. John's , Canada
| | - Melissa Kimber
- c Department of Psychiatry and Behavioural Neurosciences , McMaster University , Hamilton , Canada
| | - Sabine Johnson
- d Department of Psychology , York University , Toronto , Canada
| | - Melanie Barwick
- a Child Health Evaluative Sciences, Research Institute , Hospital for Sick Children , Toronto , Canada.,e Psychiatry and Dalla Lana School of Public Health , University of Toronto , Toronto , Canada
| |
Collapse
|
22
|
Waltman SH, Hall BC, McFarr LM, Creed TA. Clinical Case Consultation and Experiential Learning in Cognitive Behavioral Therapy Implementation: Brief Qualitative Investigation. J Cogn Psychother 2018; 32:112-127. [PMID: 32746401 DOI: 10.1891/0889-8391.32.2.112] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
There has been an increased emphasis on the implementation of cognitive behavioral therapy (CBT) to community mental health (CMH) systems due to its broad efficacy. Previous research has highlighted the importance of ongoing consultation in this process. The current study clarifies the role and process of clinical case consultation in the implementation of CBT to CMH from the consultants' perspective. Trainers from two large-scale implementation initiatives (n = 27) were surveyed regarding their strategies used in the consultation process. Historically, researchers have focused on trainees who view relational variables as the most effective elements of consultation; however, in the current study, trainers perceived experiential learning as being the most effective consultation strategy for helping CMH clinicians learn CBT. Other aspects of consultation such as case conceptualization and practice sample review are discussed in terms of their relative perceived utility in raising CMH clinician's competency to treat comorbid patients.
Collapse
Affiliation(s)
- Scott H Waltman
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania .,Warrior Resiliency Program, Brooke Army Medical Center, San Antonio, Texas
| | | | | | - Torrey A Creed
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
23
|
Friedberg RD. Best practices in supervising cognitive behavioral therapy with youth. World J Clin Pediatr 2018; 7:1-8. [PMID: 29456927 PMCID: PMC5803561 DOI: 10.5409/wjcp.v7.i1.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/08/2017] [Revised: 12/16/2017] [Accepted: 01/07/2018] [Indexed: 02/06/2023] Open
Abstract
Clinical supervision of cognitive behavioral therapy (CBT) with youth ensures better patient care and fosters trainees' professional development. However, often insufficient attention is directed toward disseminating best practices in supervision of CBT with youth. This Therapeutic Advances contribution aims to communicate the core content of supervision. Additionally, the key supervisory practices associated with CBT with youth are described. Supervisory outcomes are summarized and recommendations for supervisory practices are made.
Collapse
Affiliation(s)
- Robert D Friedberg
- Center for the Study and Treatment of Anxious Youth, Palo Alto University, Palo Alto, CA 94304, United States
| |
Collapse
|
24
|
Gunderson LM, Willging CE, Trott Jaramillo EM, Green AE, Fettes DL, Hect DB, Aarons GA. The good coach: implementation and sustainment factors that affect coaching as evidence-based intervention fidelity support. JOURNAL OF CHILDREN'S SERVICES 2018; 13:1-17. [PMID: 30906421 PMCID: PMC6426451 DOI: 10.1108/jcs-09-2017-0043] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
PURPOSE Evidence-based interventions (EBIs) for human services unfold within complicated social and organizational circumstances and are influenced by the attitudes and behaviors of diverse stakeholders situated within these environments. Coaching is commonly regarded as an effective strategy to support service providers in delivering EBIs and attaining high levels of fidelity over time. The purpose of this paper is to address a lacuna in research examining the factors influencing coaching, an important EBI support component. METHODOLOGY The authors use the Exploration, Preparation, Implementation, and Sustainment framework to consider inner- and outer-context factors that affect coaching over time. This case study of coaching draws from a larger qualitative data set from three iterative investigations of implementation and sustainment of a home visitation program, SafeCare®. SafeCare is an EBI designed to reduce child neglect. FINDINGS The authors elaborate on six major categories of findings derived from an iterative data coding and analysis process: perceptions of "good" and "bad" coaches by system sustainment status; coach as peer; in-house coaching capacity; intervention developer requirements vs other outer-context needs; outer- context support; and inner-context support. PRACTICAL IMPLICATIONS Coaching is considered a key component for effective implementation of EBIs in public-sector systems, yet is under-studied. Understanding inner- and outer-context factors illuminates the ways they affect the capacity of coaches to support service delivery. ORIGINALITY This paper demonstrates that coaching can accomplish more than provision of EBI fidelity support. Stakeholders characterized coaches as operating as boundary spanners who link inner and outer contexts to enable EBI implementation and sustainment.
Collapse
Affiliation(s)
- Lara M Gunderson
- Behavioral Health Research Center of the Southwest, Pacific Institute for Research and Evaluation, Albuquerque, New Mexico, USA
| | - Cathleen E Willging
- Behavioral Health Research Center of the Southwest, Pacific Institute for Research and Evaluation, Albuquerque, New Mexico, USA
| | - Elise M Trott Jaramillo
- Behavioral Health Research Center of the Southwest, Pacific Institute for Research and Evaluation, Albuquerque, New Mexico, USA
| | - Amy E Green
- Child and Adolescent Services Research Center, University of California, San Diego, California, USA
| | - Danielle L Fettes
- Child and Adolescent Services Research Center, University of California, San Diego, California, USA
| | - Debra B Hect
- Developmental and Behavioral Pediatrics, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma, USA
| | - Gregory A Aarons
- Child and Adolescent Services Research Center, University of California, San Diego, California, USA
| |
Collapse
|
25
|
Dorsey S, Lyon AR, Pullmann MD, Jungbluth N, Berliner L, Beidas R. Behavioral Rehearsal for Analogue Fidelity: Feasibility in a State-Funded Children's Mental Health Initiative. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2017; 44:395-404. [PMID: 26966103 PMCID: PMC5734939 DOI: 10.1007/s10488-016-0727-4] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
A substantial number of evidence-based treatments (EBTs) are available, but are delivered infrequently in public mental health. To improve the quality of care, some states and systems have focused on EBT training; however, these efforts have rarely included objective measurement of clinician fidelity because of feasibility issues. The primary goal of the current study was evaluating the feasibility of the behavioral rehearsal (BR) method to assess "analogue fidelity" in a children's mental health quality improvement initiative. Results indicated low-but representative-clinician participation. Participants demonstrated greatest improvement at post-training with maintenance or decreases at 6-months (post-consultation). Implications for future use of BR are discussed.
Collapse
Affiliation(s)
- Shannon Dorsey
- Department of Psychology, University of Washington, 335 Guthrie Hall, Box 351525, Seattle, WA, 98195, USA.
| | - Aaron R Lyon
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Michael D Pullmann
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Nathaniel Jungbluth
- Department of Psychology, University of Washington, 335 Guthrie Hall, Box 351525, Seattle, WA, 98195, USA
| | - Lucy Berliner
- Department of Psychology, University of Washington, 335 Guthrie Hall, Box 351525, Seattle, WA, 98195, USA
- Harborview Center for Sexual Assault and Traumatic Stress, Seattle, USA
| | - Rinad Beidas
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA
| |
Collapse
|
26
|
Waltman S, Hall BC, McFarr LM, Beck AT, Creed TA. In-Session Stuck Points and Pitfalls of Community Clinicians Learning CBT: Qualitative Investigation. COGNITIVE AND BEHAVIORAL PRACTICE 2017. [DOI: 10.1016/j.cbpra.2016.04.002] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
27
|
Couturier J, Lock J, Kimber M, McVey G, Barwick M, Niccols A, Webb C, Findlay S, Woodford T. Themes arising in clinical consultation for therapists implementing family-based treatment for adolescents with anorexia nervosa: a qualitative study. J Eat Disord 2017; 5:28. [PMID: 28878927 PMCID: PMC5582386 DOI: 10.1186/s40337-017-0161-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/13/2017] [Accepted: 06/08/2017] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND Our study aims to explore and describe themes arising in sessions of clinical consultation with therapists implementing Family-Based Treatment (FBT) for adolescents with Anorexia Nervosa (AN). There is currently no literature describing the content of clinical consultation for FBT. Thus, this knowledge will add to the evidence-base on what therapists need from consultants in ongoing clinical consultation. METHODS Eight therapists at four sites participated in this study, which spanned a two-year period. Following a two-day training workshop, each therapist treated at least one adolescent patient presenting with a restrictive eating disorder with FBT, focusing on adherence to the treatment manual. Clinical consultation sessions occurred monthly and were led by an external FBT expert. Thirty-five (average per site = 9) audio recorded group clinical consultation sessions were transcribed verbatim and coded for themes. Twenty percent of the transcripts were double-coded to ensure consistency. Fundamental qualitative description guided the sampling and data collection. RESULTS Thematic content analysis revealed ten common themes relating to the provision of clinical consultation to therapists implementing FBT in clinical practice: encouraging parental meal time supervision,discussing the role of mothers, how to align parents, ensuring parental buy-in, when to transition to Phase 2, weighing the patient and the patients' knowledge of their weight, the role of siblings in FBT sessions, how best to manage patient co-morbidities, the role of the father in FBT and how best to manage the family meal. CONCLUSIONS In conclusion, clinical consultation themes aligned with many of the central tenets of FBT, including how to help parents align their supportive approach during the refeeding process, and how to help parents assume control of eating disordered behaviours. This knowledge helps to guide consultants to anticipate common issues brought forward by therapists attempting to implement FBT.
Collapse
Affiliation(s)
- J Couturier
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, 1200 Main St. West, Hamilton, ON L8N3Z5 Canada.,Department of Pediatrics, McMaster University, Hamilton, Canada.,Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada
| | - J Lock
- Department of Psychiatry & Neurosciences, Stanford University, Stanford, USA
| | - M Kimber
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, 1200 Main St. West, Hamilton, ON L8N3Z5 Canada.,Offord Centre, McMaster University, Hamilton, Canada
| | - G McVey
- Toronto General Hospital Research Institute, University Health Network, Toronto, Canada
| | - M Barwick
- Dalla Lana School of Public Health, University of Toronto, Toronto, Canada.,Department of Psychiatry, University of Toronto, Toronto, Canada.,Research Institute, The Hospital for Sick Children, Toronto, Canada
| | - A Niccols
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, 1200 Main St. West, Hamilton, ON L8N3Z5 Canada.,Offord Centre, McMaster University, Hamilton, Canada
| | - C Webb
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, 1200 Main St. West, Hamilton, ON L8N3Z5 Canada
| | - S Findlay
- Department of Pediatrics, McMaster University, Hamilton, Canada
| | - T Woodford
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, 1200 Main St. West, Hamilton, ON L8N3Z5 Canada
| |
Collapse
|
28
|
Stirman SW, Pontoski K, Creed T, Xhezo R, Evans AC, Beck AT, Crits-Christoph P. A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2017; 44:55-66. [PMID: 26577646 PMCID: PMC5293547 DOI: 10.1007/s10488-015-0700-7] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Despite the central role of training and consultation in the implementation of evidence-based psychological interventions (EBPIs), comprehensive reviews of research on training have highlighted serious gaps in knowledge regarding best practices. Consultation after initial didactic training appears to be of critical importance, but there has been very little research to determine optimal consultation format or interventions. This observational study compared two consultation formats that included review of session audio and feedback in the context of a program to train clinicians (n = 85) in community mental health clinics to deliver cognitive therapy (CT). A "gold standard" condition in which clinicians received individual feedback after expert consultants reviewed full sessions was compared to a group consultation format in which short segments of session audio were reviewed by a group of clinicians and an expert consultant. After adjusting for potential baseline differences between individuals in the two consultation conditions, few differences were found in terms of successful completion of the consultation phase or in terms of competence in CT at the end of consultation or after a 2 year follow-up. However, analyses did not support hypotheses regarding non-inferiority of the group consultation condition. While both groups largely maintained competence, clinicians in the group consultation condition demonstrated increases in competence over the follow-up period, while a sub-group of those in the individual condition experienced decreases. These findings, if replicated, have important implications for EBP implementation programs, as they suggest that observation and feedback is feasible in community mental health setting, and that employing this method in a group format is an effective and efficient consultation strategy that may enhance the implementation and sustainability of evidence-based psychotherapies.
Collapse
Affiliation(s)
- Shannon Wiltsey Stirman
- Dissemination and Training Division, National Center for PTSD and Department of Psychiatry and Behavioral Sciences, VA Palo Alto Healthcare System and Stanford University, 795 Willow Road (NC-PTSD), Menlo Park, CA, 94025, USA.
| | - Kristin Pontoski
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market St., Philadelphia, PA, 19104, USA
| | - Torrey Creed
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market St., Philadelphia, PA, 19104, USA
| | - Regina Xhezo
- Department of Behavioral Health and Intellectual DisAbility Services, 801 Market Street, 7th Floor, Philadelphia, PA, 19107, USA
| | - Arthur C Evans
- Department of Behavioral Health and Intellectual DisAbility Services, 801 Market Street, 7th Floor, Philadelphia, PA, 19107, USA
| | - Aaron T Beck
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market St., Philadelphia, PA, 19104, USA
| | - Paul Crits-Christoph
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market St., Philadelphia, PA, 19104, USA
| |
Collapse
|
29
|
Stirman SW, Gutner CA, Langdon K, Graham JR. Bridging the Gap Between Research and Practice in Mental Health Service Settings: An Overview of Developments in Implementation Theory and Research. Behav Ther 2016; 47:920-936. [PMID: 27993341 DOI: 10.1016/j.beth.2015.12.001] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/02/2015] [Revised: 12/09/2015] [Accepted: 12/12/2015] [Indexed: 10/22/2022]
Abstract
Twenty years after Sobell's (1996) influential call to integrate advances in behavior therapy into clinical settings, significant progress has been made in implementation science. In this narrative review, we provide an overview of implementation research findings and highlight recent findings that can inform efforts to bridge the gap between research and practice in mental health service settings. Key findings are summarized, organized according to levels of influence described in two implementation frameworks: The Exploration, Planning, Implementation, and Sustainment Framework (EPIS; Aarons et al., 2011) and the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). Important levels of influence to consider when implementing new treatments include the outer context, inner context, characteristics of the individual, and characteristics of the innovation. Research on strategies to prepare clinicians to deliver evidence-based psychosocial treatments (EBPTs) and to address contextual barriers to implementation at each level is described, with discussion of implications for the implementation of EBPTs and next steps for research.
Collapse
Affiliation(s)
| | - Cassidy A Gutner
- National Center for PTSD, VA Boston Healthcare System; Boston University
| | - Kirsten Langdon
- Department of Psychiatry, Rhode Island Hospital, Providence, RI
| | | |
Collapse
|
30
|
Leathers SJ, Spielfogel JE, Blakey J, Christian E, Atkins MS. The Effect of a Change Agent on Use of Evidence-Based Mental Health Practices. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2016; 43:768-782. [PMID: 26487393 PMCID: PMC4838563 DOI: 10.1007/s10488-015-0694-1] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Children's service systems are faced with a critical need to disseminate evidence-based mental health interventions. Despite the proliferation of comprehensive implementation models, little is known about the key active processes in effective implementation strategies. This proof of concept study focused on the effect of change agent interactions as conceptualized by Rogers' diffusion of innovation theory on providers' (N = 57) use of a behavioral intervention in a child welfare agency. An experimental design compared use for providers randomized to training as usual or training as usual supplemented by change agent interactions after the training. Results indicate that the enhanced condition increased use of the intervention, supporting the positive effect of change agent interactions on use of new practices. Change agent types of interaction may be a key active process in implementation strategies following training.
Collapse
Affiliation(s)
- Sonya J Leathers
- Jane Addams College of Social Work, University of Illinois at Chicago, 1040 W. Harrison St., Chicago, IL, 60607-7134, USA.
| | - Jill E Spielfogel
- Jane Addams College of Social Work, University of Illinois at Chicago, 1040 W. Harrison St., Chicago, IL, 60607-7134, USA
- School of Social Service Administration, University of Chicago, Chicago, IL, USA
| | - Joan Blakey
- Jane Addams College of Social Work, University of Illinois at Chicago, 1040 W. Harrison St., Chicago, IL, 60607-7134, USA
- Helen Bader School of Social Welfare, University of Wisconsin at Milwaukee, Milwaukee, WI, USA
| | - Errick Christian
- Jane Addams College of Social Work, University of Illinois at Chicago, 1040 W. Harrison St., Chicago, IL, 60607-7134, USA
- Department of Emergency Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Marc S Atkins
- Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA
| |
Collapse
|
31
|
Beidas RS, Stewart RE, Benjamin Wolk C, Adams DR, Marcus SC, Evans AC, Jackson K, Neimark G, Hurford MO, Erney J, Rubin R, Hadley TR, Barg FK, Mandell DS. Independent Contractors in Public Mental Health Clinics: Implications for Use of Evidence-Based Practices. Psychiatr Serv 2016; 67:710-7. [PMID: 26927579 PMCID: PMC4930412 DOI: 10.1176/appi.ps.201500234] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
OBJECTIVES Community mental health clinics are increasingly utilizing independent contractors to provide clinical services. At the same time, many organizations are participating in initiatives intended to increase implementation of evidence-based practices (EBPs). The primary aim of this study was to understand the associations of utilizing independent contractors with clinician knowledge and attitudes toward EBPs and organizational culture and climate. The study also sought to understand the potential impact of using independent contractors on mental health services delivery from the perspective of organizational leadership. METHODS Quantitative data were collected from 130 therapists in 23 organizations; qualitative data were collected from executive administrators in nine of the 16 organizations participating in EBP initiatives sponsored by the City of Philadelphia. Regression with random effects was used to estimate the associations between worker status (contractor or employee) and clinician attitudes toward EBPs, knowledge of EBPs, and organizational culture and climate. Qualitative inquiry was used to understand the impact of reliance on independent contractors on organizational participation in EBP initiatives. RESULTS Independent contractors endorsed less positive attitudes toward EBPs and scored lower on knowledge of EBPs. Interviews revealed four main themes: reasons for using independent contractors, general consequences of using independent contractors, specific impact of independent contractors on participation in EBP initiatives, and suggestions for alternatives. CONCLUSIONS A growing number of community mental health clinics rely on independent contractors. There may be consequences of this shift that deserve exploration.
Collapse
Affiliation(s)
- Rinad S Beidas
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Rebecca E Stewart
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Courtney Benjamin Wolk
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Danielle R Adams
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Steven C Marcus
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Arthur C Evans
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Kamilah Jackson
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Geoffrey Neimark
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Matthew O Hurford
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Joan Erney
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Ronnie Rubin
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Trevor R Hadley
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - Frances K Barg
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| | - David S Mandell
- Dr. Beidas, Dr. Stewart, Dr. Benjamin, Ms. Adams, Dr. Hadley, and Dr. Mandell are with the Department of Psychiatry and Dr. Barg is with the Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia (e-mail: ). Dr. Marcus is with the School of Social Policy and Practice, University of Pennsylvania, Philadelphia. Dr. Evans and Dr. Hurford are with the Philadelphia Department of Behavioral Health and Intellectual disAbility Services, Philadelphia. Dr. Jackson, Dr. Neimark, Ms. Erney, and Dr. Rubin are with Community Behavioral Health, Philadelphia
| |
Collapse
|
32
|
Waltman SH, Creed TA, Beck AT. Are the effects of cognitive behavior therapy for depression falling? Review and critique of the evidence. CLINICAL PSYCHOLOGY-SCIENCE AND PRACTICE 2016. [DOI: 10.1111/cpsp.12152] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
33
|
Kerns SEU, Pullmann MD, Negrete A, Uomoto JA, Berliner L, Shogren D, Silverman E, Putnam B. Development and Implementation of a Child Welfare Workforce Strategy to Build a Trauma-Informed System of Support for Foster Care. CHILD MALTREATMENT 2016; 21:135-146. [PMID: 26928411 DOI: 10.1177/1077559516633307] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Effective strategies that increase the extent to which child welfare professionals engage in trauma-informed case planning are needed. This study evaluated two approaches to increase trauma symptom identification and use of screening results to inform case planning. The first study evaluated the impact of training on trauma-informed screening tools for 44 child welfare professionals who screen all children upon placement into foster care. The second study evaluated a two-stage approach to training child welfare workers on case planning for children's mental health. Participants included (a) 71 newly hired child welfare professionals who received a 3-hr training and (b) 55 child welfare professionals who participated in a full-day training. Results from the first study indicate that training effectively increased knowledge and skills in administering screening tools, though there was variability in comfort with screening. In the second study, participants self-reported significant gains in their competency in identifying mental health needs (including traumatic stress) and linking children with evidence-based services. These findings provide preliminary evidence for the viability of this approach to increase the extent to which child welfare professionals are trauma informed, aware of symptoms, and able to link children and youth with effective services designed to meet their specific needs.
Collapse
Affiliation(s)
| | | | - Andrea Negrete
- Department of Psychology, University of Virginia, Charlottesville, VA, USA
| | | | - Lucy Berliner
- School of Medicine, University of Washington, Seattle, WA, USA Harborview Center for Sexual Assault and Traumatic Stress, Seattle, WA, USA
| | - Dae Shogren
- Department of Social and Health Services, Children's Administration, Olympia, WA, USA
| | - Ellen Silverman
- Access, Systems, and Coordination, Healthy Starts and Transitions, WA State Department of Health, Tumwater, WA, USA
| | - Barbara Putnam
- Department of Social and Health Services, Children's Administration, Olympia, WA, USA
| |
Collapse
|
34
|
Ringle VA, Read KL, Edmunds JM, Brodman DM, Kendall PC, Barg F, Beidas RS. Barriers to and Facilitators in the Implementation of Cognitive-Behavioral Therapy for Youth Anxiety in the Community. Psychiatr Serv 2015; 66:938-45. [PMID: 25975886 PMCID: PMC4869696 DOI: 10.1176/appi.ps.201400134] [Citation(s) in RCA: 54] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
OBJECTIVE The study examined, from the perspective of therapists, the barriers to and facilitators in implementing cognitive-behavioral therapy (CBT) for anxious youths in community settings. METHODS Fifty therapists (43% of the original training sample of 115 providers) participated in a follow-up interview two years after training and consultation. They reported on barriers to and facilitators in implementation of CBT for youths with anxiety. RESULTS Qualitative analyses identified numerous barriers and facilitators, including client factors (for example, motivated clients facilitated the use of CBT, whereas clients with complex issues and numerous psychosocial stressors hindered its use), intervention factors (the structure of CBT helped facilitate its use for some providers, whereas others reported feeling constrained by such structure), and organizational factors (for example, the absence of support within one's institution served as a barrier, whereas supervision supporting the use of CBT facilitated implementation). CONCLUSIONS Findings of this implementation trial align with conceptual implementation frameworks and may guide the tailoring of future implementation efforts in order to overcome barriers and maximize facilitators.
Collapse
Affiliation(s)
- Vanesa A Ringle
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Kendra L Read
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Julie M Edmunds
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Douglas M Brodman
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Philip C Kendall
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Frances Barg
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Rinad S Beidas
- Ms. Ringle is with the Department of Psychology, University of Miami, Coral Gables, Florida (e-mail: ). Ms. Read, Mr. Brodman, and Dr. Kendall are with the Department of Psychology, Temple University, Philadelphia. Dr. Edmunds is with Judge Baker Children's Center, Boston. Dr. Barg is with the Department of Family Medicine and Community Health and Dr. Beidas is with the Department of Psychiatry, both at the University of Pennsylvania Perelman School of Medicine, Philadelphia
| |
Collapse
|
35
|
|
36
|
Edmunds JM, Read KL, Ringle VA, Brodman DM, Kendall PC, Beidas RS. Sustaining clinician penetration, attitudes and knowledge in cognitive-behavioral therapy for youth anxiety. Implement Sci 2014; 9:89. [PMID: 25030651 PMCID: PMC4223397 DOI: 10.1186/s13012-014-0089-9] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2013] [Accepted: 06/27/2014] [Indexed: 12/03/2022] Open
Abstract
Background Questions remain regarding the sustainment of evidence-based practices following implementation. The present study examined the sustainment of community clinicians’ implementation (i.e., penetration) of cognitive-behavioral therapy, attitudes toward evidence-based practices, and knowledge of cognitive-behavioral therapy for youth anxiety two years following training and consultation in cognitive-behavioral therapy for youth anxiety. Methods Of the original 115 participants, 50 individuals (43%) participated in the two-year follow-up. A t- test examined sustainment in penetration over time. Hierarchical linear modeling examined sustainment in knowledge and attitudes over time. Time spent in consultation sessions was examined as a potential moderator of the change in knowledge and attitudes. Results Findings indicated sustained self-reported penetration of cognitive-behavioral therapy for anxious youth, with low fidelity to some key CBT components (i.e., exposure tasks). Follow-up knowledge was higher than at baseline but lower than it had been immediately following the consultation phase of the study. Belief in the utility of evidence-based practices was sustained. Willingness to implement an evidence-based practice if required to do so, appeal of evidence-based practices, and openness toward evidence-based practices were not sustained. Participation in consultation positively moderated changes in knowledge and some attitudes. Conclusions Sustainment varied depending on the outcome examined. Generally, greater participation in consultation predicted greater sustainment. Implications for future training include higher dosages of consultation.
Collapse
|
37
|
Nadeem E, Gleacher A, Beidas RS. Consultation as an implementation strategy for evidence-based practices across multiple contexts: unpacking the black box. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2014; 40:439-50. [PMID: 23716145 DOI: 10.1007/s10488-013-0502-8] [Citation(s) in RCA: 106] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
There is great interest in the dissemination and implementation of evidence-based treatments and practices for children across schools and community mental health settings. A growing body of literature suggests that the use of one-time workshops as a training tool is ineffective in influencing therapist behavior and patient outcomes and that ongoing expert consultation and coaching is critical to actual uptake and quality implementation. Yet, we have very limited understanding of how expert consultation fits into the larger implementation support system, or the most effective consultation strategies. This commentary reviews the literature on consultation in child mental health, and proposes a set of core consultation functions, processes, and outcomes that should be further studied in the implementation of evidence-based practices for children.
Collapse
Affiliation(s)
- Erum Nadeem
- Department of Child and Adolescent Psychiatry, New York University, One Park Avenue, 7th Floor, New York, NY, 10016, USA,
| | | | | |
Collapse
|
38
|
An examination of behavioral rehearsal during consultation as a predictor of training outcomes. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2014; 40:456-66. [PMID: 23616234 DOI: 10.1007/s10488-013-0490-8] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
The training literature suggests that ongoing support following initial therapist training enhances training outcomes, yet little is known about what occurs during ongoing support and what accounts for its effectiveness. The present study examined consultation sessions provided to 99 clinicians following training in cognitive-behavioral therapy for youth anxiety. Recorded consultation sessions (N = 104) were coded for content and consultative methods. It was hypothesized that behavioral rehearsal (an active learning technique) would predict therapist adherence, skill, self-efficacy, and satisfaction at post-consultation. Regression analyses found no significant relation, however, clinician involvement during consultation sessions positively moderated the relationship between behavioral rehearsals and skill. Implications, limitations, and future directions are discussed.
Collapse
|
39
|
Beidas RS, Cross W, Dorsey S. Show Me, Don't Tell Me: Behavioral Rehearsal as a Training and Analogue Fidelity Tool. COGNITIVE AND BEHAVIORAL PRACTICE 2014; 21:1-11. [PMID: 25382963 DOI: 10.1016/j.cbpra.2013.04.002] [Citation(s) in RCA: 99] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Behavioral rehearsal, when a trainee engages in a simulated interaction with another individual, is an underutilized but potentially cost-effective and feasible solution for two difficult questions in implementation science: how to improve training, a commonly used implementation strategy, and how to feasibly measure fidelity using analogue methods in community settings. This paper provides practical information on how to develop and use behavioral rehearsal for both of these purposes to implementation researchers. Therefore, we focus on development and use of behavioral rehearsal as a training and analogue fidelity tool in the context of three illustrative studies.
Collapse
|
40
|
Hoagwood K, Atkins M, Ialongo N. Unpacking the black box of implementation: the next generation for policy, research and practice. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2013; 40:451-5. [PMID: 23942647 PMCID: PMC3824224 DOI: 10.1007/s10488-013-0512-6] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
Affiliation(s)
- Kimberly Hoagwood
- Department of Child and Adolescent Psychiatry, New York University Child Study Center, New York University School of Medicine, One Park Avenue at East 33rd, 8th Floor, New York, NY, 10016, USA,
| | | | | |
Collapse
|