Brief Article
Copyright ©2014 Baishideng Publishing Group Co., Limited. All rights reserved.
World J Psychiatr. Mar 22, 2014; 4(1): 13-29
Published online Mar 22, 2014. doi: 10.5498/wjp.v4.i1.13
Assimilation approach to measuring organizational change from pre- to post-intervention
Scott C Moore, Katerine Osatuke, Steven R Howe
Scott C Moore, Katerine Osatuke, United States Veterans Health Administration, National Center for Organization Development, Cincinnati, OH 45249, United States
Steven R Howe, Department of Psychology, McMicken College of Arts Sciences/University of Cincinnati, Cincinnati, OH 45221-0037, United States
Author contributions: Moore SC collected the data; Osatuke K performed the data analyses; Moore SC and Osatuke K designed the study, interpreted the results of the analyses and wrote the manuscript; Howe SR coordinated the data collection and was involved in editing the manuscript for important structural aspects of presenting the intellectual content; Osatuke K, Moore SC and Howe SR were involved in revising the manuscript and approving its final version.
Correspondence to: Katerine Osatuke, PhD, United States Veterans Health Administration, National Center for Organization Development, 11500 Northlake Drive Ste 260, Cincinnati, OH 45249, United States.
Telephone: +1-513-2472255 Fax: +1-513-2474699
Received: November 19, 2013
Revised: January 15, 2014
Accepted: March 3, 2014
Published online: March 22, 2014

AIM: To present a conceptual and measurement strategy that allows to objectively, sensitively evaluate intervention progress based on data of participants’ perceptions of presenting problems.

METHODS: We used as an example an organization development intervention at a United States Veterans Affairs medical center. Within a year, the intervention addressed the hospital’s initially serious problems and multiple stakeholders (employees, management, union representatives) reported satisfaction with progress made. Traditional quantitative outcome measures, however, failed to capture the strong positive impact consistently reported by several types of stakeholders in qualitative interviews. To address the paradox, full interview data describing the medical center pre- and post- intervention were examined applying a validated theoretical framework from another discipline: Psychotherapy research. The Assimilation model is a clinical-developmental theory that describes empirically grounded change levels in problematic experiences, e.g., problems reported by participants. The model, measure Assimilation of Problematic Experiences Scale (APES), and rating procedure have been previously applied across various populations and problem types, mainly in clinical but also in non-clinical settings. We applied the APES to the transcribed qualitative data of intervention participants’ interviews, using the method closely replicating prior assimilation research (the process whereby trained clinicians familiar with the Assimilation model work with full, transcribed interview data to assign the APES ratings). The APES ratings summarized levels of progress which was defined as participants’ assimilation level of problematic experiences, and compared from pre- to post-intervention.

RESULTS: The results were consistent with participants’ own reported perceptions of the intervention impact. Increase in APES levels from pre- to post-intervention suggested improvement, missed in the previous quantitative measures (the Maslach Burnout Inventory and the Work Environment Scale). The progress specifically consisted of participants’ moving from the APES stages where the problematic experience was avoided, to the APES stages where awareness and attention to the problems were steadily sustained, although the problems were not yet fully processed or resolved. These results explain why the conventional outcome measures failed to reflect the intervention progress; they narrowly defined progress as resolution of the presenting problems and alleviation of symptomatic distress. In the Assimilation model, this definition only applies to a sub-segment of the change continuum, specifically the latest APES stages. The model defines progress as change in psychological processes used in response to the problem, i.e., a growing ability to deal with problematic issues non-defensively, manifested differently depending on APES stages. At early stages, progress is an increased ability to face the problem rather than turning away. At later APES stages, progress involves naming, understanding and successfully addressing the problem. The assimilation approach provides a broader developmental context compared to exclusively symptom, problem-, or behavior- focused approaches that typically inform outcome measurement in interpersonally based interventions. In our data, this made the difference between reflecting (APES) vs missing (Maslach Burnout Inventory, Work Environment Scale) the pre-post change that was strongly perceived by the intervention recipients.

CONCLUSION: The results illustrated a working solution to the challenge of objectively evaluating progress in subjectively experienced problems. This approach informs measuring change in psychologically based interventions.

Keywords: Assimilation model, Change process, Outcome evaluation, Qualitative analysis

Core tip: Measuring organizational change in real-world settings presents the challenge of objectively evaluating progress in problems subjectively experienced by participants. We illustrate a working solution, discussing an organizational intervention at a United States Veterans Affairs medical center. Within a year, the intervention addressed the hospital’s initially serious problems, to multiple stakeholders’ satisfaction. Traditional quantitative outcome measures, however, failed to capture the reported strong positive impact. We applied the Assimilation model, a clinical-developmental theory from psychotherapy research, that measures change levels in participants’ problematic experiences. The model and measure sensitively captured the progress described by the participants, previously missed by conventional outcome measures.