Assimilation approach to measuring organizational change from pre- to post-intervention

Abstract

AIM: To present a conceptual and measurement strategy that allows to objectively, sensitively evaluate intervention progress based on data of participants’ perceptions of presenting problems.

METHODS: We used as an example an organization development intervention at a United States Veterans Affairs medical center. Within a year, the intervention addressed the hospital’s initially serious problems and multiple stakeholders (employees, management, union representatives) reported satisfaction with progress made. Traditional quantitative outcome measures, however, failed to capture the strong positive impact consistently reported by several types of stakeholders in qualitative interviews. To address the paradox, full interview data describing the medical center pre- and post- intervention were examined applying a validated theoretical framework from another discipline: Psychotherapy research. The Assimilation model is a clinical-developmental theory that describes empirically grounded change levels in problematic experiences, e.g., problems reported by participants. The model, measure Assimilation of Problematic Experiences Scale (APES), and rating procedure have been previously applied across various populations and problem types, mainly in clinical but also in non-clinical settings. We applied the APES to the transcribed qualitative data of intervention participants’ interviews, using the method closely replicating prior assimilation research (the process whereby trained clinicians familiar with the Assimilation model work with full, transcribed interview data to assign the APES ratings). The APES ratings summarized levels of progress which was defined as participants’ assimilation level of problematic experiences, and compared from pre- to post-intervention.

RESULTS: The results were consistent with participants’ own reported perceptions of the intervention impact. Increase in APES levels from pre- to post-intervention suggested improvement, missed in the previous quantitative measures (the Maslach Burnout Inventory and the Work Environment Scale). The progress specifically consisted of participants’ moving from the APES stages where the problematic experience was avoided, to the APES stages where awareness and attention to the problems were steadily sustained, although the problems were not yet fully processed or resolved. These results explain why the conventional outcome measures failed to reflect the intervention progress; they narrowly defined progress as resolution of the presenting problems and alleviation of symptomatic distress. In the Assimilation model, this definition only applies to a sub-segment of the change continuum, specifically the latest APES stages. The model defines progress as change in psychological processes used in response to the problem, i.e., a growing ability to deal with problematic issues non-defensively, manifested differently depending on APES stages. At early stages, progress is an increased ability to face the problem rather than turning away. At later APES stages, progress involves naming, understanding and successfully addressing the problem. The assimilation approach provides a broader developmental context compared to exclusively symptom, problem-, or behavior- focused approaches that typically inform outcome measurement in interpersonally based interventions. In our data, this made the difference between reflecting (APES) vs missing (Maslach Burnout Inventory, Work Environment Scale) the pre-post change that was strongly perceived by the intervention recipients.

CONCLUSION: The results illustrated a working solution to the challenge of objectively evaluating progress in subjectively experienced problems. This approach informs measuring change in psychologically based interventions.

Key Words: Assimilation model, Change process, Outcome evaluation, Qualitative analysis

Core tip: Measuring organizational change in real-world settings presents the challenge of objectively evaluating progress in problems subjectively experienced by participants. We illustrate a working solution, discussing an organizational intervention at a United States Veterans Affairs medical center. Within a year, the intervention addressed the hospital’s initially serious problems, to multiple stakeholders’ satisfaction. Traditional quantitative outcome measures, however, failed to capture the reported strong positive impact. We applied the Assimilation model, a clinical-developmental theory from psychotherapy research, that measures change levels in participants’ problematic experiences. The model and measure sensitively captured the progress described by the participants, previously missed by conventional outcome measures.

INTRODUCTION

The impetus for this study came from a challenge frequently encountered by organizational leaders and consultants worldwide, namely, how to determine in an unbiased way whether an intervention within an organizational setting successfully addressed the stakeholders’ presenting concerns. The reason that this question is difficult to answer is that it forces the questioner to face what has been described as the “double heritage”[1] or a “precarious epistemological tightrope”[2] of our discipline: the fact that both the scientific and humanistic traditions coexist within psychology, serving as sources of theories, concepts and methods for assessment, interventions and research. Like other professionals in psychology, organization development (OD) psychologists explain behavior of individuals and groups using the language and research tools created within science. However, also like other psychologists, they conduct interventions based on using the same psychological processes and mechanisms within individuals and groups as those processes and mechanisms that are made use of in religious, spiritual, and culturally indigenous interventions. This creates a threat of a “methodological war”[3] between the scientific subject matter and the applied methods of psychology, reflecting a potential conflict between the two paradigms (scientific and humanistic). The current authors propose a solution to the problem that this conflict creates, by offering a conceptual framework (the Assimilation model) and a measure [Assimilation of Problematic Experiences Scale (APES) scale] that allow accounting for the intervention participants’ psychological processes of meaning-making while also measuring the participants’ perceptions of change in a systematic and unbiased way.

In this study, we explain and illustrate how applying the Assimilation model and its associated measure helped clarify the outcomes from a specific intervention. This OD intervention was conducted within a medical center that was part of the Veterans Health Administration (VHA), the largest healthcare network within the United States that serves over 5 million of war Veterans and their family members. The data are presented in specific detail to substantiate the conceptual difference between the presented theory and other models, show the benefits of the proposed approach in evaluating the intervention outcomes, and illustrate how exactly the measure applies to the data. This level of specificity allows generalizing all the components of the proposed approach and replicating them in other organizational settings and within or outside of the United States.

In early 2002, a Veterans Affairs (VA) Medical Center (VAMC) in the Eastern United States was experiencing a multitude of organizational problems. The name of the organization is not indicated because of confidentiality reasons. The VAMC recently lost two of four Equal Employment Opportunity (EEO) suits, had bitter conflict between unions and management, and poor ratings on performance measures. These issues drew negative attention from the state government, media, and the National Association for the Advancement of Colored People (NAACP). The director of the regional group of VA facilities sought to remove the VAMC director, as did others both inside and outside the VHA (the branch of Veterans Affairs managing the nationwide hospital system). To manage the situation, an internal VHA oversight committee suggested calling the newly formed VHA National Center for OD (NCOD) to intervene at the hospital. NCOD intervention theory and approach to practice are described elsewhere[4,5].

A team of experienced OD consultants delivered an intervention. A year from its beginning, multiple stakeholders (employees, union representatives, the hospital management team, higher regional and national administrators, and the consultants themselves) reported clear positive changes in the organizational climate. Nevertheless, two quantitative assessment measures, the Maslach Burnout Inventory[6] and the Work Environment Scale[7], captured only a modest amount of post-intervention change. Given that the content of these measures generally matched the interviewed participants’ descriptions of both pre-intervention problems and post-intervention improvement areas, the contrast between a lack of change on standard measures and the stakeholders’ perceptions of positive change was puzzling. This study attempted to resolve the puzzle by examining verbatim interview records describing multiple participants’ first-hand experience of the VAMC workplace, organizational problems, intervention, and change. We used an empirically based, theory-informed method to make conceptual sense of these perceptions by evaluating the consistency of observation-based conclusions with the framework of the model.

Extensive quantitative and preliminary qualitative analyses of the intervention results, reported elsewhere[8], documented the inconsistency between quantitative and qualitative assessments. All quantitative measures of the working environment pre-intervention fell within the publisher’s established limits, suggesting the working environment was normal. Since the stakeholders certainly did not perceive the situation at the VAMC as normal, these findings were not useful for intervention planning. Qualitative data (interviews, focus groups) therefore became the main sources of information. Moreover, extensive statistical analyses[8] found little quantitative change over time, although multiple types of stakeholders consistently reported positive changes[8] and attributed these to the intervention. Within the real world of OD consulting, reports from satisfied organizational clients who directly experienced the intervention are usually taken as a proof of intervention success, not discounted when they are inconsistent with quantitative assessment. However, a data-driven system such as the VHA warrants a more specific measure of change than the stakeholders’ affirmations. This study undertook an in-depth secondary analysis of Moore’s[8] verbatim transcripts of semi-structured interviews with the intervention participants, applying to these data a more objective and theoretically informed perspective than the stakeholders’ own. We used the conceptual framework and empirically grounded method of the Assimilation model to isolate the participants’ perceptions of what was initially problematic and what changed during the intervention.

The Assimilation model originated from studying client change in counseling and psychotherapy[9]. It describes people’s representations of what they experience as problems and tracks change in these perceptions over time. Whereas other models of individual change (e.g., Prochaska et al[10] change model) outline stages in clients’ stance towards strictly behaviorally defined problems such as smoking, the Assimilation model applies to any (broad or narrow) problem definitions. The data of the individual’s raw experience serve for isolating what specifically feels problematic to the person at a given time. The method (see Procedure) involves clinically trained raters following standard steps to identify problematic perceptions from participants’ descriptions of particular content. This model facilitated our focus on understanding stakeholders’ personal experience of organizational problems without constricting the examined issues to well defined (i.e., cognitively processed) problems only.

In this theoretical framework, higher assimilation levels underlie better coping; one cannot successfully cope with denied or avoided content. Higher assimilation levels related to better outcomes in prior assimilation research[11], and in our study better coping is conceptualized as higher assimilation levels for specific problems. Applying the analytical method used in assimilation research, we identified the presenting problems and rated their assimilation levels in data from multiple types of participants. We then examined evidence for higher coping levels post- vs pre-intervention by evaluating pre- to post- change in ratings assigned by an expert coder to specific problems.

We explain the concepts and methods of the Assimilation model, discuss their relevance to this study of organizational change, and analyze the initial problems at the medical center, the intervention, and its results using the assimilation framework. This framework clarified the intervention effects by placing the problematic workplace experiences and problem-solving strategies in the context of a conceptually meaningful, empirically grounded developmental sequence detailing psychological stages of coping with subjectively problematic experiences.

Assimilation model and measure

Continuing a long tradition in personality, clinical psychology, and semiotics[12,13], the Assimilation model explains people as psychologically composed of traces of life experiences - e.g., thoughts, memories, affective reactions, somatic perceptions, motives, and other elements of lived experience, not limited to its cognitive representations. The experiential traces are called voices within the person. They contain motivational elements, triggered or “wanting” to be expressed (to say or do something) when the circumstances resemble the original experience. This tenet distinguishes the Assimilation model from models that describe people as using cognitive process to appraise experiences and using coping to manage them, an approach typical for most theories of workplace stress. Examples include Lazarus’s[14] transactional model, Hobfall et al[15] conservation of resources, Siegrist’s[16] effort-reward imbalance, Karasek[17] and Theorell’s demands-control, and Meurs et al[18] cognitive activation. In the assimilation model, there is no central processor because there is no person separate from experiences. Voices, multiple organizing centers of experience, are self-states from which a person thinks, acts, and speaks; e.g., the appraising capacity is also a voice, not a function separate from the contents which it evaluates. The model’s horizontal rather than vertical approach (dissecting participants’ data by experiential contents, such as “my experience of discrimination”, not by psychological functions, e.g., cognitive, affective, behavioral, or social-interpersonal aspects of handling the experience) offers researchers the methodological advantage of tracking how specific contents of participant-reported problems evolve throughout the change process. Some shifts in clients’ needs and perceptions of what constitutes the problem do occur during OD interventions[19-21]. The assimilation measure tracks change in initial perceptions, making no assumption of invariance in problem definitions typical of other change models.

The Assimilation model informed our study because it offered both the concepts and measurement tools describing participants’ personal experience of problematic issues at the healthcare workplace. Traces are left by all kinds of experiences, including those shared by people[13]; it is possible to recognize a voice of a group, society, or organization expressed through shared experiences of its members. Organizational interventions, at least those practiced by NCOD[5], facilitate client processes throughout organizational change efforts, which usually evoke some shared experiences (e.g., loss, risk, resistance, hope). In this sense, NCOD’s supportive interventions resemble therapy or counseling, but the client is the organization, not a person. Applying the individual change model helped us track evolution in participants’ perceptions of their medical center and the study shared some aspects of problematic experiences and changed therein across participants.

According to the model, the traces of experiences are integrated (assimilated) within people to various extents. Congruent voices are interlinked. The experiences dissonant from the bulk of the already assimilated ones become subjectively problematic (i.e., bring distress) because of their inconsistency, handled by warding off problematic contents. Avoidance brings short-term benefits (e.g., discomfort avoidance), but also long-term costs (e.g., ignoring a part of reality limits adaptive coping). Distress of facing problematic experiences motivates people to seek help, including interventions, psychological or organizational. In successful interventions, the clients gradually assimilate the initially problematic experiences by learning better strategies of addressing them. Assimilation progress occurs through a regular, empirically grounded developmental sequence of levels summarized in the APES[22]: 0 (warded off/dissociated), 1 (unwanted thoughts/active avoidance), 2 (vague awareness/emergence), 3 (problem statement/clarification), 4 (understanding/insight), 5 (application/working through), 6 (resourcefulness/problem solution), 7 (integration/mastery). Table 1 describes cognitive, affective, and behavioral features of each level. The APES sequence is conceptualized as universal (independent of problematic content, type of client, or intervention approach), developmental (stages build upon each other), and continuous (including substages: e.g., substages of levels 1[23] and 3[24] have been described). Stiles and colleagues[9,12,13,25,26] discussed the model in more detail. Each problem and change therein are tracked separately (not as individual or organizational composites). The organizational change event is thus examined by tracking progress on specific problems, one at a time, through the eyes of each participant.

Table 1 Assimilation of problematic experiences scale.

0	Warded off/dissociated. Client seems unaware of the problem; the problematic voice is silent or dissociated. Affect may be minimal, reflecting successful avoidance. Alternatively, the problem appears as somatic symptoms, acting out, or state switches
1	Unwanted thoughts/active avoidance. Client prefers not to think about the experience. Problematic voices emerge in response to therapist interventions or external circumstances and are suppressed or actively avoided. Affect involves unfocused negative feelings; their connection with the content may be unclear
2	Vague awareness/emergence. Client is aware of the problem but cannot formulate it clearly-can express it but cannot reflect on it. Problematic voice emerges into sustained awareness. Affect includes intense psychological pain-fear, sadness, anger, disgust-associated with the problematic experience
3	Problem statement/clarification. Content includes a clear statement of a problem-something that can be worked on. Opposing voices are differentiated and can talk about each other. Affect is negative but manageable, not panicky
4	Understanding/insight. The problematic experience is formulated and understood in some way. Voices reach an understanding with each other (a meaning bridge). Affect may be mixed, with some unpleasant recognition but also some pleasant surprise
5	Application/working through. The understanding is used to work on a problem. Voices work together to address problems of living. Affective tone is positive, optimistic
6	Resourcefulness/problem solution. The formerly problematic experience has become a resource, used for solving problems. Voices can be used flexibly. Affect is positive, satisfied
7	Integration/mastery. Client automatically generalizes solutions; voices are fully integrated, serving as resources in new situations. Affect is positive or neutral (i.e., this is no longer something to get excited about)

Note 1: Assimilation is a continuum, with intermediate levels allowed; e.g., 2.5 assimilation level is half way between vague awareness/emergence (2.0) and problem statement/clarification (3.0). Note 2: This table, authored by Stiles, appears in several of his papers. The first version was printed in Stiles et al[22] (1991).

The clients’ initial stance toward presenting problems may correspond to any level but a key theoretical concept is that any movement up the APES sequence represents increases in assimilation, which reflects progress. At intermediate APES levels, the client is clear on what constitutes the problem, whereas the problems at low APES levels are poorly formulated. For such problems, intervention approaches that emphasize exploration fit best[27,28]. This conclusion is relevant because at the VAMC, presenting problems pre-intervention had overall low APES ratings, the intervention focus was exploratory, and the clients perceived this focus as particularly helpful.

Assimilation analysis, an iterative qualitative procedure of identifying problematic experiences and rating their assimilation levels, allows tracking how the problematic experiences change from being feared or unwanted in early stages, to being understood and mastered by the end of successful interventions (cf Piaget et al[29] concepts of schemas, assimilation, and accommodation). Previous APES applications used mainly the data of individual change through counseling, although non-clinical applications exist[30,31]. We used the model and the APES to describe and compare the VAMC participants’ experience of workplace problems from pre- to post-intervention.

Organizational client background and presenting problems

Among the many problems at the VAMC, the EEO settlements were the most troubling with the highest profile of any issues. They attracted the attention of both senators of the state and the NAACP. In one EEO settlement case, a White employee, supervisor of a crew of primarily Black veterans in the carpentry shop, experienced a hostile environment from peers and supervisors for associating with Black employees. The complainant was awarded $192400, which the facility (medical center) chose not to appeal. In another case, a deaf painter was discriminated against because of his disability. His peers in the carpentry shop would sneak up on him and startle him or falsely inform him the telephone was ringing. Years earlier, this complainant had won an EEO suit dealing with promotion and was awarded $135000 for the second suit. The facility is in an economically depressed area of the Eastern United States region with around 16000 residents. VHA administrators, both at the VAMC and in Washington D.C., feared the large settlements had the potential to start a domino effect if employees came to view filing suits as a means for economic gain (S.R. Dyrenforth, NCOD director, personal communication, July 18, 2005).

Through a combination of media exposure, political involvement, and VHA internal communication, the turmoil surrounding the VAMC was brought to the attention of Laura Miller, then Associate Deputy Undersecretary for Health, who viewed the situation as a serious issue to be addressed immediately. In March 2002, an investigative team of VHA administrators and EEO experts from the VHA central office in Washington D.C. visited the VAMC to explore the racial climate through interviews with staff and human resource personnel and to scrutinize the VAMC management’s EEO complaint handling and promotion, and hiring practices. Their suggestion was not to remove the director of the hospital, a common practice for dealing with issues of similar magnitude, but to invite the NCOD to address the VAMC issues. The amount of media exposure and scrutiny from higher VHA management created much stress at the facility and fueled employees’ concerns for their current and future workplace environment. The intervention goals therefore included “easing the minds” of employees with respect to these concerns (S. R. Dyrenforth, personal communication, July 18, 2005).

Intervention approach

The NCOD intervention model and its application to the VAMC are characterized in detail elsewhere[4]. Below, we summarize the points pertinent to the current study.

Model: The NCOD intervention model is based upon action science[32,33] and process consultation[34,35]. Action science aims at increasing professional effectiveness by helping individuals in small groups shift from defensive, competitive strategies to collaborative strategies for resolving problems[33]. An important intervention component is the assessment-feedback cycle that continuously provides clients and consultants with information for future actions. Consistent with these tenets, the NCOD’s focus on small action teams and data collection forms an integral piece of the intervention strategy.

Process consultation constitutes “the reasoned and intentional interventions by the consultant into the ongoing events and dynamics of a group with the purpose of helping that group effectively attain its agreed-upon objectives”[35] (p.8). The clients are the subject matter experts; the consultants’ sole involvement is with the client process. The groups targeted by the intervention therefore provide the content and means for action and set the goals. The consultants provide input into the “how” and not the “what” of the intervention.

Practice: The NCOD interventions involve collecting work environment data using standardized instruments supplemented by semi-structured interviews. Qualitative analyses articulate organizationally relevant patterns or themes, through consensus of the interviewing practitioners. The NCOD then feeds this information back to groups of employees via a PowerPoint presentation. Everyone interested is invited; complete transparency is part of the approach. Participants are welcome to confirm or challenge the accuracy of the summaries to ensure it accurately represents all perspectives. Employee action teams are created publicly to address the undesirable aspects of the work environment, which supports process transparency and accountability for implementing the planned changes. Strategic decisions (e.g., desired directions of change at the facility, next steps needed to implement them) are the organizational client’s responsibility.

MATERIALS AND METHODS

The NCOD collected contemporary data during the intervention, in 2002 (pre) and 12 mo later in 2003 (post). The same employees (approximately 240) who participated in quantitative surveys of burnout and work environment pre- and post-intervention also participated in the interviews; all personal and organizational identities have been masked. Retrospective data were key informant interviews conducted by Moore[8] four years post-intervention. All the participants permitted using their data for research.

Contemporary qualitative interview procedures

After the standardized instruments were completed, NCOD staff of organizational consultants used the standard NCOD procedure for conducting confidential individual interviews in a private setting, scheduling 30 min per participant and asking participants (120 pre- and 120 post-intervention) the same four questions: (1) “What would you describe as strengths of (the VAMC)?”; (2) “What would you describe as weaknesses of (the VAMC)?”; (3) “Which changes would you like to see happen at (the VAMC)?”; and (4) “Is there any further information that you would like to share with me about (the VAMC)?”.

The consultants recorded the responses verbatim and read back to participants to ensure accuracy. The operational parameters of the intervention and respondents’ sensitivities regarding confidentiality prevented the matching of employee interviews from pre- to post-intervention. NCOD made a concerted effort to interview the same employees, achieving 75% overlap in the sample from pre- to post-intervention (n = 120 participants both pre- and post-intervention). Initial qualitative analyses[8] were conducted using NVivo. Secondary qualitative analyses of the transcribed interviews (current study) applied the assimilation framework and the APES to track change in participants’ problematic experiences.

Retrospective qualitative data collection procedures

Moore[8] conducted additional interviews with five NCOD members active in the intervention, two union officials who represented the VAMC employees, the director of the regional Veterans healthcare network to which the VAMC belonged, the Associate Deputy Under Secretary for Health (national-level VA administrator), and two of the four top hospital administrators from the time of the intervention (n = 11). The initial participation requests were emailed, the informed consent was obtained, and the interviews were scheduled. Three NCOD interventionists were interviewed face-to-face, all others by phone; all answered the same semi-structured questions (Table 2), with probes for detail. All questions addressed professional domains; no personal information was requested. The interviews were digitally recorded and transcribed for content analyses.

Table 2 Retrospective interview: Structured questions and domains of interest.

Questions	Domains of interest
	Events	Climate	Actions	Outcomes
In retrospect, what do you think were the key issues at the VAMC at the time of the NCOD intervention in 2002	Events that lead to intervention	Overall feelings about environment at the facility	Management, union, employee, legal
In general, what do you feel the intervention accomplished		Changes in the overall feelings of employees	Policies, plans	Lasting impacts on the VAMC, qualitative judgments of individuals who participated at a high level in the intervention
What do you think were the most helpful practices	NCOD events, management events		Management, union, employee, NCOD
What do you think were the least helpful practices	NCOD events, management events		Management, union, employee, NCOD
How do you think things could have been handled differently	Number and types of events	Party responsible for intervention	Type of intervention, management response	Ideas for improved outcomes, responses to employee ideas
Do you feel there were overall improvements at the facility		Changes in overall employee attitudes and morale, community perception		Long term evaluation, immediate improvement, sustainability
What do you feel were the causes of improvements or the lack thereof	Management, NCOD, and community events	Willingness to change of management and employees	Management, union, employee, NCOD	Effects of success or lack of success

VAMC: Veterans Affairs Medical Center; NCOD: National Center for Organization Development.

Qualitative analysis procedures

Analysis of contemporary and retrospective interview data was conducted applying the Assimilation framework. The second author, a clinician experienced in applying the model and the APES metric, read the full interview transcripts and coded participants’ verbatim statements for levels of coping with the problematic workplace experiences. The APES coding follows a 4-step iterative qualitative procedure[36]: (1) Familiarization and indexing. The data content is catalogued; extensive notes are taken and indexed to help locate passages of interest later; (2) Identifying and describing problematic experience. Using the indexes, prominent problematic experiences or themes are identified. The criteria include the content discussed, contextual events mentioned in this connection, position taken vis-à-vis content, and associated affect; (3) Selecting passages. The catalogue is searched for passages related to the distinct themes. The resulting set of passages that represent the experiences of interest are excerpted and re-read in context, to narrow and refine their understandings; and (4) Describing the process of assimilation. Each selected passage is given an APES rating with a brief rationale, based on descriptions of APES stages (Table 2) and the known markers for the stages[37].

Passage ratings thus describe assimilation levels specific to problems. Assimilation levels can be considered at given times in the change process (e.g., at the beginning, end, or during specific points within the intervention). Importantly, the model is descriptive of the stages of clients’ change rather than prescriptive of interventionist techniques; it is therefore suitable as a change metric within any conceptual approach to interventions. The interviewer, interview participants, and interventionists were not familiar with the Assimilation model at the time of intervention nor at the time of the interviews, thus the interview data were unbiased by the model’s expectations.

RESULTS

We organized the results by first reporting, for each source, the perceived main problem(s) pre-intervention. The points of interest in all the reports included identifying the perceived main problems at the VAMC, and comparing at which assimilation level they were experienced pre- vs post-intervention. For each problem, we report how it was experienced pre- and post-intervention, along with the assimilation rating and its rationale.

The results are presented first from the retrospective interviews with the VAMC administrators, then with the VAMC union leaders. Juxtaposing their perspectives on the main problems and resolution presents the core conflict as experienced by the intervention’s main participants. Extensive source material illustrates the nature of the data and the logic of the assimilation ratings that substantiate our conclusions. Presenting extensive (AKA “saturated”) data constitutes an equivalent of reliability and validity checking in qualitative analyses[38,39]. The contemporary interviews with the VAMC employees are presented following the same structure, but, unlike the retrospective interviews, these reflect the employees’ collective experience of the VAMC’s problems. As the interventionists and higher-level administrators were not direct parties to conflict and associated change, the interview data collected from them are not included in this study. Table 3 summarizes the APES levels, pre- and post-, for all interviewee types.

Table 3 Main problematic experiences by informant type, and pre- to post-intervention changes.

Type of informant	Problematic experience rated on the APES	APES level at pre	APES level at post
VAMC administrators (n = 2)	Admin 1: Employee perceptions of discriminatory practices (the allegations)	2	5, 6
	Admin 2: Personal experience of the allegations	2	5
	Admin 2: Existing practices at the VAMC	0, 1	6
	Admin 1: Management-union relationship	1.5	2.5
	Admin 2: Management-union relationship	1.5-2	3
Union representatives (n = 2)	Union 1: Lack of communication (management and unions)	2	2.5
	Union 1: Lack of employee empowerment	2	6
	Union 1: Resenting interventionists “interfering”	2	3
	Union 2: Management “doing as they please” (discriminatory hiring and promotions, lack of accountability, resulting employee disempowerment)	2	3
VAMC employees (n = 240)	Existing practices (racism, favoritism, unfairness), caused by uncaring or weak leadership, result in negative workplace climate, low morale, disempowerment of employees	1, 2, 3	5, 6
	Intolerant, adversarial attitudes by supervisors of certain areas cause no cohesion between staff	1, 2	4, 5
	VAMC-wide lack of communication, training, and support for job-related tasks	2, 3	3, 4, 5
	Understaffing creates many problems which are not addressed by leadership	2, 3	3
	Bad public image of the VAMC is unfair and depressing	0, 1, 2	3.5, 4

VAMC: Veterans Affairs Medical Center; APES: Assimilation of Problematic Experiences Scale.

VAMC administrators: Retrospective interviews

Problematic experiences: The interviews with two VAMC leaders (Admin 1 and Admin 2) described allegations of discriminatory practices in hiring, firing, promotions, and awards by the VAMC management as the primary problematic experience. Both leaders commented upon the organizational as well as personal experience of these events. Both described an adversarial relationship between management and the unions as a related, exacerbating problem as the unions’ leaders reportedly actively steered employees to file complaints against management and contacted the local newspapers.

Allegations of discrimination: Admin 1’s account of how she personally experienced the allegations of managerial misconduct pre-intervention was consistent with APES level 2. This account evidenced acute affective pain and conflict associated with the problematic content.

Admin 1: The allegation was that we conducted ourselves improperly. It was very offensive. I can tell you to me personally, my children were in the early to mid-teenage years, they read the newspaper every day. They knew who hospital management was. They had been out to eat with these people; they had been to these peoples’ homes. They knew that their mother was one of those people and that I was being charged with discrimination. My son called me at work one day and said, “Hey, Mom, you finally dropped to the lower half of the front page.” Okay? To me, it was a personal purgatory because of having children in their teenage years who question parental authority and parental wisdom. It was purgatory.

This problem reportedly became resolved through greater transparency about the management actions, which, together with a third party review, successfully cleared the allegations and became the primary cause of improvements at the facility.

Admin 1: Because of the process, we went through. The allegations were the stimulus. A review was conducted by a third party. There was a complete and total vindication. But since the employees themselves didn’t make the allegations, I don’t think that meant much to employees. It meant a lot to management because our jobs and our integrity and our-not just our jobs, our professional lives were on the line. (Interviewer: In terms of improvements among the lower-level employees, what do you feel were causes for their. Admin 1: That’s the part that was the process. I think just illuminating to people, informing people, reminding people we’ve established a shared drive within Vista, this is how you get to it, here’s reports. Here’s other meeting minutes, things like that. It was transparent but I think employees just didn’t know how to access that transparency.

Admin 1’s description of this same experiential content post-intervention was between APES 5 (applying the new understandings to work through the problematic issue) and 6 (partial success); particularly clear examples of these ratings are in bold.

Admin 1: The work of the hospital, that should be general knowledge, was general knowledge-or available, and perhaps not general knowledge. And then it was made general knowledge. It was always available. Butwe put more of an emphasis on making reports, outcomes, findings available. So people’s awareness was heightened and processes were highlighted to make sure that everyone knewthat what was appropriate to share with employees was being shared with employees.

Discriminatory practices: Whereas Admin 1 presented mainly the management’s and her personal experience of the allegations, Admin 2’s account of presenting problems emphasized the VAMC workplace climate from the whole organization’s point of view. Discrimination-related problems were described with a broader focus on the existing practices rather than the allegations made.

Admin 2: The key issue is there had been racial discrimination there for a long time. It was open and people indulged all the time. It started off, even before NCOD got in the middle of it, I got everybody together, all the offenders - which was about fifteen to twenty - it was supervisors and engineers.Mainly it was in the engineering shops. I told everybody that I didn’t put up with that crap. Well, in the ensuing melee of (senator) got in the middle of it, trying to get votes and throwing his weight around and all that stuff, we had a real shootout for about a year. Then to heal the process, that’s when I brought NCOD in. (Interviewer: Okay. So the key issue was longstanding-)

Admin 2:Oh yeah, twelve years before I got there, they were doing it.Stuff like putting hangman’s nooses up in the room, saying this was going to happen to you, boy, if you don’t get your butt in line. All that kind of crap. I thought I was out of the South but I guess (the VAMC) was still there. Then they’d open up the hood of a car, it would be a whole bunch of pump shotguns and stuff, pistols and stuff, I will use on you if you don’t get your act together. Just open intimidation.

Admin 2’s description suggests a long-standing lack of concern with the unacceptable practices, from at least a substantial part of the VAMC supervisors (in bold). This level of organizational denial and avoidance is consistent with APES 0 (successful denial) and APES 1 (attending to issues only when forced by external circumstances, here by Admin 2 and by the intervention).

Admin 2’s account of how these problems were resolved mentioned drastic measures:

Admin 2: I ended up firing about five supervisors in the engineering shops, like air conditioning shop, electric shop, ground shop. I fired the chief of engineering. Who else. Couple of other people around the hospital involved in it. Took a long time to fight that out. (Admin 2 refers to difficulties overcoming resistance to the intervention from some of the staff).

Both Admin 1 and Admin 2 attributed the resolution of the discrimination issues directly to the OD intervention. Admin 2 described the change process as developing greater awareness, inviting an open discussion of concerns, and enforcing the existing rules, sometimes quite forcefully so:

Admin 2: I think it (the intervention) calmed things down. People were bringing guns to work. I told them, you can’t do that, I’m going to start searching your cars. It also brought a fairness that wasn’t there before. There was people in supervisory positions that were bad-they got gone. They couldn’t stand, when we cranked up and started doing good, they could only survive if they could persecute somebody. So they didn’t really know how to do the workload and stuff. So they opted out on their own, most of them. Some of them we fired, because we caught something on them. But I think it helped the attitudes. It helped people’s perception of praise and reward that they got from the system.

Eventually, according to the account, discrimination-related problems at the VAMC were conquered (specific success: APES 6, in bold):

Admin 2: NCOD was an absolute essential piece of this turnaround effort. Absolutely. When I got there, we had 154 EEO complaints. We got it down to like thirty.

Admin 2: We had a much better workforce. Our performance measures went up. We started controlling the budget. Less people started quitting because they couldn’t stand to work there. People actually thought in the hospital that they had a voice, which they did. We put in systems, ways to get things done, with a peaceful-type approach. The people out in the community changed the way they looked at the VA.

Admin 2: The deputy secretary of the VA read all the reports. When I went down to get chewed outhe said, I read everything - you’re the only one that did anything about it. He was very pleased with what happened.

While the resolution of the organizationally problematic experience (above) was rated at APES 6, several statements by Admin 2 also reveal traces of a related but different problematic experience - the remaining personal hurt (in bold italics below). This echoes the problematic experience of allegations discussed by Admin 1.

Admin 2: When I left in - I guess it was March, April of 2004 - in January, (the union’s) annual award of excellence in labor relations, they gave it to me. After it was all over with and I won, they gave me these damn awards. It really wasn’t worth the fight, tell you the truth. What really was worth it is NCOD came in and that would never have happened before, and it gave people a chance to believe that management cared about them a little bit.

Prompted by the interviewer, Admin 2 explained how he coped with this difficult experience:

Admin 2: I’m older now and I don’t have that attitude. That was a hell of a fight at (the VAMC).

Interviewer: So you’re older now, you have a different attitude-did that process have a role in changing?

Admin 2: Yeah, how much I put on the table of my own substance, to get things done. I aged, I got sick and thrown in the hospital, I got adult-onstage diabetes from that. It was a lot of personal giving on my part to get this done. When I go somewhere else now, I don’t push quite hard. Instead of trying to get it done in a year, I’ll get it done in a year and a half -20 mo if it takes me. I developed a theory there, a theory of the coefficient of change. Every place is different but every place has an ability to change. If you exceed that ability, the whole thing is going to blow up. Then you have a real big problem. So I don’t push quite as hard.

Interviewer: So when you refer to not pushing quite as hard, so before NCOD got there, you were pushing hard.

Admin 2: Oh yeah, it incensed me.

This description suggests pre-intervention APES level 2 for Admin 2’s personal experience of discriminatory practices (“it incensed me”). The subsequent naming of the problem (“how much I put on the table of my own substance”-APES 3), insight into how it works (developing a personal theory about “coefficient of change”-APES 4), and application of this theory (APES 5) resulted in a working solution: instead of “incensed”, Admin 2’s stance became adjusted into “I don’t push quite as hard”. At the time of the interview he was apparently still working through some remaining personal hurt (therefore APES 5, not APES 6 ratings), but Admin 2 arrived to a relatively high level of coping with these personally difficult experiences.

Management-union conflict: Both leaders directly connected the allegations of discrimination to union activities, and offered highly consistent accounts of their pre-intervention experience of the problem (APES 1.5-2 ratings). Admin 2’s account was more elaborate.

Admin 2: The unions were just pure-they were planning stuff. They were planting nooses. They were doing anything to fuel the fire because the stated reason for the union in (the VAMC) was to bring down management. They didn’t care if they brought the whole damn hospital down. So you either gave into everything they wanted, which I ain’t never done in my life and never will, or you fought them on some kind of limited engagement. They would send stuff to (senator) that wasn’t true. Most of their people were black in the union, and I was trying to better their environment. It was the damnedest thing I ever saw. They coached people to file EEOs. They coached people to file-they brought in lawyers to coach people. Here I am trying to stop people from harassing people. I had the union president, who had a sixth-grade education, tell me she could do a better job running the hospital than me. That was the attitude. It was totally adversarial. Totally adversarial. I even had her tell me: I will say and do anything to try to win cases for my people. I said, don’t you care anything about the truth? Nope.

Admin 1 stated that, due to the union’s lack of cooperation, the adversarial relationship between the management and union remained “essentially unchanged” from pre- to post-intervention:

Admin 1: The lady who made the allegations was the co-chair and she never came to the meetings. She wouldn’t work with the members of the team. Not me, but the actual members of the team.

However, Admin 1’s description of the management-union conflict showed clear movement from APES 1.5 pre-intervention to APES 2.5 or above (but less than APES 3) post-intervention. Pre-intervention, Admin 1’s perception of the conflict included a painful experience of adversity and conflict (APES 2 features) which the management felt to be externally imposed on them by the union (APES 1 features; therefore not a full APES 2 level). Post-intervention, Admin 1’s description of this same conflict showed a less emotional, more cognitive stance:

Admin 1: It wasn’t the employees making the allegations. So in the end, all we could do is show the report. That’s kind of the proper government process that went on. I think that’s okay, I’m sure that was of benefit to some employees. But the employees who would be working with the union and saying someone’s not being fair to me - I don’t think they would believe that the report was being fair to them either.

Admin 1 also shared an initial formulation of just what the problem was (in bold below).

Admin 1: It’s not about the issues. It’s about creating stonewalling and carrying on. It’s all about perceptions than fact and reality.Fact and reality have no bearing on stirring the pot.

This problem formulation was still incomplete (not a full level 3) because it lacked a clear representation of what the other side of the conflict saw as a problem (i.e., exactly what prompted the union’s discontent). Admin 1 confirmed the interviewer’s summary suggesting some progress in the management-union conflict but no full resolution (in bold below):

Interviewer: You said it was predominantly about one person, the union president, and that if that relationship had been different these issues would not have occurred. That relationship isstill driving some issues, albeit at a much less high-profile level.

Admin 1: Yes, that’s true.

In Admin 2’s description, the management-union conflict was resolved through the management working hard together to present their true case, gaining employees’ trust, and letting go people with incompatible agendas who “stirred up trouble”.

Admin 2: I had to last the whole time out and so did the three people who worked for me. All four of us had to stick together. It wasn’t easy on anybody. We were all working extra hours to try to get this thing right and get rid of the people that needed to go, that just were troublemakers. They didn’t work. They stirred up trouble all the time.

Admin 2: Oh yeah, nobody trusted nobody. When it was over with, we had a pretty amount of good trust all over the hospital. How we did that was people who were mean, who wouldn’t work, who agitated, who persecuted - eventually they all got fired. They had to do their part in solving problems in their service. They really weren’t that good to do it, so they just retired and went away.

In this account of how the problem was resolved, taking the right side of the conflict and sticking to it is a key theme. Typical of APES 3, it includes an articulate, polarized perception of the conflict and the exact problem each side had with the other.

Union Leaders: Retrospective Interviews

Problematic experiences: The two interviewed union leaders were the main participants in the VAMC events preceding the intervention. Their somewhat different but compatible accounts described the initial problems as lack of communication between union and management (Union 1), and management “doing as they please” (Union 2).

Lack of union-management communication: According to Union 1, “the number one issue was communication, the lack thereof”, specifically “a gulf between the union and management”; “all other issues stemmed from lack of communication”. Asked whether it was a lack of ability to speak to each other or rather, hostile or unhelpful communications, Union 1 replied, somewhat indirectly: “Communication is a two-way street so nobody is exempt not being able to communicate”, and later elaborated:

Union 1: I recall that in my frustration, I would be talking quite loudly and not in complementary terms about what my frustrations were. That’s not a good way to communicate, because nothing came of it. So as a union person, I was really frustrated but I didn’t know how to break the deadlock, break the barriers. I didn’t know how to do it because I was angry and frustrated and hostile. That’s all a result of no communication.

In Union 1’s opinion, her colleagues felt similarly frustrated (consistent with APES 2 intensity of affective experience) but management reportedly avoided dealing with these issues. A description of pre-intervention meetings between union people conveys an overwhelming confusion and pain (APES 2 features):

Union 1: As I recall, we didn’t even have an agenda. We didn’t even have a goal. There was nothing. It was just horrible to even go the meetings. Without NCOD involvement, I don’t think we would have gotten anything done. It forced us to look at potential issues, talk about them, set up an action plan and set dates achieving steps along the way to achieve the action plan and the goal. (“us” referred to the union people-not employees or management).

For Union 1, the most helpful aspects of the intervention included NCOD facilitating the union council meetings; using a flipchart to capture the ideas discussed, helping formulate them into action plans, naming responsible parties, and setting deadlines. Union 1 confirmed the interviewer’s summary that it was “like a communication exercise” which is why it was helpful. She also noted: “They (NCOD) could have forced our group to mimic their technique. We never were forced to do it on our own with them as observers. so it collapsed when they left. It, meaning the technique collapsed when they left.” As these descriptions of both helpful aspects and regrets suggest, Union 1 experienced putting the concerns into words as the main need, a prerequisite to any subsequent work. This is a classic dynamic for APES 2-3 transition. While Union 1 found it extremely helpful, she also felt it was incomplete (the technique was consumed, but not internalized). These and other comments suggested APES 2.5 as a post-intervention rating of Union 1’s experiences around lacking communication.

Lack of employee empowerment

Another problem, characterized by Union 1 as stemming from poor communication, was a lack of employee empowerment; rated at APES 2 pre- and APES 6 post-intervention. Here is a representative quote:

Union 1: I think the employees felt better with the process action teams being onboard and them being able to take part in hospital improvements. I think that really helped the employees who wanted to be part of hospital improvement.

Resentment of the intervention: Union 1 also offered some revealing comments on her personal experience of the interventionists’ efforts-highly consistent with our APES 2 ratings of her main problematic experience with lacking communication.

Union 1: Can I say something that may be a personal note that doesn’t necessarily have to be part of your-unless you feel it would be important? Well, you can use your discretion. I hated the NCOD. I didn’t like it at all. I resisted it and I was not a friendly participant. They were very nice. They were very good about setting up meetings and asking for responses and they were very good at facilitating everything. All of them were very pleasant. But I didn’t like it. I didn’t like it. The word is resent. I resented it. NCOD pushed us ahead, made us do stuff, but those negative feelings were never dealt with. Maybe that’s why I resented it, because my big thing was we had real issues that were met with silence by management. They would literally stonewall us. I never got over that resentment. When you have resentments like this, that aren’t dealt with and aren’t laid on the table.

These comments reflected an acute and lasting painful affect, without cognitive understanding of the exact nature of the problem (APES 2), at the time of the intervention. The passage below shows an emergent understanding, during the interview, of what exactly the problem was (APES 2-3 transition).

Union 1: As I talk to you, I’m kind of sorting out why I didn’t like it. I think that’s why I really didn’t enjoy the experience at all, because I still hung onto the resentment. It never was resolved.

Interviewer: Are you talking about resentment towards management or-

Union 1: Resentment towards the management. We had spent years of hostility and conflict and suddenly Bugs Bunny comes in with his carrot and says, “What’s up, doc?” (¡) That would have been something that would have made it better for me, if is we could have expressed that and asked (the management), why aren’t you doing anything?

Union 1 connected her resentment about NCOD’s lack of focusing on negative aspects, to the resentment she felt about the management “silence”. Her strong negative emotional reaction to the interventionists was thereby reframed as her indirect re-experience of the root problem: lack of communication with the management. Her stating this link to the interviewer represents a classic APES 3 example: Clarifying the emotional pain and confusion by articulating the core of the unpleasant experience. Union 1 then added, “anyway, I was resentful through the whole thing. I remember not being very nice” - an accurate articulation of how her own stance was perceived by the other side of the conflict, again consistent with APES 3.

Discriminatory practices and employee disempowerment: Union 2 named management “not being held accountable and not caring about the employees” as the main problem pre-intervention-consistent with Union 1 perception, but with a more accusatory emphasis. Union 2 specifically referred to favoritism in promotions and hiring (“hiring who they wanted for positions”, “good old boy system”), and the resulting low morale:

Union 2: The employees felt it. It was like they just knew it was horrible here. They were unhappy. They weren’t recognizing employees that were going up and beyond.

Union 2 expressed her view that the VAMC problems were named but unresolved, and described the facility culture as, “Ingrained culture that’s never going to change, because management won’t recognize some of those things”. Nevertheless, her statements showed a movement from level 2 to 3 in her experience of the conflict post-intervention. In Union 2’s view, “upper management, central office” gained a “heightened awareness that there were problems here and people had to start watching their Ps and Qs” (positions and qualifications). Echoing Union 1’s comments about employee empowerment, Union 2 stated the intervention “gave people a sense of hope for a while.” However, Union 2 also felt the 18-mo intervention was too short and more of the same process would have been beneficial. Union 2 pointed to increasing awareness and naming the problem as the main helpful developments, and, like Union 1, she named facilitating communication as the most helpful intervention aspect.

Union 2: I don’t know-the most helpful practices was just the fact somebody came and listened, and pointed things out.

Interviewer: So just the presence.

Union 2: Right, of somebody from the outside coming to pay attention. A positive thing came out of it, I think, that I personally have been able to convince people that their voice does count, and they’re more involved.

Five NCOD interventionists and two top VA administrators (above the VAMC leaders) offered accounts of VAMC presenting problems and solutions, which were consistent with the presented perspectives from VAMC managers and union leaders. These additional accounts are not included here for space considerations.

VAMC Employees: Contemporary interviews

Examining group-level perceptions of the VAMC problems from pre- to post-intervention helped us describe employees’ shared organizational experiences. All of the time 1 (pre-intervention) employee interviews, collapsing specific individuals’ perspectives, served to identify common themes. The themes were then rated and compared to the common themes and ratings from all of the time 2 interviews (i.e., all the data collected post-intervention). The following sections summarize themes tracked in the participants’ pre- and post-interview data.

Pre-intervention: A predominant theme was an overall negative workplace climate: Racism, favoritism, unfairness, and no accountability for these behaviors. Employees also mentioned low morale (feeling overworked, mistrusted, watched, pressured, scared of being unfairly accused), having no voice, no autonomy in doing their job, and no awards for good performance. VAMC leaders and immediate supervisors reportedly ignored employees’ needs, whether in deserved recognition for performance or support for day-to-day operations.

Supervisors’ hostile, imperial attitudes formed a distinct theme. The intolerance for alternative views reportedly created an adversarial atmosphere, with no cohesion among staff.

Communication of the job-related information and procedural support were reportedly limited or non-existent. Respondents attributed this to large workloads and poor planning of procedures, communications, and trainings.

Understaffing created problems with too much work; a limited ability to serve patients; decreasing quality and timeliness of services; high stress, burnout, turnover; staff “bickering” and “growling”; and lacking respect and appreciation. Also mentioned was a lack of action by leadership in addressing these problems.

Public perception of the VAMC as all-bad due to bad publicity was cited as depressing, and hurting employees’ morale. Some respondents claimed this image was really unfair; a few aggressively stated nothing was wrong with the facility.

The employees also noted the facility’s strengths - patient-centered care with a strong focus on quality services; patient-friendly accessible location; a rural culture with a personal touch (relating to patients as individuals); dedicated, committed, compassionate staff; job security and benefits; staff resilience (“we can do it” attitude in spite of the challenges); some excellent supervisors; and teamwork (mentioned as strength by some, and weakness by others).

Post-intervention: Many interviewees reported positive changes in workplace climate “lately”, “recently”, or “now”, reportedly a result of “getting together”. Interviewees mentioned management improvement, and praised specific efforts of the top management, although some (few) commented about favoritism, double standards, not holding people accountable, and remaining trust issues (‘it takes 5-10 years to turn things around, and I believe we are still struggling”). Importantly, there were no comments about racism post-intervention. Representative comments were, “I think we are moving beyond some of the issues of the past. People were sick of it-it was painful after a while”; “it’s mostly sunny, but a few clouds are rolling around.” Several respondents mentioned a very positive atmosphere and elaborated how and why, thus sharing a perspective dramatically different from pre-intervention. A discontented few reportedly created much negativity (they “complain”, “do not listen to the explanations given”, “do not try to understand why”); respondents suggested to focus less on these people. Workplace problems still frequently mentioned post-intervention included insufficient educational opportunities, awards, and promotions, and lacking accountability for people who neglect their jobs.

Few if any post-intervention comments expressed generic discontent regarding supervisors’ attitudes. One negative comment was that supervisors “want to be micromanagers and this doesn’t work”. Considerably more respondents than at pre-intervention mentioned a need for better training of supervisors in specific job functions. Some wondered whether supervisors’ hands were tied for addressing specific issues (e.g., dealing with poor performers). This perspective-taking was markedly different from the predominantly blaming stance pre-intervention. Whereas pre-intervention, supervisors were described as hostile, rigid and uncaring, post-intervention comments about supervisors had a more constructive, less emotional tone and were more specific, therefore more addressable.

Many positive comments were about recent improvements in work-related communications and procedures. Importantly, few post-intervention comments were generically negative. Any noted negative aspects were specific (e.g., lack of coordination when scheduling patients for appointments between services). Respondents frequently elaborated what would improve the situation and shared their analysis of why the issues existed, which reflected greater cognitive clarity of the problems. Reasons for problems included rigidity of the bureaucratic rules that no longer fit the existing needs, difficulties getting people to think entrepreneurially, and the fragmented structure of service lines leading to poor communication between services.

Understaffing and the problems it created were still frequently noted as a weakness post-intervention. However, comments were both more positive and more specific compared to pre-intervention, with much less generic negativity.

Negative aspects of the facility culture were noted as responsible for the public image of the VAMC (a new perception compared to pre-intervention). Rigidity, political culture (rather than job- or patient care-oriented culture), and “values that don’t support diversity” were named as the organizational climate aspects that fueled prejudices and EEO tensions. Pre-intervention, respondents simply expressed acute distress over the negative public image (APES 2), externalized the reasons for negative publicity (APES 1), or bluntly stated everything was fine (APES 0).

A weakness repeatedly mentioned post- (but not pre-) intervention included the union-management conflict. According to other sources, these issues existed pre-intervention, although employee interviews did not mention them. Post-intervention comments suggested a greater awareness. Several respondents stated unions had too much power and inappropriately supported problem employees (those not willing to perform). Several employees simply noted that unresolved differences existed. No respondents blamed management for the conflict when discussing the union-management differences.

Notable topics specifically mentioned as strengths post-intervention were leadership support and efforts to turn the facility around, greatly improved communication, honesty and direct contact with top management. Civil climate (the representative comments were: “improved relationships”, “more employees are open than ever before”, “at least 98% of employees are nice”, “people are getting along and working together”, “family atmosphere”). Good customer service, trust and recognition (mentioned as strength by some and as still needing improvement by others), and well-defined roles and responsibilities (indicated as strength by some but as still lacking by others). The same strengths as pre-intervention were mentioned as well (resilient, dedicated staff, concern for the veterans, delivering compassionate care that is liked by patients, good benefits and pay).

DISCUSSION

Like others[18,40], the present authors believe that integrating “outside” theories into organizational research can be productive. The assimilation framework reveals and clarifies relevant, previously unexamined aspects of organizational settings. Doing so allows linking the otherwise separate pieces of data that together demonstrate what difference the intervention made, as well as through which psychological processes the intervention made its impact upon the participants. For example, psychological defense aspects, although found to be related to lower psychological well-being[41], remain largely unexamined in workplace stress research[18], partly because it is difficult to operationalize them (i.e., define psychological defenses in ways that lend themselves to consistent measurement). The APES contextualizes defensive responses to problems, through mapping them onto specific problematic contents and rating which assimilation level of this content is manifested in this response. The “Resentment of the Intervention” example in section 6.2.4 illustrates how a particular response (the initial dislike of the interventionists) resulted from a perceived similarity between their behavior (focus on the positives) and the highly disliked behavior by the facility’s management, whom the respondent experienced as silencing the real problems at the facility. The response of disliking the interventionists was thus an indirect way of re-experiencing the unassimilated problematic experience (the management’s perceived rejection of the employees’ perspective). This indirect way of experiencing constitutes a defense: warding off rather than facing the subjectively stressful problematic content. The APES continuum that extends from extreme avoidance to full assimilation of problematic experiences thus explains defensive reactions as pre-APES 2 stances towards a given content.

The Assimilation model can systematically summarize the intervention progress at higher levels of generality than typically measured by change models; progress is tracked as change in participants’ psychological processes that are used in response to the problem, not only as a change in symptoms or in overall distress levels. This is because the model defines progress as the client’s growing ability to non-defensively deal with the problem, which is manifested differently at different APES stages (AKA levels; Table 1). That is, depending on a stage, progress may or may not involve feeling better, reaching more advanced understandings, or showing specific accomplishments in resolving the problems. To explain this more specifically, at early stages, progress involves learning to face the problem rather than turn away, and results in an increase of negative affect[42]. This increased negativity, however, represents progress compared to the lowest APES stages (0 and 1), which are characterized by successful avoidance of the problem, and may therefore involve relatively little negative affect. In effect, increased negative affect is a prerequisite to subsequent development in the assimilation sequence; fully experiencing the problem (APES 2) precedes the work of resolving it. When the client reaches APES 2, the avoidance breaks down and emotional negativity reaches its peak as the client faces the problem without turning away or minimizing its full scope (Table 1). This sustained ability to face the problem and tolerate the negative affect without breaking the exposure or awareness of the problematic perspective is what ultimately allows to name the problem (at APES 3), then develop a more elaborate understanding of its contingencies (APES 4). From APES 3 and thereafter, progress in assimilating the problem is accompanied by relief: The emotional pain loses its edge as the client gets a grasp upon the conflict and formulates it in addressable terms. Progress is not, however, synonymous with relief at APES stages prior to 2. Conventional outcome measures do not make this crucial distinction (i.e., that relief is a marker of progress at later, but not at earlier stages of working through problems); they define improvement as a decrease in the intensity of the problems experienced, which we suggest is an oversimplification. In the assimilation framework, as we have explained, this definition of progress-as-relief fits only the middle part of the change continuum, between APES levels 2 and 4[43]. Only for this segment of the change sequence, progress is synonymous with relief of symptoms, problematic behaviors, or psychological pain. (At earlier, 0-2 APES stages, progress brings greater awareness of the problem and thus more, rather than less, distress, whereas at later, 4-7 stages, a prominent feature of progress is an increase in positive feelings, not distress reduction[42]. The APES sequence thus provides a broader developmental context compared to exclusively symptom, problem-, or behavior- focused approaches that typically inform outcome measurement. In our intervention example, this broader context made all the difference between the evaluation approaches which reflected (APES) vs missed [Maslach Burnout Inventory (MBI), Work Environment Scale (WES)] the pre-post change that was strongly perceived by the intervention recipients.

As an additional benefit of the assimilation model, its tool (the APES) represents a numeric system for measuring and tracking clients’ changing experience of their problems. The scale quantifies qualitative meanings, which allows logical manipulation of the information they express[44]. For example, quantifying the levels of coping with the organizationally problematic experiences allowed comparing the informants’ assimilation of specific problems from pre- to post-intervention, and comparing the change across informant types. This offers a solution to the difficult challenge of scientifically tracking in an unbiased way the intervention-assisted change in the interpersonal processes that are, by definition, subjective.

In assimilation studies (including this one), the raters have been clinicians. The APES served as a tool of translating their expert clinical judgments about the client’s change status into numbers (APES levels), thus incorporating both the qualitative judgments informed by clinical expertise and the properties of numbers. Since in studies of change the object of measurement is complex human experience, the APES relies heavily on raters’ interpretive and inferential processes, but APES ratings are therefore no less quantitative than other ordinal numeric scales. The present authors, like others[45-47], believe that the distinction between numbers and words is much less fundamental than many researchers think; these are compatible, mutually complementary methods of building scientific knowledge. Maintaining a balance of quantitatively informed and qualitatively enriched data appears particularly important in examining healthcare services and settings, given that most topics in this field require reflecting people’s subjective, contextually embedded experiences. Yet theoretically grounded, methodologically transparent applications of mixed methods in this field remain rare[48-50] This study seeks to contribute to the emergent mixed method tradition. We believe that properties of numbers can usefully enhance contextual interpretive inquiry. For example, the APES stages link general psychological processes (affective states, behavioral and cognitive coping strategies) to numbers that express assimilation levels of specific problems (e.g., perceptions of unfair hiring practices). The APES combines in one tool a qualitative and quantitative approach to people’s experiences and psychological change processes-a strategy that affords studying them in systematic and specific ways (by examining numbers), while also treating them as complex and unique (by incorporating qualitative meanings).

Examining qualitative data within the conceptual framework of assimilation pinpointed the specific changes at the VAMC that explained the stakeholders’ expressed satisfaction with the intervention. The APES quantified levels of change in the participants’ workplace experience from pre- to post-intervention (Table 3). Quantifying the amount of change illustrates benefits of an approach anchored in the study of general psychological processes of change, for evaluating specific changes through an OD intervention. In sum, this approach was capable of providing an answer to the otherwise puzzling inconsistency between the qualitatively vs quantitatively measured outcomes of the intervention[8] because the Assimilation model conceptualized change at a higher level of process compared to the previously used evaluation models.

We described the nature of change in the informants’ experience from pre- to post-intervention as predominantly a transition from early to middle APES: From more towards less avoidance of the problematic experience. This movement initially resulted in more, rather than less, felt conflict, distress, anger, and resentment, a typical logic of change from early to middle APES stages (Table 3). The area of growth (progress) was the increasing ability to face the problem and listen to the opposing perspectives, rather than labeling the problems or finding specific quick fixes. The conceptual logic of the APES stages and the informants’ feedback both converged in suggesting that pushing early solutions at that initial point would have been a premature direction, and possibly would have arrested any further progress at the VAMC. This important point is consistent with the assumptions made by the APES but not by conventional approaches to outcome evaluation. After the initial prolonged period of working through avoidance, the sustained exposure to the problem at the VAMC did eventually help arrive to the point where specific solutions became possible. Sustained non-avoidance (meeting, connecting, and listening) allowed the participants to size up the problematic perspective, both emotionally and cognitively. Enough good will, tolerance, and understanding of the conflict gradually developed from the process that facing the conflict and working through it became a realistic goal. The value of allowing enough time for the initial process is an important insight highlighted by the model; practitioners who conduct psychologically based group interventions and researchers who study them should keep this conclusion in mind.

Notably, multiple interviewees specifically mentioned the positive influence of the intervention process, as opposed to its specific results. Multiple interviewees also noted the importance of the intervention lasting a long time; some even felt it was too brief. Clearly, the intervention process became more important in their perception than the products, we believe most likely because of the relational aspects that it afforded employees who had been stripped of pride in their organization by a series of highly publicized, negative events. The opportunity to form groups with fellow employees and become involved in making organizational changes may have had more impact than the specific goals of those groups. The process gave the employees a way to heal, i.e., to bridge the previously incompatible perspectives and name shared agendas, a prerequisite to subsequent work of exploring solutions to specific problems. This type of achievement is characteristic for the APES levels preceding 3. Importantly, it has poor chances of being reflected in standard (non-assimilation) outcome measures, which typically assess specific outcomes in relation to named (that is, post-APES 3) problems.

Possible reasons for the standardized instruments’ failure to reflect the initial problems and post-intervention changes at the VAMC are worth considering. First, the constructs measured (burnout, general work environment) possibly did not represent the intervention targets (fallout issues from the EEO filings, perceptions of unfair HR practices). Negative workplace, burnout, and the associated stress and exhaustion were, however, strongly, frequently, and directly named as workplace problems in the employees’ pre-intervention interviews. These contents directly informed the focus of the interventions. The measured and targeted constructs thus appear related enough that the wrong construct explanation is unlikely to explain the non-significant findings, especially if the intervention had the strong positive impact the informants suggested. Possibly, the standard measures (MBI and WES) failed to capture the constructs with enough sensitivity. This happens when respondents are reluctant to rate their true experience, and could be the case at the VAMC, especially pre-intervention. While possibly explaining “normal” pre-test scores, a lack of sensitivity still would not account for the measures not capturing a post-intervention change as large as the respondents perceived. Since the MBI and WES did not capture the apparently real pre- to post-changes that described by the respondents, perhaps an alternative quantitative method would have fit the evaluation needs better. On the other hand, while the universe of standardized instruments is large, locating a quantitative measure for any specific focus of an intervention may present conceptual, pragmatic, and psychometric challenges for interventionists. This highlights the merits of qualitative measures, which in previous research have been characterized as uniquely equipped for capturing contextually sensitive, nonlinear causal relationships[51]. Demonstrating an application of a specific, clinically derived qualitative measure, the APES, to the context of evaluating a real-world impact of an OD intervention, with sufficient detail as to allow replicating this approach, represents a contribution of our study.

In conclusion, this paper presented a conceptual framework and a measurement strategy that together addressed a conceptual challenge within the field of organizational psychology and, more broadly, within any area of psychology concerned with evaluating outcomes of interpersonally based interventions. The challenge consists of identifying a strategy for objectively, scientifically evaluating changes in participants’ subjective, interpersonally based perceptions from pre- to post-intervention. This challenge stems from the inherent conflict within psychology which is created by a coexistence of the scientific tradition that informs the concepts and methods of explaining human behavior, and the humanistic tradition that informs the processes and techniques of bringing behavioral change through interventions. The two respective sets of concepts, methods and measures resulting from these alternative traditions cause long-standing conflicts within the discipline regarding an optimal balance between the art of intervention and the science of evaluation. The Assimilation model proposed, and the current authors explained, a solution to this challenge, which consists of applying a theoretically informed and empirically validated method of APES ratings to systematically, unbiasedly tracking participants’ interpersonally based, subjective psychological processes of meaning-making.

The Assimilation model and scale also demonstrate sufficient sensitivity for reflecting levels of change that are missed by the conventional outcome measures. This higher sensitivity becomes possible because the model attends to clinical-developmental aspects of the change process whereas conventional measures focus solely on its symptomatic and content aspects. The present study demonstrated how this higher-level conceptualization allowed clarifying the otherwise large and puzzling inconsistency between the outcomes of a specific intervention as reflected in quantitative vs qualitative outcome measures.

Finally, both the model and method are applicable independently of the type of intervention, setting, the specific problematic content addressed, and the client type, which presents a major advantage for generalizing this approach to other organizational and cultural contexts. The current study was the first to apply the model and measure to the organizational psychology domain. An important contribution of this research is that it offered sufficient detail regarding the current application as to support any interested audiences in their ability to assess relevance and benefits of this approach to their own contexts, including but not limited to psychological interventions within healthcare organizations in the United States.

COMMENTS

Background

Measuring change through intervention in real-world settings presents the challenge of determining in an objective way whether an intervention successfully addressed the presenting concerns that were subjectively experienced by participants. This challenge reflects a conflict between the interpersonally based intervention practices and the experimentally based evaluation methods which characterizes psychological interventions in a variety of settings and compromises the validity of evaluating their intended impact. Specifically, conclusions about the intervention impact may differ as a function of relying upon traditional quantitative outcome measures (objective approach) vs qualitative interview reports by intervention participants (subjective approach).

Research frontiers

Evaluating the impact of psychologically based interventions reflects the underlying conceptual definition of progress. Quantitative outcome measures typically define progress as (full or partial) symptomatic improvement, alleviation of distress, and/or resolution of presenting concerns. This succeeds in quantifying levels of change yet limits the evaluation scope, e.g., by failing to capture the earliest and the most advanced stages of progress. Qualitative outcome measures adopt a broader perspective by using the participants’ own definition of improvement, yet fail in quantifying improvement levels and cause a number of methodological problems associated with self-reports. The hotspot of outcome evaluation research is finding an unbiased approach that captures the full continuum of progress, is sensitive to the participants’ specific perceptions, and allows quantifying change for comparison purposes.

Innovations and breakthroughs

This study offers a strategy allowing for an objective yet context-sensitive evaluation of progress in a psychologically based intervention. The data of participants’ subjective perceptions of problems pre- and post-intervention (transcribed interviews) are rated by a trained clinician, reflecting the theoretical framework used in this study: the Assimilation model. This approach draws upon conceptual thinking and prior research from a related but different discipline: psychotherapy. The ratings represent developmental levels of participants’ psychological process in approaching the presenting problems. Comparing pre- to post-intervention levels describes the outcome. The model defines progress as change in psychological processes used in response to the problem, i.e., a growing ability to deal with problematic issues non-defensively, manifested differently depending on change stages. At early stages, progress is an increased ability to face the problem rather than turning away. At later stages, progress involves naming, understanding and successfully addressing the problem. This approach provides a broader developmental context compared to exclusively symptom, problem-, or behavior-focused approaches that typically inform outcome measurement in interpersonally based interventions. The authors found the results from the assimilation measure to be consistent with the impact described by the intervention recipients, whereas results based on conventional outcome measures previously applied to the same intervention have missed the change reported by the participants. The proposed cross-disciplinary application is comprehensive; it includes a theoretical model, an empirically derived assessment tool, and a rating procedure. A further asset of this study is the large qualitative dataset, reflecting perspectives on the same intervention from several different types of participants and stakeholders; this lends further confidence in the results and conclusions. The authors present their method, data, and results in detail, discuss how the different results across methods reflect the differences in conceptualizing the change process and in defining progress, and share multiple examples from data to allow generalizing the proposed approach to various other settings where objectively evaluating change in subjective experience of problems is of interest.

Applications

This study propose the assimilation model and assimilation measure as a working approach to the challenge of evaluating change in interpersonally based interventions.

Terminology

Assimilation is change in people’s perceptions of individually problematic (i.e., distressing, uncomfortable) experiences. In psychotherapy research, assimilation underlies therapeutic progress, independently of the presenting problem, setting, client population, or intervention type. In successful interventions, participants gradually assimilate the initially problematic experiences by learning better strategies of addressing them. Assimilation progress occurs through a regular, empirically grounded developmental sequence of levels. It proceeds from extreme to lesser avoidance, to increasing awareness of the problematic experience, to clarifying the problematic contents, developing a detailed understanding, then applying it to work through and gradually resolve the impact of the problem on daily life, and eventually integrating the formerly problematic content into a repertoire of coping resources available to deal with future challenges.

Peer review

This interesting manuscript describes an exciting organizational developmental process in details with several quotations and vignettes and introduces a scientific method for the detection of the organizational changes. The paper describes the change process of a Veterans Affairs Medical Center during an intervention done by organization development psychologists. The paper uses the technique of qualitative interviews to collect the pre and post data and a new Assimilation of Problematic Experiences Scale grounded in developmental psychology to evaluate the interviews. The most interesting finding is that the APES scale is apparently superior in describing the change process compared to classical quantitative measures on subjective stress or work environment. The paper has implications not only for organizational change but also for individual change during psychotherapy. The merits of the manuscript include the interesting topic, the quality of scientific methods used, the logical structure, the perfect English, and others.