Observational Study Open Access
Copyright ©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Psychiatry. Jul 19, 2022; 12(7): 929-943
Published online Jul 19, 2022. doi: 10.5498/wjp.v12.i7.929
Believing processes during the COVID-19 pandemic in individuals with bipolar disorder: An exploratory study
Sophie Tietz, Christof Körner, Institute of Psychology, University of Graz, Graz 8010, Austria
Sophie Tietz, Michaela Ratzenhofer, Frederike T Fellendorf, Eva Fleischmann, Eva Z Reininghaus, Nina Dalkner, Department of Psychiatry and Psychotherapeutic Medicine, Medical University of Graz, Graz 8036, Austria
Jolana Wagner-Skacel, Department of Medical Psychology and Psychotherapy, Medical University of Graz, Graz 8036, Austria
Hans-Ferdinand Angel, Department of Catechetics and Religious Education, University of Graz, Graz 8010, Austria
Rüdiger J Seitz, Department of Neurology, Centre of Neurology and Neuropsychiatry Heinrich-Heine-University Düsseldorf, Medical Faculty, Düsseldorf D-40629, Germany
ORCID number: Sophie Tietz (0000-0002-3784-4368); Jolana Wagner-Skacel (0000-0003-0771-4543); Hans-Ferdinand Angel (0000-0002-3994-2092); Michaela Ratzenhofer (0000-0003-4492-7281); Frederike T Fellendorf (0000-0001-7215-3848); Eva Fleischmann (0000-0003-0459-6553); Christof Körner (0000-0001-2153-9003); Eva Z Reininghaus (0000-0001-5964-4087); Rüdiger J Seitz (0000-0002-1168-9187); Nina Dalkner (0000-0001-7716-3674).
Author contributions: Tietz S and Fleischmann E wrote the first draft of the manuscript; Dalkner N supervised the study procedure; Wagner-Skacel J, Angel H-F, Ratzenhofer M, Fellendorf FT, Körner C, Reininghaus EZ, Seitz RJ and Dalkner N edited the manuscript and gave important intellectual input.
Institutional review board statement: The study was reviewed and approved by the local ethics committee in accordance with the current revision of the Declaration of Helsinki, ICH guideline for Good Clinical Practice and current regulations (Medical University of Graz, Austria; individuals with BD were from the BIPLONG study, EK-number: 25-335 ex 12/13; data was collected in the course of a new study, EK number: 32-363 ex 19/20).
Informed consent statement: All study participants provided informed consent prior to study enrollment.
Conflict-of-interest statement: All the authors declare no conflict of interest for this article.
Data sharing statement: No additional data are available.
STROBE statement: The authors have read the STROBE guidelines and the manuscript was prepared and revised according to the STROBE guidelines.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Nina Dalkner, MSc, PhD, Senior Scientist, Department of Psychiatry and Psychotherapeutic Medicine, Medical University of Graz, Auenbruggerplatz 31, Graz 8036, Austria. nina.dalkner@medunigraz.at
Received: December 20, 2021
Peer-review started: December 20, 2021
First decision: March 13, 2022
Revised: March 27, 2022
Accepted: June 23, 2022
Article in press: June 23, 2022
Published online: July 19, 2022

Abstract
BACKGROUND

Believing or “credition” refers to psychological processes that integrate the cognitions and emotions that influence our behavior. In the credition model by Angel and Seitz, four parameters are postulated: proposition, certainty, emotion and mightiness. It is assumed that believing processes are influenced by both the individual as well as socio-cultural factors and external circumstances. External or environmental circumstances can include threatening situations such as the ongoing pandemic. It has been hypothesized that believing processes related to the pandemic differ between individuals with bipolar disorder (BD) and healthy controls (HC).

AIM

To investigate credition in individuals with BD during the coronavirus disease 2019 (COVID-19) pandemic.

METHODS

Psychiatrically stable individuals with BD (n = 52) and age- and sex matched HC (n = 52) participated in an online survey during the first lockdown of the COVID-19 pandemic. The survey took place between April 9th and June 4th, 2020, in Austria. Participants completed the Brief Symptom Inventory-18, the Beck Depression Inventory-II, the Altman Self-Rating Mania Scale, the Pittsburgh Sleep Quality Index and a dedicated Believing Questionnaire assessing four parameters of credition (proposition, certainty, emotion and mightiness). The MAXQDA software was used to analyze the qualitative data. Statistical analyses included analyses of variance, a multivariate analysis of variance and a multivariate analysis of co-variance.

RESULTS

Individuals with BD reported significantly more negative propositions [F (1,102) = 8.89, P = 0.004, η2p = 0.08] and negative emotions [Welch´s F (1,82.46) = 18.23, P < 0.001, η2p = 0.18], while HC showed significantly more positive propositions [F (1,102) = 7.78, P = 0.006, η2p = 0.07] and emotions [F (1,102) = 14.31, P < 0.001, η2p = 0.12]. In addition, individuals with BD showed a higher incongruence between their propositions and their emotions [F (1,102) = 9.42, P = 0.003, η2p = 0.08] and showed strong correlations between the parameters of the Believing Questionnaire and their psychiatric symptoms (r = 0.51-0.77, all P < 0.001). Positive as well as negative emotions and propositions were associated with scores measuring symptoms of depression, anxiety and sleep quality.

CONCLUSION

Believing parameters were associated with psychiatric symptoms in BD during the pandemic. Findings broaden knowledge about the susceptibility of believing processes for ambient challenges in individuals with BD.

Key Words: COVID-19, Bipolar disorder, Cognition, Emotions, Judgement, Evaluation study

Core Tip: Research concerning believing processes (“creditions”) in individuals with bipolar disorder (BD) during the coronavirus disease 2019 pandemic showed that patients reported more negative emotions and propositions than healthy controls who reported more positive emotions and propositions. Individuals with BD had a higher incongruence between their propositions and their emotions and strong correlations between the parameters of the Believing Questionnaire and psychiatric symptoms. These findings provide insight into the attitudes and beliefs of people with BD during a crisis.



INTRODUCTION

Believing is a fundamental cognitive process involving belief formation, updating and evaluation[1,2]. Importantly, beliefs determine an individual’s behavior by allowing predictions of future events[3]. In the past century, believing has widely been neglected as an object of scientific interest since it was associated with spirituality or considered as abnormal[4]. Recently, however, there is growing interest in the processes of believing and beliefs in evolutionary biology, cognitive neuroscience, psychology and psychiatry[5]. This is corroborated by neuroimaging evidence revealing underlying neural correlates of believing[6-10]. To contextualize this research within the realm of cognition and emotion, the term “credition” is derived from the Latin word “credere” (which means “to believe”). This term highlights psychodynamic activities which underpin the believing processes resulting in stable but still modifiable states of belief[4].

Creditions are understood as dynamic processes that can activate at any time and influence existing states of belief which were constructed by a person's prior experiences[4]. Accordingly, creditions are an important part of our lives as they influence our thinking, feeling and acting and vice versa[11]. Sacks and Hirsch[12] postulated that people tend to accept something as reality until they are proven wrong and that belief formation can be understood as the result of perceptual and affective information processing. Supporting this notion, prior work demonstrates that integration of cognition and emotion occurs in the lateral prefrontal cortex[13]. In the credition model, four characteristic parameters are differentiated: proposition, certainty, emotion and mightiness[1]. “Proposition” represents the content of the statement. “Certainty” reflects the person's inclination to believe the proposition. “Emotion” reflects the affective valence of the proposition for a person. “Mightiness” reflects the degree of relevance of the proposition. It is assumed that believing processes are influenced by the individual themself as well as by socio-cultural factors and external circumstances[14]. Such external or environmental circumstances can include threatening situations including the ongoing pandemic.

The coronavirus disease 2019 (COVID-19) pandemic has deeply influenced the lives of the global population. Caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), it was labelled as a pandemic by the WHO on March 11th, 2020[15]. Since its outbreak, the virus has infected over 274 million and claimed the lives of more than five million people (December 2021)[16]. To contain the virus spread, several periods of lockdown have been implemented across the world contributing to far-reaching effects of the pandemic on the environment[17], economy, social life[18] and mental health[19]. The psychological consequences have been numerous[20], particularly for individuals with pre-existing psychiatric disorders including affective disorders[21].

Bipolar disorder (BD) is a neuropsychiatric affective disorder characterized by severe changes in mood ranging from depression to mania. Typically emerging in early adulthood, bipolar spectrum disorders have a prevalence of 1% to 6%[22,23]. Stressful events and exposure to life stress increase the recurrence of affective episodes in BD[24], making individuals with BD a vulnerable group during the COVID-19 pandemic[25]. Increased depressive symptoms, fatigue[26], psychological distress[27,28], post-traumatic stress symptoms[29], fear and sleeping problems[29,30] have all been shown in this population during this period. An increase in subjective cognitive dysfunction has also been found in this population and was associated with negative symptoms and poor quality of life[31].

In Beck’s cognitive model of depression[32], dysfunctional cognitive schemata are assumed to be the basis for the development of a depressive episode. This leads to a cognitive bias in information processing as attention is selectively directed towards negative aspects and experiences while positive events and memories are blocked[32,33]. Regarding the COVID-19 pandemic, the specific believing processes possibly contributing to the development of a depressive episode remain largely unknown.

The aims of the current study were: (1) To analyze believing processes, in particular the four parameters of proposition, certainty, emotion and mightiness in individuals with BD during the COVID-19 pandemic compared with healthy controls (HC); and (2) to investigate correlations between these parameters and psychiatric symptoms in BD.

Due to the still lacking empirical evidence in this field, this study utilized an exploratory approach. However, based on the literature, it was expected that in accordance with Beck's cognitive theory of depression[32], individuals with BD would show more negative propositions and emotions in their verbalized believing processes than HC. Further, it was expected that HC would show more positive propositions and emotions in their verbalized believing processes compared to individuals with BD. Additionally, it was hypothesized that current psychiatric symptoms would be related to possible differences in parameters between the two groups.

MATERIALS AND METHODS
Participants

In total, 260 individuals were recruited; 208 of these had complete data sets. After matching for sex and age, the final sample was comprised of 52 stable, medicated individuals diagnosed with BD (29 males and 23 females) and 52 HC. The individuals with BD were previously diagnosed at the outpatient center for BD at the Medical University of Graz, Department of Psychiatry and Psychotherapeutic Medicine, using the Structured Clinical Interview for DSM-IV[34]. Exclusion criteria for HC were any psychiatric diagnosis, taking psychiatric medication or first-degree relatives with psychiatric disorders. Inclusion criteria for all participants were voluntary participation and e-mail access. All participants with BD and the majority of HC were recruited from the pool of the ongoing BIPLONG study which assesses lifestyle, metabolism and cognitive function in individuals with BD compared to HC. Additional HC were recruited by word of mouth and social media.

Procedure

An online survey was sent out via the survey tool LimeSurvey (Version 3.27.4, Limesurvey GmbH) between April 9th and June 4th, 2020, starting 3 wk after the beginning of the first lockdown in Austria. Participants were thus experiencing travel restrictions, social distancing measures, and the closure of institutions such as schools, leisure venues and nonessential shops while completing measures for this study.

Participants gave informed consent before pseudo-anonymously responding to questionnaires. This study was approved by the local ethics committee in accordance with the current revision of the Declaration of Helsinki, ICH guideline for Good Clinical Practice and current regulations (Medical University of Graz, Austria; individuals with BD were from the BIPLONG study, EK-number: 25-335 ex 12/13; data was collected in the course of a new study, EK number: 32-363 ex 19/20).

Psychological inventories

Analyses in the current study were conducted on the following psychological inventories:

The Brief Symptom Inventory-18 (BSI-18) was constructed by Derogatis and Fitzpatrick[35], a short version of the Symptom-Checklist-90-Revised (SCL-90-R) by Derogatis and Savitz[36]. The BSI-18 was used to measure psychological symptoms during the last week. This measure yields a Global Severity Index (GSI) and three subscales: anxiety, depression and somatization, each with acceptable internal consistency (Cronbach’s alpha: GSI α = 0.93, anxiety α = 0.84, depression α = 0.87, and somatization α = 0.82).

The Beck Depression Inventory (BDI-II) by Beck et al[37] assessed the severity of depressive symptoms within the last week with a score of 18 or higher indicating clinically relevant depression. Assessed quality criteria were Cronbach’s alpha (α ≥ 0.84) and reliability (r ≥ 0.75)[38].

The Altman Self-Rating Mania Scale (ASRM) is a 5-item questionnaire that determines the extent of manic symptoms in the course of 1 wk[39]. Assessing self-confidence, mood, speech, activity level and need to sleep, each item is rated on a five-point scale (0-4). A score of five or more is considered clinically relevant.

The Pittsburgh Sleep Quality Index (PSQI) was constructed by Buysse et al[40] and measures sleep quality in the last month. The 19 items constitute seven components: subjective sleep quality, sleep latency, sleep duration, sleep efficiency, sleep disturbances, use of sleep medication and daytime sleepiness. The sum score has a range of 0-21, with a higher score indicating worse sleep quality. A total score > 5 has diagnostic sensitivity of 89.6% and specificity of 86.5% (κ = 0.75, P < 0.001) when differentiating between poor and good sleepers.

The Believing Questionnaire (BQ) was created by JWS and ND of the Department of Psychiatry and Psychotherapeutic Medicine in consultation with HFA and RS. The BQ assessed the characteristic credition parameters (proposition, certainty, emotion, mightiness) during the COVID-19 pandemic. Consisting of six open-ended questions, the BQ was developed to gain insight in believing processes during such a precarious and challenging situation:

When I think of the current, very special situation, I believe;

When I think of my body, I believe;

When I think of my mental/emotional situation, I believe;

When I think of the coronavirus, I believe;

When I think of what the future holds 3 mo from now, I believe;

When I think of what the future holds 6 mo from now, I believe.

Participants were asked to answer as spontaneously and honestly as possible. Each question was then rated on a scale from 0 (= not sure) to 100 (= very sure) regarding the certainty of belief. Additionally, participants were asked to identify the emotion that arose for them in the context of that belief using an “Emotion Wheel” consisting of three concentric circles. The innermost circle showed the six basic emotions according to Ekman[41]: fear, anger, joy, sadness, disgust and surprise. The surrounding two circles provided options to further differentiate the basic emotion. The intensity of the emotion (sense of mightiness) was rated on a scale from 0 (= not sure) to 100 (= very sure) as well.

Statistical analysis

To analyze the qualitative data of the BQ in a standardized and transparent manner, MAXQDA 2020[42] software was used; this has previously proven useful in research of BD in clinical settings[43,44]. The process of analysis consisted of two independent raters categorizing individuals’ propositions and emotions into the three categories positive, negative and indifferent, generating six codes (positive propositions, negative propositions, indifferent propositions, positive emotions, negative emotions and indifferent emotions). The interrater reliability of κ = 0.92 was satisfactory.

To make data suitable for analysis with the Statistical Package for Social Sciences (SPSS version 26, IBM), six new variables were calculated to reflect the frequency of each code. Additionally, a variable reflecting the frequency of incongruencies between the valence of a person’s proposition and the identified emotion was created.

A multivariate analysis of variance (MANOVA) with group (BD vs HC) as the independent variable was calculated to test for between-subject differences in the credition parameters. Positive propositions, negative propositions and positive emotions were included in the MANOVA as variables because they were moderately intercorrelated (r = 0.64-0.78, all P < 0.001). Not all credition parameters were intercorrelated; as such, single ANOVAs were utilized for indifferent propositions, negative emotions, indifferent emotions, incongruence, certainty and mightiness. A multivariate analysis of co-variance with the same design but controlling for psychiatric symptomatology was then conducted. The psychiatric symptomatology consisted of the individual total scores in the BSI-18 (GSI), BDI-II and PSQI. The total score of the ASRM was not included because the two groups did not differ in their manic symptomatology. Spearman correlation analyses were used to test for associations between credition parameters and psychological test scores (BSI-18, BDI-II, ASRM and PSQI). Bonferroni correction (P < 0.003) was used to correct for conducting multiple tests. To test for normal distribution for the psychological test scores and credition parameters of both groups, Shapiro-Wilk tests were run and skewness and kurtosis were calculated. All data met the assumed criteria of linearity. The criterion of normality was not met for several variables (sex, negative propositions, positive emotions, negative emotions, indifferent emotions, and certainty) however, the sample size was adequate (n ≥ 30) and thus, normal distribution could be assumed, according to the central limit theorem. The criterion of variance was also not met for several variables (GSI, BDI-II, PSQI, negative propositions, positive emotions, negative emotions). However, the analyses were continued given that MANOVAs are relatively robust to violations of equality of variance[45] and for ANOVAs the Welch-ANOVA could be interpreted[46,47].

In addition, word clouds in MAXQDA were used to present propositions and emotions for each item of the BQ. Word clouds are a useful method to simultaneously visualize the actual words as well as their frequency[48]. The word clouds show the most frequently used words for each BQ item. Prepositions and conjunctions were ignored and added to a stop list in MAXQDA. The word clouds were translated from German into English for the present paper. It is noted that a loss of information may occur due to translation.

RESULTS
Sample characteristics

Each group consisted of 29 males and 23 females. Mean age was 50.2 years (individuals with BD; SD = 14.5) and 49.0 years (HC; SD = 13.3). The individuals with BD showed higher scores of psychiatric symptoms than the HC group [F (3,100) = 11.53, P < 0.001, Roy's Largest Root = 0.346, η2p = 0.257; see Table 1 for statistics].

Table 1 Differences in the psychiatric symptoms and the credition parameters between the bipolar individuals and controls.

Bipolar
Control
F
P value
η2p
M
SD
M
SD
BSI-1813.21 13.45 3.83 4.23 23.05< 0.0010.18
BDI-II12.0810.92 2.77 3.54 34.22< 0.0010.25
ASRM1.52 2.73 0.94 1.60 1.731,20.192
PSQI7.08 4.26 4.12 2.34 19.30< 0.0010.16
Positive propositions3.15 1.80 4.08 1.57 7.78 0.0060.07
Negative propositions1.63 1.70 0.811.10 8.89 0.0040.08
Indifferent propositions1.23 1.17 1.12 1.00 0.290.590
Positive emotions3.10 2.31 4.54 1.49 14.31< 0.0010.12
Negative emotions2.65 2.34 1.06 1.34 18.232,3< 0.0010.18
Indifferent emotions0.25 0.650.380.75 0.960.329
Incongruence42.13 1.39 1.38 1.09 9.42 0.0030.08
Certainty580.4915.6786.3510.44 5.030.0270.05
Mightiness578.11 13.9177.2015.980.100.757
Differences in believing parameters between the individuals with BD and HC

The MANOVA testing for group differences in the believing parameters showed significant differences for the combined dependent variables of positive propositions, negative propositions and positive emotions [F (3,100) = 4.93, P = 0.003, η2p = 0.13, Roy´s Largest Root = 0.15]. Specifically, the HC group showed more positive propositions and emotions in their verbal believing processes than the individuals with BD. In contrast, the individuals with BD revealed more negative propositions and emotions compared with HC (Table 1). The individuals with BD showed a stronger incongruence between the valence of their propositions and the valence of their emotions. Furthermore, HC were more certain about their propositions than were individuals with BD. However, this difference did not remain significant after Bonferroni correction. No statistically significant differences emerged between the two groups in indifferent propositions and emotions or in mightiness.

After controlling for psychiatric symptoms (GSI score of the BSI-18, total scores of BDI-II and PSQI), the group differences in credition parameters were no longer significant.

Word clouds

Figures 1-3 show word clouds for the six BQ items. Results of item 2 (“When I think about my body, I believe”) and item 3 (“When I think about my mental/emotional situation, I believe”) are presented in more detail because there were significant differences between the two groups for both the valence of propositions [item 2: χ2(2) = 12.45, P = 0.002, Cramér’s V = 0.22 item 3: χ2(2) = 8.03, = 0.018, Cramér’s V = 0.27] and the valence of emotions [item 2: χ2(2) = 10.44, P = 0.004, Cramér’s V = 0.31; item 3: χ2(2) = 9.61, P = 0.005, Cramér’s V = 0.30]. For item 2, individuals with BD used a total of 352 words and the HC 273 words. For item 3, individuals with BD used a total of 486 words and the HC 292 words. The word frequencies of the different words for the two items are shown in Tables 2 and 3.

Figure 1
Figure 1 Word clouds showing the most frequently used words of individuals with bipolar disorder and healthy controls for items 2 and 3 concerning their beliefs during the first lockdown of the COVID-19 pandemic.
Figure 2
Figure 2 Word clouds showing the most frequently used words of individuals with bipolar disorder and healthy controls for items 1 and 4 concerning their beliefs during the first lockdown of the COVID-19 pandemic.
Figure 3
Figure 3 Word clouds showing the most frequently used words of individuals with bipolar disorder and healthy controls for items 5 and 6 concerning their beliefs during the first lockdown of the COVID-19 pandemic.
Table 2 Word frequencies for item 2 (“When I think about my body, I believe”) in the Believing Questionnaire for the two groups.
BD
HC
Word
Frequency
%1
Word
Frequency
%1
I318.8 I2810.3
Optimistic82.3 Good103.7
More72.0 Content82.9
My72.0 Body62.2
Not61.7 Everything62.2
Should61.7 Healthy62.2
Content51.4 More62.2
Could51.4 Exercise51.8
Happy51.4 My51.8
Much51.4 Optimistic51.8
Thick51.42 Fit41.47
Better41.14 Fitter41.47
Expectant41.14 Myself41.47
Good41.14 Should41.47
Age30.85 Unconcerned41.47
Always30.85 Worried41.47
Confident30.85 Calm31.10
Fearful30.85 Could31.10
Fits30.85 Fits31.10
Make30.85 Self-confident31.10
Me30.85 Sports31.10
Myself30.85
Okay30.85
Quite30.85
Unhappy30.85
Very30.85
Yr30.85
Table 3 Word frequencies of item 3 (“When I think about my mental/emotional situation, I believe”) in the Believing Questionnaire for the two groups.
BD
HC
Word
Frequency
%1
Word
Frequency
%1
I5511.3 I3211.0
Good122.5 Content134.5
Me112.3 Good113.8
Myself112.3Balanced72.4
Content102.1 Calm72.4
Not81.7 Me72.4
Better71.4 Myself72.4
Everything71.4 Optimistic72.4
Completely61.2 Stable72.4
Happy61.2Very62.1
My61.23 Reassured51.71
Like51.03 Carefree41.37
Can40.82 Everything41.37
Confident40.82 Expectant41.37
Depressed40.82 Happy41.37
Much40.82 Not41.37
Optimistic40.82 Better31.03
Sad40.82 Confident31.03
Stable40.82 Psychologically31.03
Balanced30.62 Self-confident31.03
Currently30.62 Strong31.03
Feel30.62
Now30.62
Room30.62
Still30.62
Time30.62

For item 2 (Figure 2, Table 2), both groups used the word “I” most often, followed by positive emotional words. The HC predominantly used positive words about their body, such as “fit”, “fitter” and “healthy”. In contrast, the individuals with BD, predominantly used the word “fat” for their body. In addition, the individuals with BD often used negative emotion words such as “fearful” and “unhappy”. For item 3 (Figure 2, Table 3), both groups again used the word “I” most frequently. However, the individuals with BD more frequently utilized self-centered words, such as “me” and “myself”, followed by positive emotion words, such as “good” and “content”. The HC used positive emotion words most often, such as “content”, “good”, “balanced” and “calm”.

For item 4 (“When I think of the coronavirus, I believe") and item 6 (“When I think of what the future holds 6 mo from now, I believe"), it was notable that individuals with BD used the word “I” and the HC the word “We” most frequently (see Figures 1 and 3).

Correlations between believing parameters and psychiatric symptoms

Spearman correlation analyses between the believing parameters and psychiatric symptoms showed significant correlations in individuals with BD (P < 0.001 after Bonferroni correction; Table 4) for both positive as well as negative propositions and psychiatric symptoms. Specifically, there were significant negative correlations between the positive propositions and emotions and the BSI-18 scales GSI, depression and anxiety, the BDI-II and the PSQI sum score. Positive correlations were found between the negative emotions and the BSI-18 scales GSI, depression and anxiety, the BDI-II and the PSQI sum score. Additionally, a negative correlation was found between negative propositions and depression while GSI, the BDI-II and the PSQI sum score were positively correlated. No correlations were found with the scores in the ASRM.

Table 4 Bipolar group: Bonferroni adjusted Spearman correlations between the parameters of the Believing Questionnaire and the psychiatric symptoms.

BSI-18
Somatization2
Depression2
Anxiety2
BDI-II
ASRM
PSQI
Positive propositions-0.66c-0.38b -0.72c-0.56c -0.56c0.20 -0.51c
Negative propositions0.54c0.38b -0.60c0.42b 0.63c-0.21 0.54c
Indifferent propositions0.28a0.11 0.25 0.32a -0.05 -0.02 0.06
Positive emotions-0.70c -0.47b -0.71c-0.62c-0.79c0.09 -0.59c
Negative emotions0.66c0.44b 0.68c0.58c0.77c -0.14 0.62c
Indifferent emotions0.07 -0.04 -0.04 0.19 -0.10 0.10 -0.22
Incongruence10.28a 0.18 0.31a 0.18 0.20 -0.16 0.17
Certainty-0.20 -0.21 -0.17 -0.12 -0.17 0.03 -0.16
Mightiness0.03 -0.05 0.08 0.07 0.09 0.05 0.08

In HC, there was only one negative correlation between certainty about the proposition and the BSI-18 GSI score (r = -0.48, P < 0.001, after Bonferroni correction).

Correlations between the believing parameters in the Believing Questionnaire

Spearman correlation analyses were used to examine the extent to which the believing parameters depend on each other (Tables 5 and 6). The analyses were calculated for both groups separately. After Bonferroni correction, both groups showed significant correlations between propositions and emotions with a positive and negative valence. There was another significant correlation between certainty and mightiness. Furthermore, the controls showed significant correlations between the positive and negative propositions and incongruence.

Table 5 Bipolar group: Bonferroni adjusted Spearman correlations between the parameters of the Believing Questionnaire.

Pos. Pro.
Neg. Pro.
Ind. Pro.
Pos. Emo.
Neg. Emo.
Ind. Emo.
Incongruence1
Certainty
Mightiness
Pos. Pro.
Neg. Pro.-0.78c
Ind. Pro.-0.44b-0.15
Pos. Emo.0.67c-0.61c-0.10
Neg. Emo.-0.68c0.63c0.10-0.96c
Ind. Emo.0.00-0.140.19-0.09 -0.11
Incongruence-0.37b0.040.55c-0.31a0.29a0.11
Certainty0.23-0.190.040.26-0.26 -0.04 -0.07
Mightiness-0.050.120.01-0.01 0.02-0.18 -0.04 0.66c
Table 6 Control group: Bonferroni adjusted Spearman correlations between the parameters of the Believing Questionnaire.

Pos. Pro.
Neg. Pro.
Ind. Pro.
Pos. Emo.
Neg. Emo.
Ind. Emo.
Incongruence1
Certainty
Mightiness
Pos. Pro.
Neg. Pro.-0.74c
Ind. Pro.-0.77c0.16
Pos. Emo.0.61c-0.40b-0.50c
Neg. Emo.-0.56c0.52c0.35a-0.86c
Ind. Emo.-0.20 -0.060.33a-0.51c0.11
Incongruence-0.58c0.33a0.55c-0.34a0.44b0.01
Certainty0.110.01-0.10 0.12-0.13 -.12 -0.18
Mightiness0.26-0.12 -0.22 0.26-0.19 -0.15 -0.16 0.60c
DISCUSSION

In this study, creditions of individuals with BD were investigated using questionnaires tapping into beliefs and believing processes during the first wave of the COVID-19 pandemic. Findings showed that individuals with BD differed from controls in the believing parameters of propositions and emotions. Results confirmed our hypothesis that individuals with BD would show more negative propositions and emotions in their verbalized believing processes than HC. This corresponds to Beck's cognitive theory about negative dysfunctional cognitive schemata in depressive disorders[32]. Accordingly, a cognitive bias in information processing renders depressive individuals likelier to focus their attention more on negative aspects of life and to block positive aspects. This change in perception can result in negative believing processes. In addition, we found that individuals with BD showed greater incongruence between the valence of their propositions and the valence of their emotions compared to HC. Carl Rogers[49] has suggested incongruence as the root cause for the development of mental disorders. According to this concept, the actual experience does not match one´s own self-image. Recently, it was proposed that incongruence stems from a mismatch between the internal and external experiences and a person's self-concept, resulting in a state of tension[50]. The high incongruence we found in individuals with BD could therefore reflect an experiential incongruence within the individuals with BD themselves.

Furthermore, we observed that believing parameters were strongly related to psychiatric symptoms in the bipolar group. The correlations were particularly strong between the propositions or emotions and the total scores in the BSI-18, BDI-II and PSQI. The believing parameters related to propositions and emotions were more negative as psychiatric symptoms were more severe and more positive as psychiatric symptoms were less severe. In contrast, only a moderate negative association between GSI and the believing parameter of certainty was observed. One possible explanation for the fact that psychiatric symptoms and believing parameters were highly correlated in individuals with BD could be that individuals with BD generally experienced more psychiatric symptoms during the COVID-19 pandemic. Another possible explanation is that individuals with BD ruminate more on emotional experiences, both negative and positive[51-53]. Consequently, cognitions and emotions appear to have a profound impact on mental health in individuals with BD.

An interesting result of this study was that after controlling for psychiatric symptoms, i.e. total scores in BSI-18, BDI-II and PSQI, the differences in believing parameters between the two groups disappeared. This finding suggests that believing processes reflect important aspects of life that are also represented by the questionnaire for psychological symptoms. This could be explained by an intrinsic modulator function that is accounted for by the credition model[1]. On the molecular level, the modulator function may be linked to the dopamine system which plays an important role in believing processes[54,55] as well as for abnormal believing processes in psychiatric disorders[56-58].

The word clouds we created showed that the individuals with BD used words of emotion with negative connotations more often compared to the control group. Moreover, those with BD used more self-centered language than HC. We, therefore, assume that individuals with BD tend to consider the self in the focus of their believing processes more often than the HC. Interestingly, individuals with BD more frequently answered the two items “When I think of the coronavirus, I believe” and “When I think of what the future holds 6 mo from now, I believe” from an individual perspective using the word “I”, whereas HC more often answered from a group perspective with “We”. Perhaps individuals with BD depend more upon the self in the context of coping while healthy individuals are more likely to refer to social reasoning. Another explanation for this finding is that individuals with BD could be the disorder itself and the associated introspection processes and self-awareness, possibly learned in psychotherapy, as all patients were treated at the outpatient center for BD at the Medical University of Graz.

Limitations

The present study had several limitations. Due to the lockdown in Austria at the time of study, testing was limited to online questionnaires precluding face-to-face interactions with study participants. Nevertheless, scores from self-report did allow us to capture and control for current symptoms. Another potential problem of this, as well as other online studies, is that of sampling bias. Only data from individuals who were motivated to participate in the survey were collected. Thus, results may not apply to the general population. Furthermore, believing processes themselves could not be studied, as only the verbalized expressions could be directly assessed. It may be inferred that believing processes were influenced by the introspective ability of the subjects; however, introspective ability was not measured in the present study. A further limitation of the current study is that the qualitative data of the BQ had to be transformed into positive, negative and indifferent categories, that is, the data were reduced profoundly and may thus miss some important information. Lastly, as this was a cross-sectional study, causality cannot be determined.

Practical Implications

Our findings offer practical implications for the treatment of individuals with BD. Because differences in believing parameters were no longer present after controlling for psychiatric symptoms, the credition model could contribute to a better understanding of negative or even positive dysfunctional schemata in psychiatric individuals, as well as be adopted into the psychotherapeutic context. Since humans tend to believe their perceptions to be true and beliefs can be updated by confirming or disproving new evidence[57], dysfunctional cognitive schemata could presumably be changed if they were revealed. It is hypothesized that belief evaluation is one approach to help with this and could be used as a tool for cognitive treatment approaches in psychotherapy.

CONCLUSION

The present study showed that the model of credition is applicable in the clinical context regarding the postulated believing processes. Individuals with BD differed in their believing processes regarding the COVID-19 pandemic from healthy persons. Thus, this study provides a deep insight into the attitudes and beliefs of particularly vulnerable people during a global crisis. Believing parameters should be examined in other clinical groups in future studies.

ARTICLE HIGHLIGHTS
Research background

Believing, or “credition,” refers to psychological processes that integrate the cognitions and emotions influencing our behavior. Angel and Seitz created a model consisting of four credition parameters: proposition, certainty, emotion and mightiness. Believing processes are postulated to be influenced by external or environmental circumstances, such as the coronavirus disease 2019 (COVID-19) pandemic.

Research motivation

As empirical evidence about believing processes is lacking, studies examining this field of research are needed. Investigating credition during a crisis, such as the COVID-19 pandemic, will hopefully provide valuable insight into the mind of individuals with bipolar disorder (BD) and might be able to offer implications for treatment.

Research objectives

The purpose of this study was to explore credition in individuals with BD as well as healthy controls (HC) during the COVID-19 pandemic.

Research methods

Euthymic individuals with BD (n = 52) and age- and sex matched HC (n = 52) from Austria participated in an online survey taking place from April 9th to June 4th, 2020. The following questionnaires were completed: Brief Symptom Inventory-18, Beck Depression Inventory-II, Altman Self-Rating Mania Scale, Pittsburgh Sleep Quality Index and a Believing Questionnaire assessing four parameters of credition (proposition, certainty, emotion and mightiness). The MAXQDA software was used to analyze data about believing processes. Statistical analyses included analyses of variance, a multivariate analysis of variance and a multivariate analysis of co-variance.

Research results

Individuals with BD showed significantly more negative propositions and negative emotions, whereas HC reported significantly more positive propositions and emotions. Moreover, individuals with BD showed a higher incongruence between their propositions and emotions. Positive as well as negative emotions and propositions were associated with scores measuring symptoms of depression, anxiety and sleep quality.

Research conclusions

During the COVID-19 pandemic, believing parameters were associated with psychiatric symptoms in BD and differed from HC. Results demonstrate the sensitivity of believing processes to external influences in individuals with BD.

Research perspectives

Believing processes should be further examined in future studies, especially regarding cognitive treatment approaches in psychotherapy.

ACKNOWLEDGEMENTS

A sincere thank you to Ms. Nina Bonkat and Ms. Mary Beth Spitznagel for their diligent proofreading of this paper.

Footnotes

Provenance and peer review: Invited article; Externally peer reviewed.

Peer-review model: Single blind

Corresponding Author's Membership in Professional Societies: INGE St. Initiative Gehirnforschung Steiermark; and Berufsverband Österreichischer Psychologlnnen.

Specialty type: Psychiatry

Country/Territory of origin: Austria

Peer-review report’s scientific quality classification

Grade A (Excellent): A

Grade B (Very good): B

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): E

P-Reviewer: Apiratwarakul K, Thailand; Munteanu C, Romania; Sivanand N, India; Tazegul G, Turkey S-Editor: Gong ZM L-Editor: Filipodia P-Editor: Gong ZM

References
1.  Angel HS, Seitz RJ. Process of believing as fundamental brain function: the concept of Credition. SFU Research Bulletin. 2016;1:1-20.  [PubMed]  [DOI]  [Cited in This Article: ]
2.  Connors MH, Halligan PW.   Belief and belief formation: Insights from delusions. In: Angel H-F, Oviedo RF, Paloutzian ALC, Seitz RJ. Processes of believing: The acquisition, maintenance, and change in creditions. Basel: Springer International Publishing AG, 2017: 153-165.  [PubMed]  [DOI]  [Cited in This Article: ]
3.  Friston KJ, Parr T, de Vries B. The graphical brain: Belief propagation and active inference. Netw Neurosci. 2017;1:381-414.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 171]  [Cited by in F6Publishing: 190]  [Article Influence: 27.1]  [Reference Citation Analysis (0)]
4.  Angel HF  Credition: from the question of belief to the question of believing. In Angel H-F, Oviedo L, Paloutzian RF, Runehov A, Seitz RJ. Processes of believing: the acquisition, maintenance, and change in creditions. New approaches to the scientific study of religion. Basel: Springer International Publishing AG, 2017: 17-36.  [PubMed]  [DOI]  [Cited in This Article: ]
5.  Angel H-F, Oviedo L, Paloutzian RF, Runehov AL, Seitz RJ.   Processes of believing: The acquisition, maintenance, and change in creditions. Basel: Springer International Publishing AG, 2017.  [PubMed]  [DOI]  [Cited in This Article: ]
6.  Cristofori I, Grafman J.   Neural underpinnings of the human belief system. In Angel H-F, Oviedo L, Paloutzian RF, Runehov A, Seitz RJ. Processes of believing: The acquisition, maintenance, and change in creditions. Basel: Springer International Publishing AG, 2017: 111-123.  [PubMed]  [DOI]  [Cited in This Article: ]
7.  Goel V, Dolan RJ. Explaining modulation of reasoning by belief. Cognition. 2003;87:B11-B22.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 274]  [Cited by in F6Publishing: 303]  [Article Influence: 14.4]  [Reference Citation Analysis (0)]
8.  Han X, Zhang T, Wang S, Han S. Neural correlates of believing. Neuroimage. 2017;156:155-165.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 10]  [Cited by in F6Publishing: 11]  [Article Influence: 1.6]  [Reference Citation Analysis (0)]
9.  Seitz RJ, Angel HF. Processes of believing - a review and conceptual account. Rev Neurosci. 2012;23:303-309.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 33]  [Cited by in F6Publishing: 35]  [Article Influence: 2.9]  [Reference Citation Analysis (0)]
10.  Sugiura M, Seitz RJ, Angel H-F. Models and neural bases of the believing process. J Behav Brain Sci. 2015;5:12-23.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 30]  [Cited by in F6Publishing: 33]  [Article Influence: 3.7]  [Reference Citation Analysis (0)]
11.  Paloutzian RF, Mukai K.   Believing, remembering, and imaging: The roots and fruits of meanings made and remade. In Angel H-F, Oviedo L, Paloutzian RF, Runehov A, Seitz RJ. Processes of believing: the Acquisition, maintenance, and change in creditions. Basel: Springer International Publishing AG, 2017: 39-49.  [PubMed]  [DOI]  [Cited in This Article: ]
12.  Sacks O, Hirsch J. A neurology of belief. Ann Neurol. 2008;63:129-130.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 11]  [Cited by in F6Publishing: 12]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
13.  Gray JR, Braver TS, Raichle ME. Integration of emotion and cognition in the lateral prefrontal cortex. Proc Natl Acad Sci U S A. 2002;99:4115-4120.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 503]  [Cited by in F6Publishing: 427]  [Article Influence: 19.4]  [Reference Citation Analysis (0)]
14.  Seitz RJ, Paloutzian RF, Angel HF. From Believing to Belief: A General Theoretical Model. J Cogn Neurosci. 2018;30:1254-1264.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 29]  [Article Influence: 4.8]  [Reference Citation Analysis (0)]
15.  Adhanom T  WHO Director-General’s opening remarks at the media briefing on COVID-19 - 11 March 2020. 2020 March 11 [cited 18 December 2021]. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020.  [PubMed]  [DOI]  [Cited in This Article: ]
16.  Johns-Hopkins University  COVID-19 dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). 18 December 2021 [cited 18 December 2021]. Available from: https://coronavirus.jhu.edu/map.html.  [PubMed]  [DOI]  [Cited in This Article: ]
17.  Rasheed R, Rizwan A, Javed H, Sharif F, Zaidi A. Socio-economic and environmental impacts of COVID-19 pandemic in Pakistan-an integrated analysis. Environ Sci Pollut Res Int. 2021;28:19926-19943.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 46]  [Cited by in F6Publishing: 30]  [Article Influence: 10.0]  [Reference Citation Analysis (0)]
18.  Mofijur M, Fattah IMR, Alam MA, Islam ABMS, Ong HC, Rahman SMA, Najafi G, Ahmed SF, Uddin MA, Mahlia TMI. Impact of COVID-19 on the social, economic, environmental and energy domains: Lessons learnt from a global pandemic. Sustain Prod Consum. 2021;26:343-359.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 281]  [Cited by in F6Publishing: 199]  [Article Influence: 66.3]  [Reference Citation Analysis (0)]
19.  Daly M, Sutin AR, Robinson E. Longitudinal changes in mental health and the COVID-19 pandemic: evidence from the UK Household Longitudinal Study. Psychol Med. 2020;1-10.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 298]  [Cited by in F6Publishing: 334]  [Article Influence: 167.0]  [Reference Citation Analysis (0)]
20.  Zandifar A, Badrfam R. Iranian mental health during the COVID-19 epidemic. Asian J Psychiatr. 2020;51:101990.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 397]  [Cited by in F6Publishing: 340]  [Article Influence: 85.0]  [Reference Citation Analysis (0)]
21.  Quittkat HL, Düsing R, Holtmann FJ, Buhlmann U, Svaldi J, Vocks S. Perceived Impact of Covid-19 Across Different Mental Disorders: A Study on Disorder-Specific Symptoms, Psychosocial Stress and Behavior. Front Psychol. 2020;11:586246.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 44]  [Cited by in F6Publishing: 59]  [Article Influence: 14.8]  [Reference Citation Analysis (0)]
22.  Blanco C, Compton WM, Saha TD, Goldstein BI, Ruan WJ, Huang B, Grant BF. Epidemiology of DSM-5 bipolar I disorder: Results from the National Epidemiologic Survey on Alcohol and Related Conditions - III. J Psychiatr Res. 2017;84:310-317.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 82]  [Cited by in F6Publishing: 79]  [Article Influence: 11.3]  [Reference Citation Analysis (0)]
23.  Merikangas KR, Jin R, He JP, Kessler RC, Lee S, Sampson NA, Viana MC, Andrade LH, Hu C, Karam EG, Ladea M, Medina-Mora ME, Ono Y, Posada-Villa J, Sagar R, Wells JE, Zarkov Z. Prevalence and correlates of bipolar spectrum disorder in the world mental health survey initiative. Arch Gen Psychiatry. 2011;68:241-251.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1693]  [Cited by in F6Publishing: 1560]  [Article Influence: 120.0]  [Reference Citation Analysis (0)]
24.  Weiss RB, Stange JP, Boland EM, Black SK, LaBelle DR, Abramson LY, Alloy LB. Kindling of life stress in bipolar disorder: comparison of sensitization and autonomy models. J Abnorm Psychol. 2015;124:4-16.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 37]  [Cited by in F6Publishing: 40]  [Article Influence: 5.0]  [Reference Citation Analysis (0)]
25.  Stefana A, Youngstrom EA, Chen J, Hinshaw S, Maxwell V, Michalak E, Vieta E. The COVID-19 pandemic is a crisis and opportunity for bipolar disorder. Bipolar Disord. 2020;22:641-643.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 55]  [Cited by in F6Publishing: 55]  [Article Influence: 13.8]  [Reference Citation Analysis (0)]
26.  Campos JADB, Campos LA, Martins BG, Valadão Dias F, Ruano R, Maroco J.   The psychological impact of COVID-19 on individuals with and without mental health disorders. Psychol Rep 2020; 332941211026850.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3]  [Cited by in F6Publishing: 6]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
27.  Di Nicola M, Dattoli L, Moccia L, Pepe M, Janiri D, Fiorillo A, Janiri L, Sani G. Serum 25-hydroxyvitamin D levels and psychological distress symptoms in patients with affective disorders during the COVID-19 pandemic. Psychoneuroendocrinology. 2020;122:104869.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 44]  [Cited by in F6Publishing: 47]  [Article Influence: 11.8]  [Reference Citation Analysis (0)]
28.  Van Rheenen TE, Meyer D, Neill E, Phillipou A, Tan EJ, Toh WL, Rossell SL. Mental health status of individuals with a mood-disorder during the COVID-19 pandemic in Australia: Initial results from the COLLATE project. J Affect Disord. 2020;275:69-77.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 153]  [Cited by in F6Publishing: 136]  [Article Influence: 34.0]  [Reference Citation Analysis (0)]
29.  Asmundson GJG, Paluszek MM, Landry CA, Rachor GS, McKay D, Taylor S. Do pre-existing anxiety-related and mood disorders differentially impact COVID-19 stress responses and coping? J Anxiety Disord. 2020;74:102271.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 312]  [Cited by in F6Publishing: 261]  [Article Influence: 65.3]  [Reference Citation Analysis (0)]
30.  Winkler P, Formanek T, Mlada K, Kagstrom A, Mohrova Z, Mohr P, Csemy L. Increase in prevalence of current mental disorders in the context of COVID-19: analysis of repeated nationwide cross-sectional surveys. Epidemiol Psychiatr Sci. 2020;29:e173.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 126]  [Cited by in F6Publishing: 108]  [Article Influence: 27.0]  [Reference Citation Analysis (0)]
31.  Karantonis JA, Rossell SL, Berk M, Van Rheenen TE. The mental health and lifestyle impacts of COVID-19 on bipolar disorder. J Affect Disord. 2021;282:442-447.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 23]  [Cited by in F6Publishing: 27]  [Article Influence: 9.0]  [Reference Citation Analysis (0)]
32.  Beck AT. The evolution of the cognitive model of depression and its neurobiological correlates. Am J Psychiatry. 2008;165:969-977.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 864]  [Cited by in F6Publishing: 825]  [Article Influence: 51.6]  [Reference Citation Analysis (0)]
33.  Lex C, Hautzinger M, Meyer TD. Cognitive styles in hypomanic episodes of bipolar I disorder. Bipolar Disord. 2011;13:355-364.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 23]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
34.  Wittchen HU, Wunderlich U, Gruschwitz S, Zaudig M.   SCID: Clinical Interview for DSM-IV (German Version). Göttingen: Verlag Für Psychologie, 1997.  [PubMed]  [DOI]  [Cited in This Article: ]
35.  Derogatis LR, Fitzpatrick M.   The SCL-90-R, the Brief Symptom Inventory (BSI), and the BSI-18. In: Maurish ME. The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults. New Jersey: Lawrence Erlbaum Associates Publishers, 2004: 1-41.  [PubMed]  [DOI]  [Cited in This Article: ]
36.  Derogatis LR, Savitz KL.   The SCL-90-R, Brief Symptom Inventory, and matching clinical rating scales. In Maruish ME. The use of psychological testing for treatment planning and outcomes assessment. New Jersey: Lawrence Erlbaum Associates Publishers, 1999: 679-724.  [PubMed]  [DOI]  [Cited in This Article: ]
37.  Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561-571.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 23191]  [Cited by in F6Publishing: 22440]  [Article Influence: 863.1]  [Reference Citation Analysis (0)]
38.  Kühner C, Bürger C, Keller F, Hautzinger M. [Reliability and validity of the Revised Beck Depression Inventory (BDI-II). Results from German samples]. Nervenarzt. 2007;78:651-656.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 483]  [Cited by in F6Publishing: 560]  [Article Influence: 32.9]  [Reference Citation Analysis (0)]
39.  Altman EG, Hedeker D, Peterson JL, Davis JM. The Altman Self-Rating Mania Scale. Biol Psychiatry. 1997;42:948-955.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 526]  [Cited by in F6Publishing: 533]  [Article Influence: 19.7]  [Reference Citation Analysis (0)]
40.  Buysse DJ, Reynolds CF 3rd, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28:193-213.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 17520]  [Cited by in F6Publishing: 18854]  [Article Influence: 538.7]  [Reference Citation Analysis (0)]
41.  Ekman P. An argument for basic emotions. Cogn Emot. 1992;6:169-200.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4011]  [Cited by in F6Publishing: 2047]  [Article Influence: 127.9]  [Reference Citation Analysis (0)]
42.  VERBI Software  MAXQDA 2020 [Computer-Software]. Available from: https://www.maxqda.de/.  [PubMed]  [DOI]  [Cited in This Article: ]
43.  Maassen EF, Regeer BJ, Regeer EJ, Bunders JFG, Kupka RW. The challenges of living with bipolar disorder: a qualitative study of the implications for health care and research. Int J Bipolar Disord. 2018;6:23.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 15]  [Article Influence: 2.5]  [Reference Citation Analysis (0)]
44.  Stevens AWMM, Daggenvoorde TH, van der Klis SMD, Kupka RW, Goossens PJJ. Thoughts and Considerations of Women With Bipolar Disorder About Family Planning and Pregnancy: A Qualitative Study. J Am Psychiatr Nurses Assoc. 2018;24:118-126.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 6]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
45.  Ateş C, Kaymaz Ö, Kale HE, Tekindal MA. Comparison of Test Statistics of Nonnormal and Unbalanced Samples for Multivariate Analysis of Variance in terms of Type-I Error Rates. Comput Math Methods Med. 2019;2019:2173638.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 37]  [Cited by in F6Publishing: 33]  [Article Influence: 6.6]  [Reference Citation Analysis (0)]
46.  Levy KJ. Some empirical power results associated with welch´s robust analysis of variance technique. J Stat Comput Simul. 2007;8:43-48.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 11]  [Cited by in F6Publishing: 10]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
47.  Lix LM, Keselman JC, Keselman HJ. Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance F test. Rev Educ Res. 1996;66:579-619.  [PubMed]  [DOI]  [Cited in This Article: ]
48.  Hunt CA, Gao J, Xue L. A visual analysis of trends in the titles and keywords of top-ranked tourism journals. Curr Issues Tour. 2014;17:849-855.  [PubMed]  [DOI]  [Cited in This Article: ]
49.  ROGERS CR. The necessary and sufficient conditions of therapeutic personality change. J Consult Psychol. 1957;21:95-103.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2456]  [Cited by in F6Publishing: 1417]  [Article Influence: 21.1]  [Reference Citation Analysis (0)]
50.  Hoyer J, Knappe S.   Klinische Psychologie & Psychotherapie. Basel: Springer International Publishing AG, 2020.  [PubMed]  [DOI]  [Cited in This Article: ]
51.  Johnson SL, McKenzie G, McMurrich S. Ruminative Responses to Negative and Positive Affect Among Students Diagnosed with Bipolar Disorder and Major Depressive Disorder. Cognit Ther Res. 2008;32:702-713.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 141]  [Cited by in F6Publishing: 157]  [Article Influence: 9.2]  [Reference Citation Analysis (0)]
52.  Gruber J, Eidelman P, Johnson SL, Smith B, Harvey AG. Hooked on a feeling: rumination about positive and negative emotion in inter-episode bipolar disorder. J Abnorm Psychol. 2011;120:956-961.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 104]  [Cited by in F6Publishing: 112]  [Article Influence: 8.6]  [Reference Citation Analysis (0)]
53.  Kovács LN, Takacs ZK, Tóth Z, Simon E, Schmelowszky Á, Kökönyei G. Rumination in major depressive and bipolar disorder - a meta-analysis. J Affect Disord. 2020;276:1131-1141.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 23]  [Cited by in F6Publishing: 23]  [Article Influence: 5.8]  [Reference Citation Analysis (0)]
54.  Babayan BM, Uchida N, Gershman SJ. Belief state representation in the dopamine system. Nat Commun. 2018;9:1891.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 64]  [Cited by in F6Publishing: 51]  [Article Influence: 8.5]  [Reference Citation Analysis (0)]
55.  Seitz RJ, Paloutzian RF, Angel HF. Believing is representation mediated by the dopamine brain system. Eur J Neurosci. 2019;49:1212-1214.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 5]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
56.  Paloutzian RF, Seitz RJ, Angel H-F. The process of believing and psychiatric symptoms. Religion Brain Behav. 2018;10:184-191.  [PubMed]  [DOI]  [Cited in This Article: ]
57.  Seitz RJ. Beliefs: A challenge in neuropsychological disorders. J Neuropsychol. 2022;16:21-37.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 9]  [Article Influence: 3.0]  [Reference Citation Analysis (0)]
58.  Seitz RJ, Paloutzian RF, Angel HF. Processes of believing: Where do they come from? F1000Res. 2016;5:2573.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 14]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]