Meta-Analysis
Copyright ©The Author(s) 2017. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Psychiatr. Mar 22, 2017; 7(1): 60-76
Published online Mar 22, 2017. doi: 10.5498/wjp.v7.i1.60
Consequences of bullying victimization in childhood and adolescence: A systematic review and meta-analysis
Sophie E Moore, Rosana E Norman, Shuichi Suetani, Hannah J Thomas, Peter D Sly, James G Scott
Sophie E Moore, Peter D Sly, Child Health Research Centre, the University of Queensland, South Brisbane, QLD 4101, Australia
Rosana E Norman, Institute of Health and Biomedical Innovation, Queensland University of Technology, Kelvin Grove, QLD 4059, Australia
Rosana E Norman, School of Public Health and Social Work, Queensland University of Technology, Kelvin Grove, QLD 4059, Australia
Shuichi Suetani, Queensland Centre for Mental Health Research, the Park Centre for Mental Health, Wacol, QLD 4076, Australia
Shuichi Suetani, Faculty of Medicine, the University of Queensland, Herston, QLD 4029, Australia
Hannah J Thomas, James G Scott, the University of Queensland Centre for Clinical Research, the University of Queensland, Herston, QLD 4029, Australia
James G Scott, Metro North Mental Health, Royal Brisbane and Women’s Hospital, Herston, QLD 4029, Australia
Author contributions: Norman RE and Scott JG designed the study, supervised the systematic review and meta-analysis and supervised the writing of the manuscript; Moore SE and Suetani S conducted the systematic review; Moore SE conducted the meta-analysis and wrote the first draft of the manuscript; Thomas HJ drafted sections of the manuscript related to bullying measurement and supervised the manuscript content; all authors contributed to and approved the final manuscript.
Conflict-of-interest statement: The authors have no conflicts of interest to declare.
Data sharing statement: No additional data is available.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: James G Scott, Associate Professor, the University of Queensland Centre for Clinical Research, the University of Queensland, Bowen Bridge Rd, Herston, QLD 4029, Australia. james.scott@health.qld.gov.au
Telephone: +61-73-6368111 Fax: +61-73-6361166
Received: September 13, 2016
Peer-review started: September 14, 2016
First decision: October 21, 2016
Revised: December 4, 2016
Accepted: December 27, 2016
Article in press: December 28, 2016
Published online: March 22, 2017

Abstract
AIM

To identify health and psychosocial problems associated with bullying victimization and conduct a meta-analysis summarizing the causal evidence.

METHODS

A systematic review was conducted using PubMed, EMBASE, ERIC and PsycINFO electronic databases up to 28 February 2015. The study included published longitudinal and cross-sectional articles that examined health and psychosocial consequences of bullying victimization. All meta-analyses were based on quality-effects models. Evidence for causality was assessed using Bradford Hill criteria and the grading system developed by the World Cancer Research Fund.

RESULTS

Out of 317 articles assessed for eligibility, 165 satisfied the predetermined inclusion criteria for meta-analysis. Statistically significant associations were observed between bullying victimization and a wide range of adverse health and psychosocial problems. The evidence was strongest for causal associations between bullying victimization and mental health problems such as depression, anxiety, poor general health and suicidal ideation and behaviours. Probable causal associations existed between bullying victimization and tobacco and illicit drug use.

CONCLUSION

Strong evidence exists for a causal relationship between bullying victimization, mental health problems and substance use. Evidence also exists for associations between bullying victimization and other adverse health and psychosocial problems, however, there is insufficient evidence to conclude causality. The strong evidence that bullying victimization is causative of mental illness highlights the need for schools to implement effective interventions to address bullying behaviours.

Key Words: Bullying, Victimization, Systematic review, Meta-analysis, Child, Adolescent

Core tip: There is convincing evidence of a causal association between exposure to bullying victimization in children and adolescents and adverse health outcomes including anxiety, depression, poor mental health, poor general health, non-suicidal self-injury, suicidal ideation and suicide attempts. It is probable that bullying victimization also causes an increased risk of cigarette smoking and illicit drug use. This review highlights that bullying victimization is associated with a wide and diverse range of problems and reinforces the need for effective interventions to be implemented in schools to address the high prevalence of children and adolescents engaging in bullying behaviours.



INTRODUCTION

Bullying victimization among children and adolescents is a global public health issue, well-recognised as a behaviour associated with poor adjustment in youth[1]. There is evidence suggesting bullying victimization in children and adolescents has enduring effects which may persist into adulthood[2-4]. Bullying victimization is most commonly defined as exposure to negative actions repeatedly and over time from one or more people, and involves a power imbalance between the perpetrator(s) and the victim[5]. Traditional bullying includes physical contact (pushing, hitting) as well as verbal harassment (name calling, verbal taunting), rumour spreading, intentionally excluding a person from a group, and obscene gestures. In recent years cyberbullying has emerged as a significant public health problem[5-8].

The estimated prevalence of bullying victimization is wide-ranging, with 10% and 35% of adolescents experiencing recurrent bullying victimization[9-16]. While contextual and cultural differences influence prevalence estimates[17], this variation is most frequently explained by differences in measurement strategy[18-20]. As a result, researchers continue to call for greater consensus in the definition and measurement of bullying behaviours[17,21,22]. Cook et al[17] examined the variability in prevalence of bullying victimization in a meta-analysis, and more recently Modecki et al[20] synthesised studies measuring both traditional bullying and cyberbullying. Mean prevalence was 36% for traditional bullying victimization and 15% for cyberbullying victimization[20]. There was significant overlap between bullying victimization in traditional and online settings[20]. A meta-analysis by Kowalski et al[23] showed that the strongest predictor of cyber-victimization was traditional bullying victimization.

Many studies have examined adverse health and psychosocial problems associated with bullying victimization. Those most commonly reported are mental health problems, specifically depression, anxiety, self-harm, and suicidal behaviour[2,14,24-28]. Over the past two decades researchers have conducted a number of systematic reviews to examine the relationship between bullying victimization and ill mental health.

The first systematic investigation by Hawker and Boulton[1] was a meta-analysis of cross-sectional studies of peer victimization published between 1978 and 1997. The authors reported victimization was significantly associated with depression, loneliness, reduced self-esteem and self-concept, as well as anxiety. To understand the temporal sequence between peer victimization and mental health problems, Reijntjes and colleagues conducted a pair of meta-analyses of longitudinal studies to examine internalizing (depression, anxiety, withdrawal, loneliness, and somatic complaints) and externalizing behaviours (aggression and delinquency) and peer victimization[29,30]. They examined two prospective paths: (1) peer victimization at baseline and changes in internalizing and externalizing problems at a second time point; and (2) internalizing and externalizing problems at baseline and changes in peer victimization at follow-up. The two meta-analyses demonstrated internalizing and externalizing behaviours are both antecedents and consequences of bullying victimization[29,30].

Another meta-analytic review on bullying victimization and depression by Ttofi et al[31] found those children who were bullied at school were twice as likely to develop depression compared to those who had not been bullied. In addition, another meta-analytic review found those children involved in any bullying behaviour were more likely to develop psychosomatic problems[32]. Finally, three systematic reviews have shown an association between bullying victimization and increased risk of adolescent suicidal ideation and behaviours[33-35].

In contrast to mental health, there is mixed evidence for the relationship between bullying victimization and substance use. Some studies report that bullying victimization is associated with a reduced risk of engaging in harmful alcohol use in later life[16,28], whereas others suggest that being bullied may result in an increased probability of later harmful alcohol use[36,37]. Similarly, some studies have shown an association between being bullied and later illicit drug use and smoking[36-39], whereas others have found no association at all[2,14,24,40].

The association between bullying victimization and psychosocial problems, such as academic achievement and school functioning/connectedness and criminal behaviour has also been examined. A meta-analysis by Nakamoto and Schwartz[41] found a small but significant negative association between bullying victimization and academic achievement. However, Kowalski et al[23] found no significant relationship between cyberbullying victimization and academic achievement. Another study found those exposed to bullying victimization in adolescence were at increased risk of involvement in criminal behaviour such as carrying a weapon[40].

There are now a large number of studies examining associations between bullying victimization and a wide range of adverse health and psychosocial problems. However, many of these have not been systematically examined and many existing systematic reviews did not include cyberbullying. Furthermore, although associations exist, it is unclear if there is a causal relationship. It is plausible that there are common factors that predispose individuals to being bullied in childhood but independently also increase the risk of adverse health and other psychosocial problems. Rigorous appraisal is required to consider both the possibility of a causal association but also other plausible explanations for any significant associations. Given the variation between studies, this study aimed to investigate adverse outcomes of both traditional and cyber bullying victimization and conduct a meta-analysis to summarize each association. Furthermore, we critically evaluated whether sufficient evidence existed to establish a causal relationship between bullying victimization and each of the adverse health and psychosocial problems. This is the first study to complete a summary of the evidence for all adverse health and psychosocial problems that are potentially a consequence of traditional and cyber bullying victimization.

MATERIALS AND METHODS

This study followed the recommendations from the PRISMA 2009 revision[42] and the guidelines outlined by the Meta-analysis of Observational Studies in Epidemiology[43] (Supplementary material S1). Methods and inclusion and exclusion criteria were specified in advance in the review protocol (Supplementary material S2).

Inclusion and exclusion criteria

This systematic review and meta-analysis included studies meeting the following inclusion criteria: (1) reported original, empirical research published in a peer reviewed journal; (2) examined the relationship between exposure to bullying victimization as a child or adolescent and one or more consequences of the bullying exposure; and (3) is a population based study. This study did not examine a particular type of bullying victimization therefore all direct and indirect forms of bullying including cyberbullying were included. Included studies reported odds ratios (ORs) and confidence intervals (CIs) comparing those exposed to bullying victimization and those not exposed to bullying victimization or, alternatively, provided information from which effect sizes (ORs and CIs) could be calculated between those exposed to bullying victimization and an outcome.

Search strategy

Four electronic databases (PsycINFO, ERIC, EMBASE and PubMed) were used to search for literature on the adverse correlates of bullying victimization as either a child or adolescent from inception up to 28 February 2015. The search was not restricted to the English language nor by any other means. The searches of the databases were conducted using the terms: “bullying”, “bullied”, “harassment”, “intimidation”, “victimization” along with “child” and “adolescent”. As this study aimed to examine all correlates that were potentially a consequence of bullying victimization, the search terms used in conjunction with those above were broader terms such as “outcome”“harm”“consequence” and “risk”. In addition, reference lists of selected studies were screened for any other relevant study and articles in languages other than English were translated (Supplementary material S2).

Data collection and quality assessment

The full text of articles that met all inclusion criteria were retrieved and examined. Data extracted using a data extraction template included publication details, country where study was conducted, methodological characteristics such as sample size and study design, exposure and outcome measures, type of bullying and frequency (Supplementary material S2). Each study was then subjected to a quality assessment in order for the reviewers to rate the quality of each study. Two reviewers independently reviewed the included articles and completed the quality assessment and any disagreements were resolved by a third reviewer. The quality assessment tool was based on the Newcastle-Ottawa Scale for assessing the quality of observational studies[44] as used by Norman et al[45] (Supplementary material S2).

Statistical analysis

Following the method used by Norman et al[45], MetaXL version 2.1[46], an add-in for Microsoft Excel was used in this study to conduct the meta-analysis. ORs were chosen as the summary measure. Heterogeneity was assessed using the Cochran’s Q and I2 statistics[47]. This meta-analysis used a quality effects model[48], a modified version of the fixed-effects inverse variance model that additionally allows greater weight to be given to studies of higher quality vs studies of lesser quality. The quality effects model avoids the limitation in random-effects models of returning to equal weighting irrespective of sample size if heterogeneity is large[47,48]. Furthermore, in order to address the effects of important study characteristics and explore heterogeneity this study conducted subgroup analyses, dependent on data availability, for sex of participants in the sample, geographic location and income level (high income vs low-to-middle income as per the World Bank classification criteria), severity of the bullying (frequent - at least once a month, vs sometimes - less than once a month), age of bullying victimization (before 13 years of age vs after 13 years of age), and type of study (prospective vs cross-sectional).

RESULTS

A total of 8231 articles were primarily identified by the search, of which 3270 were duplicates. Titles and abstracts for the 4961 remaining unduplicated references were reviewed and 15 additional articles were found from reference lists. From reviewing the title and abstracts a further 4659 articles were excluded. This left 317 articles meeting the following criteria: (1) original research extracted from a peer reviewed journal; and (2) examined the bullying victimization as a child or adolescent and one or more outcomes. Of the 317 articles reviewed, a further 152 articles were excluded as they did not use a population based sample or did not report enough information to calculate an effect size. The remaining 165 articles provided evidence of an effect size for bullying victimization and an outcome (Figure 1) - either odds ratio with confidence intervals or provided data which enabled the calculation of effect sizes. The majority (n = 142) were from high income regions. There were far fewer studies (n = 22) from low- and middle-income countries, and only one study utilized cross-national samples from different income-level countries. Of the articles included, 57 had a prospective cohort design and the remaining 108 were cross-sectional. The majority of studies measured self-reported bullying victimization. Some were from samples collected from a state or regions where as others were nationally representative (Supplementary material S3).

Figure 1
Figure 1 PRISMA flow diagram showing process of study selection for inclusion in systematic review and meta-analyses.
Bullying victimization in children and adolescents and mental health

Bullying victimization in children and adolescents was associated with a wide range of adverse mental health outcomes (Table 1) including poor mental health (OR = 1.60; 95%CI: 1.42-1.81), syndromes such as depression and anxiety, and symptoms and behaviours such as psychotic symptoms, suicidal ideation and attempts. Specifically, those exposed to bullying victimization had an increased risk of depression (OR = 2.21; 95%CI: 1.34-3.65). This association remained significant for all the sub-group analyses including prospective studies, age bullying occurred, sex and severity of the bullying. A dose response existed between being “sometimes bullied” and “frequently bullied” and depression (OR = 1.78; 95%CI: 1.39-2.28 and OR = 3.26; 95%CI: 2.45-4.34 respectively). In comparing high- and low-to-middle income countries, there was no significant difference in the odds of developing depression. Those exposed to bullying victimization were significantly more likely to experience anxiety (OR = 1.77; 95%CI: 1.34-2.33) and exposure to bullying victimization was associated with a wide range of anxiety spectrum disorders such as social phobia and post-traumatic stress disorder. This association remained after conducting subgroup analyses including study type, sex and severity of the bullying; however, the association between bullying victimization and anxiety was not significant in children under 13 years (Table 1).

Table 1 Associations between bullying victimization in children and adolescents and mental health outcomes.
Data pointsPooled OR95%CI lower bound95%CI upper boundCochran’s QI² (%)Test for heterogeneity (P value)
Poor mental health
Pooling all391.61.421.81303.7987.49< 0.01
Study type
Retrospective/cross-sectional251.81.442.25211.7888.67< 0.01
Prospective cohort141.391.291.4922.1241.240.05
Sex
Male32.491.863.320.4400.8
Female32.381.4146.9571.220.03
Twins31.411.271.562.520.090.29
Severity of bullying
Sometimes81.51.271.7647.6885.23< 0.01
Frequent81.521.181.9551.486.38< 0.01
Anxiety
Pooling all581.771.342.333816.2398.51< 0.01
Anxiety321.561.391.75434.6192.87< 0.01
Social phobia82.481.593.8611.0136.410.14
Generalised anxiety disorder22.831.385.840.1100.74
PTSD126.411.9321.22497.1197.79< 0.01
Specific phobia12.415.6---
Separation anxiety disorder14.6210.6---
Panic disorder13.11.56.5---
Agoraphobia14.61.712.5---
Study type
Retrospective/cross-sectional392.021.213.383697.4898.97< 0.01
Prospective cohort191.291.061.5584.0378.58< 0.01
Age of bullying
Less than 13 yr131.40.583.41123.1290.25< 0.01
Older than 13 yr451.811.292.563688.8298.81< 0.01
Sex
Male161.841.32.59112.2386.63< 0.01
Female152.461.743.48124.3988.74< 0.01
Severity of bullying
Sometimes71.461.06243.4186.18< 0.01
Frequent252.471.943.14122.4780.4< 0.01
Geographic location and income level
Low-to-middle income222.411.753.32175.9788.07< 0.01
High income361.671.242.253441.1798.98< 0.01
Depression
Pooling all922.211.343.6514525.3299.37< 0.01
Major depressive disorder22.270.687.572.0751.630.15
Study type
Retrospective/cross-sectional631.951.243.072594.9797.61< 0.01
Prospective cohort293.031.316.984583.0599.39< 0.01
Age of bullying
Less than 13 yr362.111.632.72544.993.58< 0.01
Older than 13 yr562.291.244.2313806.7299.6< 0.01
Sex
Male272.071.482.89443.8494.14< 0.01
Female212.131.183.86313.593.62< 0.01
Severity of bullying
Sometimes151.781.392.2878.6182.19< 0.01
Frequent283.262.454.34224.4387.97< 0.01
Geographic location and income level
Low-to-middle income132.531.753.68143.9891.67< 0.01
High income792.151.263.6814351.7299.46< 0.01
Psychotic symptoms
Specific psychiatric symptoms62.071.492.8720.5775.69< 0.01
Non-clinical psychotic experiences92.682.033.5415.147.030.06
Psychotic symptoms52.731.973.7710.8663.160.03
Personality disorders
Anti-social personality disorder20.580.152.282.5360.480.11
Borderline personality disorder32.21.43.464.858.310.09
Eating disorders
Bulimia nervosa131.46.2---
Anorexia nervosa10.0040251---
Non-suicidal self injury
Pooling all301.751.42.19749.0296.13< 0.01
Study type
Retrospective/cross-sectional211.551.092.22721.1597.23< 0.01
Prospective cohort91.651.342.0219.3958.750.01
Sex
Male64.863.357.0713.5663.120.02
Female42.723.658.9266.370.03
Twins22.571.793.70.1200.73
Severity of bullying
Sometimes61.571.092.2534.0485.31< 0.01
Frequent72.521.63.9754.4988.99< 0.01
Suicidal ideation
Pooling all1051.771.562.022093.595.03< 0.01
Study type
Retrospective/cross-sectional861.81.562.092037.4695.83< 0.01
Prospective cohort191.681.382.0538.9853.82< 0.01
Age of bullying
Less than 13 yr221.851.482.374.471.77< 0.01
Older than 13 yr831.751.512.031984.0695.87< 0.01
Sex
Male211.951.642.3276.673.89< 0.01
Female182.151.842.5233.1548.720.01
Severity of bullying
Sometimes161.531.281.8235.1957.38< 0.01
Frequent212.592.063.2549.8359.87< 0.01
Geographic location and income level
Low-to-middle income111.311.061.6160.0283.34< 0.01
High income941.81.432.261894.9195.09< 0.01
Suicide attempt
Pooling all482.131.662.731110.4695.77< 0.01
Suicidal attempt/non-suicidal self injury32.971.685.236.3368.420.04
Study type
Retrospective/cross-sectional402.031.462.841105.6596.47< 0.01
Prospective cohort82.041.383.014.3400.74
Age of bullying
Less than 13 yr112.111.652.6911.4912.980.32
Older than 13 yr371.520.822.83579.7593.79< 0.01
Sex
Male72.931.655.1815.3854.50.02
Female72.891.525.4924.2371.11< 0.01
Severity of bullying
Sometimes92.191.712.87.5200.48
Frequent123.772.555.5828.3161.14< 0.01
Geographic location and income level
Low-to-middle income41.911.073.4322.1886.47< 0.01
High income442.171.692.81084.6196.04< 0.01
Behavioural problems
Pooling all541.371.181.59862.493.85< 0.01
Study type
Retrospective/cross-sectional291.180.991.41311.2191< 0.01
Prospective cohort251.560.942.58413.5394.2< 0.01
Sex
Male91.350.682.67450.0698.22< 0.01
Female71.990.974.188.0793.19< 0.01
Twins21.190.941.57.3186.310.01
Severity of bullying
Sometimes81.950.924.1243.7197.13< 0.01
Frequent82.260.766.69163.5195.72< 0.01
Externalising behaviours201.291.121.5157.0987.9< 0.01
Delinquent/deviant behaviour241.991.243.2423.7894.57< 0.01
Missed school71.490.992.2338.484.37< 0.01
Disruptive behavioural disorders
Attention deficit hyperactivity disorder12.60.88.5---
Oppositional defiant disorder10.80.32.5---
Conduct disorder12.60.88.8---
Other mental health outcomes (not included above)
Nervousness91.821.512.2135.5994.1< 0.01
Powerlessness21.060.951.181.6439.180.2
Feeling low72.261.663.08299.4798< 0.01
Irritability or bad temper71.821.512.2125.0795.2< 0.01
Feel helpless53.22.015.09273.5198.54< 0.01
Feeling tense33.072.064.563.0233.750.22
Unhappy/sad121.250.552.84668.6198.35< 0.01
Worried41.271.091.47314.3599.05< 0.01
Afraid32.681.186.099.4978.920.01

Bullying victimization was also associated with non-suicidal self-injury (OR = 1.75; 95%CI: 1.40-2.19) and increased risk of suicidal ideation (OR = 1.77; 95%CI: 1.56-2.02) and the association remained significant for all subgroup analyses (Table 1). A dose response existed between being “sometimes bullied” and “frequently bullied” and suicide ideation (OR = 1.53; 95%CI: 1.28-1.82 and OR = 2.59; 95%CI: 2.06-3.25 respectively). Bullying victimization was associated with an increase in suicide attempts (OR = 2.13; 95%CI: 1.66-2.73). Subgroup analysis showed both males and females were approximately three times more likely to attempt suicide if they were bullied (OR = 2.93; 95%CI: 1.65-5.18 and OR = 2.89; 95%CI: 1.52-5.49 respectively). There was nearly a fourfold increase in suicide attempts for individuals who experienced frequent bullying victimization (OR = 3.77; 95%CI: 2.55-5.58) (Table 1). When comparing high income countries to those with low and middle income, the odds of bullying victims developing suicidal ideation or attempting suicide were similar.

Although bullying victimization in children and adolescents was associated with the pooling of all behavioural problems (OR = 1.37; 95%CI: 1.18-1.59), this association was not significant in prospective cohort studies and no dose-response was observed. Diagnoses of disruptive behavioural disorders were not associated with bullying victimization in children and adolescents (Table 1).

Bullying victimization in children and adolescents and substance use

Table 2 presents the associations between bullying victimization and substance use. When all studies were pooled together there was a significant association between bullying victimization and alcohol use (OR = 1.26; 95%CI: 1.00-1.58). A subgroup analysis showed a significant association between bullying victimization and the risk of tobacco use for prospective studies (OR = 1.62; 95%CI: 1.31-1.99). Furthermore, a dose response was present with frequent bullying victimization being associated with tobacco use (OR = 3.19; 95%CI: 1.19-8.58), whereas no significant association was found with those who were “sometimes bullied” (Table 2).

Table 2 Associations between bullying victimization in children and adolescents and substance use.
Data pointsPooled OR95%CI lower bound95%CI upper boundCochran’s QI² (%)Test for heterogeneity (P value)
Alcohol use
Pooling all531.261.001.5810328.1899.5< 0.01
Study type
Retrospective/cross-sectional381.280.881.8410256.1599.64< 0.01
Prospective cohort151.190.871.6267.8779.37< 0.01
Sex
Male60.610.490.774.890.000.43
Female40.880.501.576.3652.860.1
Severity of bullying
Sometimes61.720.843.5086.1794.20< 0.01
Frequent131.530.783.03332.2796.39< 0.01
Frequency of alcohol consumption
Sometimes241.521.082.138416.9499.73< 0.01
Frequent290.990.861.14251.8188.88< 0.01
Age of bullying
Less than 13 yr161.230.931.631618.1499.07< 0.01
Older than 13 yr371.310.961.806896.7199.48< 0.01
Geographic location and income level
Low-to-middle income111.370.752.498328.2199.88< 0.01
High income421.080.911.27462.8491.14< 0.01
Tobacco use
Pooling all351.360.961.92418.7191.88< 0.01
Study type
Retrospective/cross-sectional261.170.592.31394.9293.67< 0.01
Prospective cohort91.621.311.9911.4830.330.18
Sex
Male30.970.591.588.2375.70.02
Female30.510.370.681.7800.41
Severity of bullying
Sometimes41.890.834.3371.3895.8< 0.01
Frequent43.191.198.5839.8592.47< 0.01
Frequency of smoking
Sometimes281.360.892.06400.7293.26< 0.01
Frequent71.351.001.8416.2863.160.01
Illicit drug use
Pooling all341.411.101.81677.6295.13< 0.01
Study type
Retrospective/cross-sectional112.431.424.15297.396.64< 0.01
Prospective cohort231.271.121.4467.2367.28< 0.01
Sex
Male71.040.811.3332.8481.73< 0.01
Female51.171.031.333.260.000.52
Severity of bullying
Sometimes71.220.781.90160.4796.26< 0.01
Frequent81.140.433.00465.4398.50< 0.01
Geographic location and income level
Low-to-middle income54.052.187.5598.6195.94< 0.01
High income291.311.151.48103.7373.01< 0.01
Cannabis only all91.420.962.1223.3265.70< 0.01
Study type
Retrospective/cross-sectional12.461.533.95---
Prospective cohort81.360.902.0518.1461.410.01

Bullying victimization was associated with an increased risk of illicit drug use (OR = 1.41; 95%CI: 1.10-1.81). Subgroup analysis revealed that the association between bullying victimization and increased risk of using illicit drugs was significant in both cross-sectional (OR = 2.43; 95%CI: 1.42-4.15) and prospective studies (OR = 1.27; 95%CI: 1.12-1.44). A subgroup analysis also revealed bullying victims in low-to-middle income countries are at an increased risk of illicit drug use (OR = 4.05; 95%CI: 2.18-7.55) compared with bullying victims in high income countries (OR = 1.31; 95%CI: 1.15-1.48). No significant association was found in a sub group analysis examining cannabis and bullying victimization in children and adolescence (Table 2).

Bullying victimization in children and adolescents and other health outcomes

Table 3 presents the association between bullying victimization and other health outcomes. Bullying victimization was associated with increased risk of somatic symptoms, the most common being stomach ache (OR = 1.76; 95%CI: 1.53-2.03), sleeping difficulties (OR = 1.73; 95%CI: 1.46-2.05), headaches (OR = 1.64; 95%CI: 1.38-1.94), dizziness (OR = 1.64; 95%CI: 1.38-1.95), and back pain (OR = 1.67; 95%CI: 1.43-1.95). Bullying victimization was also associated with an increased risk of being overweight and obese (OR = 1.68; 95%CI: 1.21-2.33 and OR = 1.78; 95%CI: 1.42-2.21, respectively). These associations were significant for cross-sectional studies only and there was no dose response.

Table 3 Associations between bullying victimization in children and adolescents and other health outcomes.
Data pointsPooled OR95%CI lower bound95%CI upper boundCochran’s QI² (%)Test for heterogeneity (P value)
Somatic symptoms
Unspecified psychosomatic symptoms252.001.542.60232.0289.66< 0.01
Stomach ache251.761.532.03138.7382.7< 0.01
Sleeping difficulties241.731.462.05574.9196< 0.01
Headache261.641.381.94169.1685.22< 0.01
Bedwetting32.511.444.374.9359.450.08
Feeling tired22.681.395.191.2217.870.27
Poor appetite22.231.603.12000.95
Back pain81.671.431.9573.5390.48< 0.01
Skin problems11.821.33251---
Dizziness91.641.381.9576.5789.55< 0.01
Eating and weight related problems
Binge eating22.661.684.220.5700.45
Non-diet soft drink consumption11.211.041.41---
Skips breakfast61.411.201.6511.8957.940.04
Underweight
Pooling all21.270.732.21000.96
Sex
Male11.280.692.37---
Female11.240.192.29---
Overweight
Pooling all141.681.212.3382.6984.28< 0.01
Study type
Retrospective/cross- sectional121.991.392.8565.4583.19< 0.01
Prospective cohort20.980.641.491.9247.970.17
Sex
Male71.220.991.498.0425.40.23
Female72.221.283.8450.1788.04< 0.01
Severity of bullying
Sometimes21.321.001.740.0900.77
Frequent61.140.881.477.3732.20.19
Obese
Pooling all131.781.422.2114.6818.280.26
Study type
Retrospective/cross-sectional101.971.532.537.2200.61
Prospective cohort31.570.892.776.8970.970.03
Sex
Male61.941.452.604.8800.43
Female62.151.572.942.2200.82
Severity of bullying
Sometimes21.631.112.380.5200.47
Frequent62.091.592.755.264.860.39
Sexual behaviour problems
Teen parent51.260.811.9715.1173.53< 0.01
Risky sexual behaviour42.280.955.4823.4387.2< 0.01
Early onset of sexual activities31.440.902.307.3872.910.02
Pooling all121.511.012.2585.6687.16< 0.01
Study type
Retrospective/cross-sectional31.770.427.5254.9796.36< 0.01
Prospective cohort91.340.981.8423.8866.51< 0.01
Severity of bullying
Sometimes20.810.511.282.4659.320.12
Frequent42.381.055.4127.5589.11< 0.01
Health services utilised
Pooling all161.200.991.4534.3756.36< 0.01
Study type
Retrospective/cross-sectional141.140.941.3927.9153.420.01
Prospective cohort21.540.653.614.4377.430.04
Sex
Male71.170.951.433.5400.74
Female71.411.121.7711.1646.250.08
General medication use
Pooling all121.160.801.70117.9890.68< 0.01
Study type
Retrospective/cross-sectional110.990.561.75112.2691.09< 0.01
Prospective cohort11.671.092.58---
Sex
Male
Medication for headache21.431.061.932.3457.210.13
Medication for stomach-ache21.090.721.651.947.50.17
Female
Medication for headache21.190.981.451.3324.790.25
Medication for stomach-ache21.231.011.50.2700.61
Severity of bullying
Sometimes51.260.991.5915.4674.13< 0.01
Frequent51.721.112.6724.983.93< 0.01
Over the counter drug misuse30.950.194.6676.3497.38< 0.01
Psychotropic medication use
Pooling all131.280.722.26205.7694.17< 0.01
Study type
Retrospective/cross-sectional110.950.322.8195.3494.88< 0.01
Prospective cohort21.310.662.65.6182.180.02
Sex
Male
Medication for nervousness21.320.424.114.9893.32< 0.01
Medication for sleeping21.891.332.671.5937.280.21
Female
Medication for nervousness21.971.492.591.065.880.3
Medication for sleeping21.831.422.360.2700.6
Severity of bullying
Sometimes61.661.262.1818.5473.04< 0.01
Frequent61.881.173.0331.9384.34< 0.01
Prescription drug misuse30.920.175.0788.1697.73< 0.01
Poor general health
Pooling all291.831.452.31133.3179< 0.01
Study type
Retrospective/cross-sectional221.711.212.42112.0681.26< 0.01
Prospective cohort71.561.072.2819.7269.57< 0.01
Sex
Male91.951.163.2720.560.980.01
Female92.361.115.0452.7884.84< 0.01
Severity of bullying
Sometimes42.161.104.269.1567.210.03
Frequent46.962.1722.3516.7282.05< 0.01

When all studies were pooled, bullying victimization in children and adolescents was associated with increased risk of sexual behaviour problems (OR = 1.51; 95%CI: 1.01-2.25) which included teenage pregnancy, early onset of sexual activities and risky sexual behaviour. Subgroup analyses were not significant with the exception of those frequently bullied. Although bullying victimization was associated with increased likelihood of poor general health (OR = 1.83; 95%CI: 1.45-2.31) and this association persisted when restricted to prospective cohort studies (OR = 1.56; 95%CI: 1.07-2.28). There was no consistent increase in utilisation of health services or medications in those exposed to bullying victimization during childhood or adolescence (Table 3).

Bullying victimization in children and adolescents and academic and social functioning

The association between bullying victimization and functioning at school was inconsistent. There was a robust association between bullying victimization in childhood or adolescence and poor academic achievement. Those who had exposure to bullying victimization were more likely to have poor academic achievement (OR = 1.33; 95%CI: 1.06-1.66), whilst those with good academic achievement were less likely to have been exposed to bullying victimization (OR = 0.71; 95%CI: 0.60-0.85); however, all studies except one were cross-sectional. Bullying victimization was not associated with later financial or occupational functioning.

Similarly, there were inconsistent associations between bullying victimization and social problems. Those exposed were approximately twice as likely to report loneliness (OR = 1.89; 95%CI: 1.39-2.57) and poor life satisfaction (OR = 2.26; 95%CI: 1.41-3.60) and were significantly less likely to have a good quality of life (OR = 0.85; 95%CI: 0.78-0.93). Bullying victimization was not consistently associated with low self-esteem, social problems, or criminal behaviours (Table 4).

Table 4 Associations between bullying victimization and academic and social functioning.
Data pointsPooled OR95%CI lower bound95%CI upper boundCochran’s QI² (%)Test for heterogeneity (P value)
Poor school functioning
Pooling all61.100.871.388293.9< 0.01
Study type
Retrospective/cross-sectional31.241.221.270.3300.85
Prospective cohort30.900.761.088.1575.460.02
Severity of bullying
Sometimes10.960.881.04---
Frequent10.980.761.19---
Academic achievement
Poor academic achievement
Pooling all61.331.061.6611.1755.250.02
Study type
Retrospective/cross-sectional61.331.061.6611.1755.250.02
Good academic achievement
Pooling all40.710.600.858.8165.970.07
Study type
Retrospective/cross-sectional30.860.80.922.8930.690.58
Prospective cohort10.460.280.76---
Sex
Male21.240.881.742.4959.80.65
Female21.320.991.751.428.70.84
Severity of bullying
Sometimes10.880.830.93---
Frequent10.800.700.93---
Poor financial and occupational functioning
Pooling all prospective cohort161.140.871.5092.9783.86< 0.01
Severity of bullying
Sometimes31.000.91.110.0400.98
Frequent30.810.611.072.6825.320.26
Social isolation
Loneliness
Pooling all131.891.392.573120.6699.62< 0.01
Study type
Retrospective/cross-sectional131.891.392.573120.6699.62< 0.01
Sex
Male42.581.624.10222.2198.65< 0.01
Female33.921.957.9019.5389.76< 0.01
Severity of bullying
Sometimes22.091.982.200.3900.53
Frequent44.122.247.6023.3287.13< 0.01
Self esteem
Pooling all140.990.921.0793.7386.13< 0.01
Study type
Retrospective/cross-sectional41.130.831.5476.5896.08< 0.01
Prospective cohort100.970.931.0112.3226.930.2
Sex
Male50.960.881.0620.6580.63< 0.01
Female40.950.881.035.7447.70.13
Severity of bullying
Sometimes50.990.951.041.6100.81
Frequent50.950.871.049.6558.540.05
Social problems
Pooling all221.020.741.42427.1395.08< 0.01
Study type
Retrospective/cross-sectional52.861.425.7638.0989.5< 0.01
Prospective cohort170.890.741.0672.3677.89< 0.01
Sex
Male12.891.455.73---
Female18.104.6014.26---
Severity of bullying
Sometimes30.90.830.960.500.78
Frequent30.810.720.923.6745.490.16
Criminal behaviour
Pooling all331.040.781.39133.3676.01< 0.01
Carrying a weapon81.591.271.9819.1663.470.01
Violent offense/behaviour61.251.011.562.4600.78
Study type
Retrospective/cross-sectional91.010.472.14106.8392.51< 0.01
Prospective cohort241.050.921.1925.7210.580.31
Sex
Male111.000.821.2213.7827.430.18
Female40.700.461.040.3800.94
Severity of bullying
Sometimes50.970.751.267.3745.740.12
Frequent81.220.861.7416.3357.130.02
Other outcomes reported
Good quality of later life60.850.780.9317.4271.29< 0.01
Poor life satisfaction62.261.413.603.7900.58
Problematic internet usage12.361.583.54---
Picked on by siblings11.691.382.07---
DISCUSSION

This paper provides the most comprehensive critical analysis of the association between bullying victimization and a wide range of health and psychosocial problems. The primary and sub-group analyses allow for interpretation of the evidence of causality within the Bradford-Hill Framework, based on the following: Biological plausibility, the temporal relationship of the association, strength and consistency of the association, the presence of a dose-response relationship, and whether an alternate explanation for the associations is possible[49]. We used the grading system developed by the World Cancer Research Fund[50] as used in the Global Burden of Disease study as a guideline for evaluation of the level of evidence.

Temporality

In this meta-analysis, both longitudinal (n = 57) and cross-sectional (n = 108) studies showed associations between bullying victimization and many adverse health and psychosocial problems. Prospective studies provided evidence of a temporal relationship showing bullying victimization preceded the later adverse consequences.

A temporal relationship exists between bullying victimization and outcomes such as anxiety, depression, non-suicidal self-injury, suicide ideation and suicide attempts. As poor mental health is also a known risk factor for bullying victimization[51], it is with caution we say that an independent temporal relationship exists between bullying victimization and these adverse mental health outcomes. Many studies did not control for pre-existing mental health and could be reporting a continuation of pre-existing psychopathology and not a direct outcome of the bullying victimization. Nonetheless, two recent studies have found that even when controlling for pre-existing mental health, bullying victimization was strongly associated with later adverse mental health consequences such as non-suicidal self-injury and depression[27,28].

Strength of the association

Both prospective and population-based studies demonstrated significant associations between bullying victimization and adverse health and psychosocial problems. After adjusting for confounding variables, there was generally a reduction in the strength of these associations. Furthermore, the magnitude of the associations diverged depending on the sub-group analysis performed. Despite some variability, bullying victimization was found to significantly increase the likelihood of mental ill health suggesting significant and robust associations.

Consistency of the association

Consistency of the associations between bullying victimization and mental ill health was demonstrated in the estimated effect sizes across studies. It is possible that publication bias affected the results for some of the outcomes. Direction of the association (as estimated through risk estimates) was consistent across different geographic regions, samples, study designs, and income levels investigated, particularly for anxiety, depression, non-suicidal self-injury, suicide ideation and suicide attempts (Supplementary material Figures S4, S5, S7, S8, S9). Inconsistent associations were observed for certain outcomes such as behavioural problems (Supplementaty material Figure S6).

Dose-response relationship

Available evidence suggests that experiencing more severe or frequent forms of adversity in childhood increases the risk of adverse outcomes compared to a lower exposure to adversity[45,52-56], particularly for mental health problems. Similarly, this study demonstrated a dose-response relationship between bullying victimization and detrimental effects on health, in particular for mental health problems. After summarizing the evidence through a meta-analysis, dose-response relationships were observed between bullying victimization and depression, suicide ideation, cigarette smoking and loneliness[45,52-56]. An increase in the dose of bullying victimization (frequent vs sometimes) resulted in non-significantly greater point estimates for other problems such as anxiety, medication use (general and psychotropic), suicide attempts and non-suicidal self-injury.

Plausibility

Due to a lack of animal models, the majority of inferences for biological plausibility arise from observational rather than experimental data. However, one model of social defeat in rats has been used to understand bullying victimization[57,58]. Two male rats are placed into a cage together, and after fighting, one rat becomes dominant and the other subordinate. The subordinate rat experiences social defeat and after a single experience demonstrates signs of stress. One study found that the subordinate rat demonstrated behaviours representative of depression in humans when exposed to multiple social defeats over several weeks[59].

Observational data has also been used to explain the association between bullying victimization in childhood and adolescence and the later development of mental health problems. First, early adverse experiences (i.e., bullying victimization) that occur during vulnerable developmental periods can cause neurobiological[60,61] or inflammatory[62] changes expressed as illnesses in later life[61]. Moreover, those individuals exposed to frequent bullying victimization who develop mental health problems may self-medicate their distress and negative emotions with alcohol, illicit drugs, medications, tobacco or disengaging from school.

Taking into account both the limited animal studies[57-59] and observational studies[60-62], it can be understood as to why bullying victimization can affect the immediate and long-term health and non-health related outcomes of the individual.

Consideration of alternate explanations

The relationship between bullying victimization and adverse health and psychosocial problems are thought to be complex and influenced by both genetics and environmental factors; however, there are limited twin studies available to inform these associations[27,63-66]. One study[64] found that being bullied in childhood is an environmentally mediated contributing factor to poor childhood mental health. Another found victimized twins were more likely to self-harm than their non-victimized twin sibling[27]. Exposure to bullying victimization has also been found to be associated with socioeconomic status[51,67] which is also known to play a role in the development of mental health problems and other health and non-health related outcomes[68].

It is further acknowledged that the association between bullying victimization and adverse outcomes is not necessarily an independent relationship. As early emotional and behavioural problems are known risk factors for bullying victimization, without adequate statistical adjustment, some studies may risk reporting pre-existing psychopathology rather than a direct outcome of bullying. The available evidence suggests a complex relationship between genetics and environment and neither can solely explain the relationship between bullying victimization and adverse outcomes. Even though some of the effects of bullying victimization on adverse outcomes reported may be a result of confounding factors, generally the association with mental health problems was significant after controlling for potential confounding factors.

Assessment of causality

Using the grading system developed by the World Cancer Research Fund (WCRF)[50] as a guideline for evaluation of the level of evidence, we concluded that there was “convincing evidence” for a causal relationship between bullying victimization and anxiety, depression, poor general and mental health, non-suicidal self-injury, suicide attempts, and suicide ideation. This evidence was based on a substantial number of epidemiological studies identified in this systematic review including prospective observational studies of sufficient size, duration, and quality showing consistent effects. In addition, the association was considered biologically plausible. We concluded that “probable evidence” of a causal relationship existed between exposure to bullying victimization and illicit drug and tobacco use based on the epidemiological evidence. Possible causal associations existed between bullying victimization and lower academic achievement, alcohol use, loneliness, obesity, overweight and psychosomatic symptoms. This evidence was based mainly on findings from cross-sectional studies and a few prospective studies showing inconsistent associations between exposure and disease. More studies are needed to support these tentative associations, which are also considered to be biologically plausible.

All other significant associations reported in this study were classified as having insufficient evidence of a causal relationship (Table 5). This is not suggesting that there is no causal relationship. Further research is needed to better examine if any associations that exist are causal or due to other confounding factors. Furthermore, the use of WCRF grading system, although appropriate for dietary risk factors, might not be adequate for psychosocial factors particularly newly emerging risks.

Table 5 Strength of evidence for a causal relationship between bullying victimization and adverse health or psychosocial problems.
Strength of evidenceAdverse health or psychosocial problem
ConvincingAnxiety; depression; poor mental health; poor general health; non-suicidal self-injury; suicide attempts; suicide ideation
ProbableTobacco use; illicit drug use
PossibleAlcohol use; psychotic symptoms; increased use of health services in females; lower academic achievement; social isolation; loneliness; psychosomatic symptoms, overweight and obesity
InsufficientBinge eating; bulimia nervosa; borderline personality disorder; behavioural problems; carrying a weapon; general medication use; health services sought; poor financial and occupational functioning; psychotropic medication use; poor school functioning; sexual behavioural problems; poor life satisfaction
Limitations

While we followed rigorous methodological steps, some limitations are notable. As studies with non-significant findings are less likely to be published, there may be a publication bias within this meta-analysis resulting in the association between bullying victimization and some adverse outcomes being overstated[69,70]. Additionally, inconsistencies would have occurred in the analysis due to methodological differences in the way bullying victimization is defined and measured throughout the studies as there is no consensus on the best way to measure bullying victimization[18,19]. In order to address this, a quality effects model was used giving higher scores to those studies which provided respondents with a definition and utilised a validated measure of bullying. There are also methodological issues in regards to the adverse outcomes reported, as some have been self-reported, while others were reported by teachers, parents, clinicians or through objective measures. This issue was also addressed with the use of a quality effects model in which higher quality scores were given to those studies where standardised validated diagnostic instruments were used to assess the outcome relative to those where outcomes were self-reported on a non-validated scale[44]. In spite of this methodology, the assessment of exposure to bullying and the assessment of a wide range of outcomes remains a challenge. In particular, there will always be some uncertainty pertaining to the measurement of bullying, especially when retrospectively reported as a result of the respondent’s subjective perception of the actions and behaviours of others.

As a research question involving bullying victimization can only be observational and not experimental, a further limitation of this meta-analysis are those limitations that come with observational studies[71]. First, we acknowledge the issue of confounding. It is appropriate to adjust for these confounders in the statistical analyses by either stratification or multivariate analysis[71]. Although many studies controlled for socio-demographic and other variables[2,27], some reported unadjusted odds ratios between bullying victimization and adverse outcomes, or provided only basic adjustment for sex and age[72,73]. This was addressed in this meta-analysis through the use of the quality score of studies where confounding factors were not adequately adjusted and by conducting further analyses where data were available[44]. Generally, after controlling for the effects of confounding variables, the associations between bullying victimization and adverse outcomes were attenuated. The majority of studies included in this meta-analysis did not identify individuals who were both victims and perpetrators of bullying. Previous research has suggested those who are both perpetrators and victims are at even greater risk of adverse mental health outcomes[28]; however, we were unable to confirm this with the current study.

In the majority of primary analyses of the association between bullying victimization and adverse outcomes, significant heterogeneity was present. This heterogeneity remained significant in most subgroup analyses even after controlling for study quality in the quality effects models[44].

In conclusion, evidence suggests a causal relationship between bullying victimization and mental health outcomes. There were also associations between bullying victimization and other adverse health and psychosocial problems which require further research to accurately measure the negative impact of bullying victimization and the broad health and economic costs. Through the implementation of school wide interventions that involve the entire school community (i.e., staff, students, and parents) bullying behaviour is considered a modifiable risk factor[25,74]. This review highlights the increased likelihood of a wide and diverse range of problems that are experienced by those exposed to bullying victimization. These findings reinforce the need for implementation of effective interventions in schools to address the high prevalence of children and adolescents engaging in bullying behaviours.

COMMENTS
Background

Bullying victimization (including traditional and cyberbullying) among children and adolescents is a global public health issue, well-recognised as a behaviour associated with poor adjustment in youth. There is evidence suggesting bullying victimization in children and adolescents has enduring effects which may persist into adulthood.

Research frontiers

There have been many studies examining the association between bullying victimization in children and adolescents and adverse health and social problems. However, many of these have not been systematically examined and existing systematic reviews did not include cyberbullying. Furthermore, although associations exist, it is unclear if there is a causal relationship.

Innovations and breakthroughs

The authors found convincing evidence of a causal relationship between bullying victimization in children and adolescents and adverse health outcomes including anxiety, depression, poor mental health, poor general health, non-suicidal self-injury, suicidal ideation and suicide attempts. It is probable that bullying victimization also causes an increased risk of cigarette smoking and illicit drug use.

Applications

Given the convincing evidence of a causal association, there is an urgent need for effective interventions to be implemented in schools to address the high prevalence of children and adolescents engaging in bullying behaviours.

Peer-review

This is an important topic on consequences of bullying victimization in childhood and adolescence. This area for sure needs more attention. The authors have done a great job presenting a large systematic review and meta-analysis of studies correlating the history of bullying victimization with different mental health problems in childhood and adolescence.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Psychiatry

Country of origin: Australia

Peer-review report classification

Grade A (Excellent): 0

Grade B (Very good): B, B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Alavi N, Classen CF S- Editor: Ji FF L- Editor: A E- Editor: Wu HL

References
1.  Hawker DS, Boulton MJ. Twenty years’ research on peer victimization and psychosocial maladjustment: a meta-analytic review of cross-sectional studies. J Child Psychol Psychiatry. 2000;41:441-455.  [PubMed]  [DOI]
2.  Copeland WE, Wolke D, Angold A, Costello EJ. Adult psychiatric outcomes of bullying and being bullied by peers in childhood and adolescence. JAMA Psychiatry. 2013;70:419-426.  [PubMed]  [DOI]
3.  Takizawa R, Maughan B, Arseneault L. Adult health outcomes of childhood bullying victimization: evidence from a five-decade longitudinal British birth cohort. Am J Psychiatry. 2014;171:777-784.  [PubMed]  [DOI]
4.  Sigurdson JF, Undheim AM, Wallander JL, Lydersen S, Sund AM. The long-term effects of being bullied or a bully in adolescence on externalizing and internalizing mental health problems in adulthood. Child Adolesc Psychiatry Ment Health. 2015;9:42.  [PubMed]  [DOI]
5.  Olweus D Bullying at school: What we know and what we can do. Oxford, UK: Blackwell 1993; .  [PubMed]  [DOI]
6.  Olweus D. Bullying at school: basic facts and effects of a school based intervention program. J Child Psychol Psychiatry. 1994;35:1171-1190.  [PubMed]  [DOI]
7.  Smith PK, Mahdavi J, Carvalho M, Fisher S, Russell S, Tippett N. Cyberbullying: its nature and impact in secondary school pupils. J Child Psychol Psychiatry. 2008;49:376-385.  [PubMed]  [DOI]
8.  Sourander A, Brunstein Klomek A, Ikonen M, Lindroos J, Luntamo T, Koskelainen M, Ristkari T, Helenius H. Psychosocial risk factors associated with cyberbullying among adolescents: a population-based study. Arch Gen Psychiatry. 2010;67:720-728.  [PubMed]  [DOI]
9.  Cross D, Shaw T, Hearn L, Epstein M, Monks H, Lester L, Thomas L.  Australian covert bullying prevalence study (ACBPS). Perth, Australia Child Health Promotion Research Centre, Edith Cowan University. 2009;.  [PubMed]  [DOI]
10.  Jansen PW, Verlinden M, Dommisse-van Berkel A, Mieloo C, van der Ende J, Veenstra R, Verhulst FC, Jansen W, Tiemeier H. Prevalence of bullying and victimization among children in early elementary school: do family and school neighbourhood socioeconomic status matter? BMC Public Health. 2012;12:494.  [PubMed]  [DOI]
11.  Rigby K, Slee PT. Bullying among Australian school children: reported behavior and attitudes toward victims. J Soc Psychol. 1991;131:615-627.  [PubMed]  [DOI]
12.  Solberg ME, Olweus D. Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire. Aggress Behav. 2003;29:239-268.  [PubMed]  [DOI]
13.  Due P, Holstein BE, Soc MS. Bullying victimization among 13 to 15-year-old school children: results from two comparative studies in 66 countries and regions. Int J Adolesc Med Health. 2008;20:209-221.  [PubMed]  [DOI]
14.  Sourander A, Jensen P, Rönning JA, Niemelä S, Helenius H, Sillanmäki L, Kumpulainen K, Piha J, Tamminen T, Moilanen I. What is the early adulthood outcome of boys who bully or are bullied in childhood? The Finnish “From a Boy to a Man” study. Pediatrics. 2007;120:397-404.  [PubMed]  [DOI]
15.  Hemphill SA, Kotevski A, Herrenkohl TI, Bond L, Kim MJ, Toumbourou JW, Catalano RF. Longitudinal consequences of adolescent bullying perpetration and victimisation: a study of students in Victoria, Australia. Crim Behav Ment Health. 2011;21:107-116.  [PubMed]  [DOI]
16.  Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B, Scheidt P. Bullying behaviors among US youth: prevalence and association with psychosocial adjustment. JAMA. 2001;285:2094-2100.  [PubMed]  [DOI]
17.  Cook CR, Williams KR, Guerra NG, Kim TE. Variability in the prevalence of bullying and victimization: A cross-national and methodological analysis. Handbook of bullying in schools: An international perspective. New York: Routledge 2010; 347-362.  [PubMed]  [DOI]
18.  Griffin RS, Gross AM. Childhood bullying: Current empirical findings and future directions for research. Aggress Violent Beh. 2004;9:379-400.  [PubMed]  [DOI]
19.  Shaw T, Dooley JJ, Cross D, Zubrick SR, Waters S. The Forms of Bullying Scale (FBS): validity and reliability estimates for a measure of bullying victimization and perpetration in adolescence. Psychol Assess. 2013;25:1045-1057.  [PubMed]  [DOI]
20.  Modecki KL, Minchin J, Harbaugh AG, Guerra NG, Runions KC. Bullying prevalence across contexts: a meta-analysis measuring cyber and traditional bullying. J Adolesc Health. 2014;55:602-611.  [PubMed]  [DOI]
21.  Thomas HJ, Connor JP, Scott JG. Integrating traditional bullying and cyberbullying: Challenges of definition and measurement in adolescents - a review. Educ Psychol Rev. 2015;27:135-152.  [PubMed]  [DOI]
22.  Ybarra ML, Boyd D, Korchmaros JD, Oppenheim JK. Defining and measuring cyberbullying within the larger context of bullying victimization. J Adolesc Health. 2012;51:53-58.  [PubMed]  [DOI]
23.  Kowalski RM, Giumetti GW, Schroeder AN, Lattanner MR. Bullying in the digital age: a critical review and meta-analysis of cyberbullying research among youth. Psychol Bull. 2014;140:1073-1137.  [PubMed]  [DOI]
24.  Coomber K, Toumbourou JW, Miller P, Staiger PK, Hemphill SA, Catalano RF. Rural adolescent alcohol, tobacco, and illicit drug use: a comparison of students in Victoria, Australia, and Washington State, United States. J Rural Health. 2011;27:409-415.  [PubMed]  [DOI]
25.  Vreeman RC, Carroll AE. A systematic review of school-based interventions to prevent bullying. Arch Pediatr Adolesc Med. 2007;161:78-88.  [PubMed]  [DOI]
26.  Klomek AB, Sourander A, Kumpulainen K, Piha J, Tamminen T, Moilanen I, Almqvist F, Gould MS. Childhood bullying as a risk for later depression and suicidal ideation among Finnish males. J Affect Disord. 2008;109:47-55.  [PubMed]  [DOI]
27.  Fisher HL, Moffitt TE, Houts RM, Belsky DW, Arseneault L, Caspi A. Bullying victimisation and risk of self harm in early adolescence: longitudinal cohort study. BMJ. 2012;344:e2683.  [PubMed]  [DOI]
28.  Moore SE, Norman RE, Sly PD, Whitehouse AJ, Zubrick SR, Scott J. Adolescent peer aggression and its association with mental health and substance use in an Australian cohort. J Adolesc. 2014;37:11-21.  [PubMed]  [DOI]
29.  Reijntjes A, Kamphuis JH, Prinzie P, Boelen PA, van der Schoot M, Telch MJ. Prospective linkages between peer victimization and externalizing problems in children: a meta-analysis. Aggress Behav. 2011;37:215-222.  [PubMed]  [DOI]
30.  Reijntjes A, Kamphuis JH, Prinzie P, Telch MJ. Peer victimization and internalizing problems in children: a meta-analysis of longitudinal studies. Child Abuse Negl. 2010;34:244-252.  [PubMed]  [DOI]
31.  Ttofi MM, Farrington DP, Lösel F, Loeber R. Do the victims of school bullies tend to become depressed later in life? A systematic review and meta‐analysis of longitudinal studies. J Aggress Confl Peace Res. 2011;3:63-73.  [PubMed]  [DOI]
32.  Gini G, Pozzoli T. Association between bullying and psychosomatic problems: a meta-analysis. Pediatrics. 2009;123:1059-1065.  [PubMed]  [DOI]
33.  Holt MK, Vivolo-Kantor AM, Polanin JR, Holland KM, DeGue S, Matjasko JL, Wolfe M, Reid G. Bullying and suicidal ideation and behaviors: a meta-analysis. Pediatrics. 2015;135:e496-e509.  [PubMed]  [DOI]
34.  van Geel M, Vedder P, Tanilon J. Relationship between peer victimization, cyberbullying, and suicide in children and adolescents: a meta-analysis. JAMA Pediatr. 2014;168:435-442.  [PubMed]  [DOI]
35.  Kim YS, Leventhal B. Bullying and suicide. A review. Int J Adolesc Med Health. 2008;20:133-154.  [PubMed]  [DOI]
36.  Tharp-Taylor S, Haviland A, D’Amico EJ. Victimization from mental and physical bullying and substance use in early adolescence. Addict Behav. 2009;34:561-567.  [PubMed]  [DOI]
37.  Goebert D, Else I, Matsu C, Chung-Do J, Chang JY. The impact of cyberbullying on substance use and mental health in a multiethnic sample. Matern Child Health J. 2011;15:1282-1286.  [PubMed]  [DOI]
38.  Niemelä S, Brunstein-Klomek A, Sillanmäki L, Helenius H, Piha J, Kumpulainen K, Moilanen I, Tamminen T, Almqvist F, Sourander A. Childhood bullying behaviors at age eight and substance use at age 18 among males. A nationwide prospective study. Addict Behav. 2011;36:256-260.  [PubMed]  [DOI]
39.  Vieno A, Gini G, Santinello M. Different forms of bullying and their association to smoking and drinking behavior in Italian adolescents. J Sch Health. 2011;81:393-399.  [PubMed]  [DOI]
40.  Nansel TR, Craig W, Overpeck MD, Saluja G, Ruan WJ. Cross-national consistency in the relationship between bullying behaviors and psychosocial adjustment. Arch Pediatr Adolesc Med. 2004;158:730-736.  [PubMed]  [DOI]
41.  Nakamoto J, Schwartz D. Is peer victimization associated with academic achievement? A meta-analytic review. Soc Dev. 2010;19:221-242.  [PubMed]  [DOI]
42.  Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264-269, W64.  [PubMed]  [DOI]
43.  Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008-2012.  [PubMed]  [DOI]
44.  Wells G, Shea B, O’connell D, Peterson J, Welch V, Losos M, Tugwell P.  The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa: Ottawa Hospital Research Institute, 2000. [Accessed 2016; Feb] Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.  [PubMed]  [DOI]
45.  Norman RE, Byambaa M, De R, Butchart A, Scott J, Vos T. The long-term health consequences of child physical abuse, emotional abuse, and neglect: a systematic review and meta-analysis. PLoS Med. 2012;9:e1001349.  [PubMed]  [DOI]
46.  Barendregt J, Doi SA.  MetaXL version 2.1 [computer program]. Brisbane: EpiGear International 2012; .  [PubMed]  [DOI]
47.  Doi SA, Barendregt JJ, Mozurkewich EL. Meta-analysis of heterogeneous clinical trials: an empirical example. Contemp Clin Trials. 2011;32:288-298.  [PubMed]  [DOI]
48.  Doi SA, Thalib L. A quality-effects model for meta-analysis. Epidemiology. 2008;19:94-100.  [PubMed]  [DOI]
49.  Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965;58:295-300.  [PubMed]  [DOI]
50.  World Cancer Research Fund American Institute for Cancer Research. Food, Nutrition, Physical Activity, and the Prevention of Cancer: a Global Perspective. Washington, DC: AICR 2007; .  [PubMed]  [DOI]
51.  Jansen DE, Veenstra R, Ormel J, Verhulst FC, Reijneveld SA. Early risk factors for being a bully, victim, or bully/victim in late elementary and early secondary education. The longitudinal TRAILS study. BMC Public Health. 2011;11:440.  [PubMed]  [DOI]
52.  Von Schoon I, Montgomery SM. The relationship between early life experiences and adult depression. Z Psychosom Med Psychoanal. 1997;43:319-333.  [PubMed]  [DOI]
53.  Brunstein Klomek A, Marrocco F, Kleinman M, Schonfeld IS, Gould MS. Bullying, depression, and suicidality in adolescents. J Am Acad Child Adolesc Psychiatry. 2007;46:40-49.  [PubMed]  [DOI]
54.  van der Wal MF, de Wit CA, Hirasing RA. Psychosocial health among young victims and offenders of direct and indirect bullying. Pediatrics. 2003;111:1312-1317.  [PubMed]  [DOI]
55.  Due P, Holstein BE, Lynch J, Diderichsen F, Gabhain SN, Scheidt P, Currie C. Bullying and symptoms among school-aged children: international comparative cross sectional study in 28 countries. Eur J Public Health. 2005;15:128-132.  [PubMed]  [DOI]
56.  Due P, Hansen EH, Merlo J, Andersen A, Holstein BE. Is victimization from bullying associated with medicine use among adolescents? A nationally representative cross-sectional survey in Denmark. Pediatrics. 2007;120:110-117.  [PubMed]  [DOI]
57.  Björkqvist K. Social defeat as a stressor in humans. Physiol Behav. 2001;73:435-442.  [PubMed]  [DOI]
58.  Watt MJ, Burke AR, Renner KJ, Forster GL. Adolescent male rats exposed to social defeat exhibit altered anxiety behavior and limbic monoamines as adults. Behav Neurosci. 2009;123:564-576.  [PubMed]  [DOI]
59.  Koolhaas J, Hermann P, Kemperman C, Bohu sB, Van Den Hoofdakker R, Beersma D. Single social defeat in male rats induces a gradual but long-lasting behavioral change: A model of depression. Neurosci Res Commun. 1990;7:35-41.  [PubMed]  [DOI]
60.  Anda RF, Felitti VJ, Bremner JD, Walker JD, Whitfield C, Perry BD, Dube SR, Giles WH. The enduring effects of abuse and related adverse experiences in childhood. A convergence of evidence from neurobiology and epidemiology. Eur Arch Psychiatry Clin Neurosci. 2006;256:174-186.  [PubMed]  [DOI]
61.  Shonkoff JP, Boyce WT, McEwen BS. Neuroscience, molecular biology, and the childhood roots of health disparities: building a new framework for health promotion and disease prevention. JAMA. 2009;301:2252-2259.  [PubMed]  [DOI]
62.  Copeland WE, Wolke D, Lereya ST, Shanahan L, Worthman C, Costello EJ. Childhood bullying involvement predicts low-grade systemic inflammation into adulthood. Proc Natl Acad Sci USA. 2014;111:7570-7575.  [PubMed]  [DOI]
63.  Arseneault L, Cannon M, Fisher HL, Polanczyk G, Moffitt TE, Caspi A. Childhood trauma and children’s emerging psychotic symptoms: A genetically sensitive longitudinal cohort study. Am J Psychiatry. 2011;168:65-72.  [PubMed]  [DOI]
64.  Arseneault L, Milne BJ, Taylor A, Adams F, Delgado K, Caspi A, Moffitt TE. Being bullied as an environmentally mediated contributing factor to children’s internalizing problems: a study of twins discordant for victimization. Arch Pediatr Adolesc Med. 2008;162:145-150.  [PubMed]  [DOI]
65.  Arseneault L, Walsh E, Trzesniewski K, Newcombe R, Caspi A, Moffitt TE. Bullying victimization uniquely contributes to adjustment problems in young children: a nationally representative cohort study. Pediatrics. 2006;118:130-138.  [PubMed]  [DOI]
66.  Brent BK. Increased risk of psychotic symptoms at 12 years in children who have been maltreated by adults or bullied by peers. Evid Based Ment Health. 2011;14:65.  [PubMed]  [DOI]
67.  von Rueden U, Gosch A, Rajmil L, Bisegger C, Ravens-Sieberer U. Socioeconomic determinants of health related quality of life in childhood and adolescence: results from a European study. J Epidemiol Community Health. 2006;60:130-135.  [PubMed]  [DOI]
68.  Currie J. Healthy, wealthy, and wise: Socioeconomic status, poor health in childhood, and human capital development. J Econ Lit. 2009;47:87-122.  [PubMed]  [DOI]
69.  Dubben HH, Beck-Bornholdt HP. Systematic review of publication bias in studies on publication bias. BMJ. 2005;331:433-434.  [PubMed]  [DOI]
70.  Egger M, Smith GD. Misleading meta-analysis. BMJ. 1995;310:752-754.  [PubMed]  [DOI]
71.  Jepsen P, Johnsen SP, Gillman MW, Sørensen HT. Interpretation of observational studies. Heart. 2004;90:956-960.  [PubMed]  [DOI]
72.  Fekkes M, Pijpers FI, Verloove-Vanhorick SP. Bullying behavior and associations with psychosomatic complaints and depression in victims. J Pediatr. 2004;144:17-22.  [PubMed]  [DOI]
73.  McMahon EM, Reulbach U, Corcoran P, Keeley HS, Perry IJ, Arensman E. Factors associated with deliberate self-harm among Irish adolescents. Psychol Med. 2010;40:1811-1819.  [PubMed]  [DOI]
74.  Scott JG, Moore SE, Sly PD, Norman RE. Bullying in children and adolescents: a modifiable risk factor for mental illness. Aust N Z J Psychiatry. 2014;48:209-212.  [PubMed]  [DOI]