How does a real-world child psychiatric clinic diagnose and treat attention deficit hyperactivity disorder?

Abstract

AIM: To investigate child and adolescent psychiatrists’ (CAPs) attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) diagnoses and treatments in real-world clinical practice.

METHODS: The medical records of 69 ADHD children (mean age = 9.5 years), newly referred to the ADHD clinic, were reviewed for their scores of parent- and teacher-reported Vanderbilt ADHD Diagnostic Rating Scales (VADRSs), CAPs’ diagnoses of ADHD and ODD, and CAPs’ treatment recommendations. Among 63 ADHD subjects who completed both parent and teacher VADRSs, we examined the agreement of the parent and teacher VADRSs. We also examined the concurrent validity of CAPs’ ODD diagnoses against the results from the VADRSs. In addition, we compared CAPs’ treatment recommendations against established ADHD and ODD guidelines.

RESULTS: Among 63 ADHD subjects, the majority of the subjects (92%) met full ADHD diagnostic criteria at least in one setting (parent or teacher) on the VADRSs. Nearly half of the patients met full ADHD diagnostic criteria in two settings (parent and teacher). Relatively low agreement between the parent and teacher VADRSs were found (95%CI: -0.33 to 0.14). For 29 children who scored positive for ODD on the rating scales, CAPs confirmed the ODD diagnosis in only 12 of these case-positives, which is considered as a fair agreement between CAPs and VADRSs (95%CI: 0.10-0.53). For 27 children with no ODD diagnosis made by either CAP or VADRS, more than half of them were recommended for medication only. In contrast, where CAPs made the diagnosis of ODD, or where the parent or teacher VADRS was positive for ODD, almost all of the patients received recommendations for medication and behavior therapy.

CONCLUSION: CAPs’ ADHD diagnoses have strong concurrent validity against valid rating scales, but ADHD’s most common comorbid condition - ODD - may be under-recognized.

Key Words: Attention deficit hyperactivity disorder, Oppositional defiant disorder, Vanderbilt attention deficit hyperactivity disorder Diagnostic Rating Scale, Quality assessment, Clinical practice

Core tip: Given the concerns about possible attention deficit hyperactivity disorder (ADHD) over-diagnosis and over-treatment, within a newly diagnosed sample of consecutive ADHD patients, we examined the concurrent validity of child and adolescent psychiatrists’ (CAPs) ADHD and oppositional defiant disorder (ODD) diagnoses against the results from the Vanderbilt ADHD Diagnostic Rating Scales. We also evaluated CAPs’ ADHD and ODD treatment recommendations and discussed clinical implementations of the established treatment guidelines into CAPs’ practice. In our samples, CAPs diagnosed ADHD strongly agreeing with the rating scales, but given our results showing the relatively low prevalence rates of ODD diagnosis within ADHD, ODD may be under-recognized.

INTRODUCTION

In view of national concerns about the rising rate of diagnoses of attention deficit hyperactivity disorder (ADHD), increased understanding of the diagnostic procedures and accuracy of clinicians’ diagnoses of ADHD and its associated conditions is needed[1,2]. Although diagnostic criteria for ADHD and other psychiatric disorders are clearly specified in the DSM-5; Diagnostic and Statistical Manual of Mental Disorders 5^th ed[3], it is less clear whether clinicians use these criteria in clinical practices. Relatedly, ADHD frequently occurs with comorbid conditions, particularly oppositional defiant disorder (ODD), its most common comorbidity with 54% to 84% of prevalence in ADHD patients reported in the American Academy of Child and Adolescent Psychiatry (AACAP) practice parameter[4,5]. However, relatively lower prevalence of ODD in ADHD children has been reported in different samples most likely due to heterogeneity of outcome measures[6]. Given the greater risks for substance use, academic disability, and social dysfunction in ADHD children comorbid with ODD[7,8], identification and accurate diagnosis of ODD in the early stages is essential, to improve the prognosis[5,9,10].

However, implementing these guidelines in real-world clinical practice can be a time-consuming task[4,5,10,11]. To make the process more practical and feasible, many rating scales comprised of the DSM diagnostic criteria have been published as valid assessment tools for both ADHD and ODD[5,10]. Among many ADHD rating scales, the Vanderbilt ADHD Diagnostic Rating Scale (VADRS), published in 2002 by the AAP and National Institute for Children’s Healthcare Quality, was designed to capture standardized ADHD symptom information from parents and teachers reporting on children’s behaviors[12]. This rating scale was also designed to assist providers in screening ODD and other common comorbidities[13,14].

Furthermore, the parent VADRS has shown high concurrent validity with ADHD diagnoses (γ = 0.79) made via structured diagnostic interviews (C-DISC-IV) in elementary school children[12,15]. Recently, Wolraich et al[12] evaluated the validities of parent and teacher VADRS in a larger sample of children and concluded that the agreement between the teacher VADRS’s symptoms and DISC-IV was statistically significant (P < 0.05; inattention γ = 0.33, hyperactive γ = 0.29), but of less magnitude of agreement compared to the parent VADRS with the DISC-IV[16,17]. The parent and teacher VADRSs have also been evaluated for their utility in assessing comorbid conditions such as ODD/CD and learning disorders[13,14]. Similarly, Becker et al[13] noted that a total score > 10 of the 8 ODD items on the parent VADRS demonstrated high sensitivity and specificity (0.88, 0.85 respectively) against C-DISC-IV diagnoses, with 91% of true positive cases identified.

The AAP and AACAP guidelines for ADHD[4,5] recommend that ADHD diagnoses be made via an intensive process that requires a careful diagnostic interview and review of other records (e.g., school evaluations, other medical reports, rating scales, etc.). Ultimately, the physician’s “best clinical judgment” must integrate all this information to render the final diagnoses. Despite of the importance of the physician’s “best clinical judgment” final step, little research has examined clinicians’ best judgments in diagnosing ADHD and ODD[7,8,18], comparing clinicians’ diagnostic judgments with the results from parent and teacher rating scales with a validated measure such as the VADRS.

Assuming that the correct diagnosis has been rendered, additional questions arise: Do physicians’ recommendations follow treatment guidelines in clinical practice of ADHD? While ADHD guidelines generally recommend medication as a first-line treatment, combining medication with behavior therapy is frequently recommended and is the best option for most cases. Behavior therapy is a first-line intervention for ODD, which is also advisable as an initial approach for some ADHD cases, such as younger children with ADHD. Behavior therapy promotes positive parenting and provides an opportunity for social training for disruptive children, which may not be achieved by medication treatment only[2,19-24].

Given previous research showing the high agreement of specific VADRS thresholds against DISC-derived diagnoses[13,16,17], we hypothesized that physicians’ clinical diagnoses of ADHD and ODD might be evaluated by comparing the results of the parent and/or teacher VADRSs with clinicians’ final diagnosis of these conditions. We also reasoned that we could evaluate physicians’ treatment recommendations by comparing them with the treatment guidelines for ADHD and ODD.

MATERIALS AND METHODS

Sampling frame

From June 1 to November 30, 2011, all records of pediatric patients newly referred to the ADHD Clinic at the Mayo Clinic in Rochester, Minnesota for initial evaluation were identified (n = 120, age range 3-18, and mean 9.4 years). Of these, 69 patients were selected through the inclusion and exclusion criteria described below.

Inclusion criteria

Patients, living in the immediate geographic area, who received their routine care either within the Mayo Clinic or its larger regional system, Mayo Clinic Health System, were included. Across the system, electronic medical records were available for examination by the study team. Thus, to be eligible, patients must have had an initial evaluation by CAPs within the ADHD clinic, and received their follow-up care either within the ADHD clinic or in the surrounding Mayo primary care settings.

Exclusion criteria

All patients who were followed up by non-Mayo Clinic providers were excluded (40 subjects). All patients not receiving an ADHD diagnosis after clinical evaluation were excluded as this study was a part of the naturalistic study to assess clinical implementations of VADRSs for diagnoses and treatment outcomes in ADHD children in the ADHD clinic. No comorbid conditions were used as exclusion criteria.

Records abstraction

Eight months after their initial diagnostic visit, patients’ medical records were reviewed in order to gather diagnostic and treatment-related data, i.e., ADHD and comorbid diagnoses, medications, therapies, and VADRS scores.

Sample characteristics

The mean age of the 69 children identified with ADHD was 9.5 years (age range, 4-18). As expected, males predominated, with a male/female ratio of 2.8:1. Eighty-two percent were Caucasians, 6% were African-American, 3% were Asian, and 6% were another ethnicity. Expected proportions of the various ADHD subtypes were found (combined subtype 41%, inattentive subtype 32%), except that nearly one-quarter of children were diagnosed with ADHD - not otherwise specified.

VADRS

As a part of its standard of care, the Parent and Teacher VADRS were used to assist in the assessment of ADHD symptoms and comorbid conditions at initial visits to the ADHD clinic. Among 69 patients, all of them had a completed parent VADRS at the initial visit, except for two incomplete ODD ratings. Six patients did not have teacher VADRS available for scoring at the initial visit. Among 63 subjects who had both parent and teacher VADRS completed, 59 subjects had completed parent and teacher ODD rating scales available for scoring. Accepted procedures for scoring the VADRS were used[25] as follows: On the VADRS, all ADHD and ODD symptoms were rated on a 0 to 3 scale (0 = none, 1 = minimal, 2 = often, 3 = very often). For any symptom to be considered “positive” for diagnostic purposes, it must be scored at a 2 or 3 (often, very often). When scoring the parent and teacher VADRS for diagnostic purposes, each of the two ADHD symptom subtypes (inattentive and hyperactive-impulsive) is considered to be screen-positive if > 6 of 9 of their respective symptoms are scored at 2 or 3. Additionally, the ADHD and ODD scores in each rating scale were added up to calculate the total ADHD and ODD scores (range of total possible ADHD symptom scores: 0-54; range of possible ODD symptom scores: 0-24 Parent VADRS, 0-30 Teacher VADRS). We applied the cut point of a total score > 10 of the eight ODD items on the parent VADRS considering its high sensitivity and specificity for a screening of ODD. However, for teacher VADRS, ODD is considered to be screen-positive when > 3 of 10 of their ODD symptoms are scored at 2 or 3 on the teacher VADRS.

Additional items on the VADRS also ascertain the presence or absence of impairment in functioning, another pre-requisite before making any ADHD or ODD diagnoses. Both the parent and teacher VADRSs have eight items to assess children’s performance levels, which are rated on a 1 to 5 scale (1 = excellent, 2 = above average, 3 = average, 4 = somewhat of a problem, 5 = problematic). The performance domains in the parent rating scale range from academic performance of reading, writing, and mathematics to social skills by assessing relationship qualities with parents, siblings, and peers as well as behavioral skills to attend organized activities. The performance domains in the teacher rating scale similarly include academic performance of reading, mathematics, and written expression, as well as social skills in terms of relationships with peers, but also behavioral skills such as following directions, not disrupting class, assignment completion and organizational skills. To consider the performance score as a functional impairment either at home or school, at least one item among 8 items must be scored at a 4 or 5 (somewhat of a problem, problematic).

In summary, we considered it as a positive diagnostic result when either parent or teacher’s VADRS was positive (> 6 of 9 on either or both of the inattentive and hyperactive-impulsive subscales), along with the presence of impairment. For an ODD screening, either a total ODD score > 10 on parent VADRS or > 3 of 10 ODD items scored at 2 or 3 on teacher VADRS.

Clinical process to diagnose ADHD and ODD

As a part of the initial assessments, comprehensive assessments were conducted by board-certified child and adolescent psychiatrists (CAPs), capturing information from the child’s guardians pertaining to the child’s growth and development, medical and psychosocial history, school history and performance, test scores, and mental status examinations, as well as semi-structured diagnostic interviews to the parents for ADHD and related conditions, using DSM-IV criteria. The CAPs were free to examine the results of both the parents and teachers VADRS, which were collected not only at the initial evaluation but throughout the 8-mo period to track the child’s symptoms and treatment response. No steps were taken to require CAPs to include the VADRS results as a part of their final diagnostic considerations, however.

Analytic approach

For all subjects, we determined the number and proportion of them who met criteria for ADHD and ODD in either a parent or teacher rating scale as well as in both of the scales, based on the recognized scoring criteria for parent and teacher VADRS. We also calculated the number of symptom criteria (items scored at 2 or 3) for ADHD and ODD scales as well as the numerical totals for ADHD and ODD symptoms scales, reporting separate counts for both parent and teacher informants. Then, we computed cross-tabulations of psychiatric diagnoses of ADHD and ODD, referencing CAPs’ diagnoses against ADHD and ODD diagnoses derived from the parent and teacher VADRS scores. Cohen’s Kappa was calculated to evaluate the degree of concordance between the VADRS and CAP diagnoses. To understand diagnostic discrepancies between CAP and VADRS-derived diagnoses, we further examined ADHD and ODD total scores as well as the numbers of impairment domains from parent and teacher VADRS, comparing groups of patients with/without CAP diagnosis of ODD, using ANOVAs and t-tests to evaluate these subgroup differences in VADRS scores. Finally, to evaluate whether CAPs’ treatment practices are consistent with recognized guidelines for management of ADHD and ODD, we compared their recommendations for medication and/or behavior therapy as a function of the presence or absence of ODD, examining these recommendations against CAP-rendered ODD diagnoses vs VADRS-rendered ODD diagnoses.

RESULTS

ADHD diagnosis and the VADRS

As hypothesized, an initial comparison of parent and teacher VADRS scores indicates that a majority of CAP-diagnosed subjects (92%) met full ADHD diagnostic criteria on either the parent or teacher VADRS. Nearly half of the children (44%) met ADHD criteria on both the parent and teacher VADRSs (Table 1). Five children did not meet full DSM criteria regardless of the informant. A single chi-square for the overall table was χ² = 0.6, df = 1, P = 0.5. The observed percentage agreement between parent and teacher VADRSs was 52.4%, slightly less than expected percentage agreement by chance alone (56.4%). Calculated agreement by Cohen’s Kappa was -0.09, indicating “disagreement” between parents and teachers (95%CI: -0.33 to 0.14).

Table 1 Comparison of vanderbilt attention deficit hyperactivity disorders Diagnostic Rating Scales parent vs teacher results n (%).

		Parent vanderbiltrating scale		Total
		Negative	Positive
Teacher Vanderbilt	Negative	5 (8)	17 (27)	22 (35)
Rating scale	Positive	13 (21)	28 (44)	41 (65)
Total		18 (29)	45 (71)	63 (100)

Of note, DSM requires that a child meets full diagnostic criteria, with sufficient symptoms and impairment in at least one setting, and that the child also presents several ADHD symptoms in at least one other setting. Thus, for 28 of 63 subjects where both parent and teacher rating scales were positive, the child psychiatrists’ diagnoses were strongly supported, with evidence of concurrent validity with strong evidence of functional impairments in both parent and teacher impairment domains (Table 2)[5]. Of note, there were 30 cases in Table 1 with only one of the two informants’ ratings met full diagnostic symptom criteria (17 parent positive/teacher negative; 13 parent negative/teacher positive).

Table 2 Total attention deficit hyperactivity disorder scores, parent and teacher Vanderbilt attention deficit hyperactivity disorder diagnostic rating scale by positive vs negative diagnostic results (n = 63).

Vanderbilt rating scale	ADHD	n	ParentVADRS total ADHD score	TeacherVADRS total ADHD score	No. of parent’s impairment domains	No. of teacher’s impairment domains
Parent	Negative	5	16.6	10.2	3.4	1.8
Teacher	Negative				(SD ± 0.9)	(SD ± 1.5)
Parent	Negative	13	26.4	32.6	1.1	4.4
Teacher	Positive				(SD ± 0.5)	(SD ± 1.9)
Parent	Positive	17	33.7	16.6	2.9	1.7
Teacher	Negative				(SD ± 0.4)	(SD ± 1.6)
Parent	Positive	28	37.3	34.0	3.7	5.7
Teacher	Positive				(SD ± 1.8)	(SD ± 1.6)
Statistics			11.3	21.4	7.5	24.7
F ratio, df = 3			(P < 0.00011)	(P < 0.00011)	(P < 0.00031)	(P < 0.00011)
Total mean score (%)			32.4	27.1	2.9	4.0

¹Statistically significant at a significance level of 0.05. ADHD: Attention deficit hyperactivity disorder; VADRS: Vanderbilt ADHD diagnostic rating scale.

To better understand whether these cases showed sufficient evidence of a presence of several ADHD symptoms in at least two settings, we computed the parent and teacher total ADHD scores (adding all 18 ADHD items) for all groups as a function of positive/negative VADRS scores. The results in Table 2 indicate that even when one informant shows “negative results”, that informant’s total scores are still highly elevated, generally just below diagnostic threshold, providing supportive evidence of concurrent validity of CAPs’ ADHD diagnoses for these 30 cases.

ODD diagnosis and the VADRS

Among 69 ADHD patients, 59 subjects had completed ADHD and ODD rating scales. Twenty-nine children scored positive for ODD on the VADRS, but CAPs confirmed the ODD diagnosis in only 12 (less than half) of these case-positives, as well as in three additional ODD-negative children. As seen in Table 3, the actual number of observed agreements between CAP and VADRS was 39 (66%), a slight improvement in the number of agreements expected by chance. The calculated agreement by Cohen’s Kappa was 0.32, which is considered “fair” (95%CI: 0.10-0.53). A single χ² of the over all table was χ² = 7.7 (P = 0.0057).

Table 3 Comparison of psychiatrists’ diagnoses of oppositional defiant disorder with parent or teacher Vanderbilt attention deficit hyperactivity disorder Diagnostic Rating Scale n (%).

		Psychiatrist ODDdiagnosis		Total
		Negative	Positive
Vanderbilt	Negative	27 (46)	3 (5)	30 (51)
ODD
screening	Positive	17 (29)	12 (20)	29 (49)
Total		44 (75)	15 (25)	59

ODD: Oppositional defiant disorder.

To better understand the discrepancies seen in the VADRSs’ and CAPs’ ODD diagnoses (Table 3), we calculated the total ODD scores from the parents and teachers’ VADRS, comparing their mean total scores across the 4 ODD diagnostic agreement/disagreement categories. Four parents’ total ODD scores and two teachers’ total ODD sores were missing for incomplete scoring. As seen in Table 4, when comparing parent and teacher total ODD scores, parents rated children with more ODD symptoms than teachers across all of the agreement/disagreement categories. As expected, when both the CAP and the VADRS scales were in agreement for a positive ODD diagnosis (12 cases, fourth row), total parents and teachers’ VADRS scores were elevated. However, surprisingly, another 17 cases had similarly elevated parent ODD scores and evident functional impairments on the VADRS, but they were not diagnosed by CAPs with ODD, a particularly surprising finding in view of the fact that CAPs did diagnose 5 additional subjects with ODD, even though their average parents and teachers’ ODD scores and number of impairment domains were generally lower than the aforementioned 17 subjects.

Table 4 Total parent and teacher oppositional defiant disorder scale scores, grouped by positive/negative oppositional defiant disorder diagnostic criteria, child and adolescent psychiatrist vs vanderbilt attention deficit hyperactivity disorder diagnostic rating scale.

Diagnostic source	ODD	n	Parent VADRS total ODD score	Teacher VADRS total ODD score	No. of parent’s impairment domains	No. of teacher’s impairment domains
CAP	Negative	27	6.1	1.1	2.3	3.7
VADRS	Negative
CAP	Negative	17	14.2	4.0	3.7	3.9
VADRS	Positive
CAP	Positive	3	4.0	2.0	3.3	2.3
VADRS	Negative
CAP	Positive	12	15.2	10.3	3.2	5.4
VADRS	Positive
Statistics F Ratio, df = 3			24.9 (P < 0.00011)	12.0 (P < 0.00011)	2.4 (P = 0.07)	1.9 (P = 0.14)
Total mean score (%)			10.2 (10.2/24 = 43%)	3.8 (3.8/30 = 13%)	2.9	4.0

¹Statistically significant at a significance level of 0.05. ODD: Oppositional defiant disorder; CAP: Child and adolescent psychiatrist; VADRS: Vanderbilt ADHD diagnostic rating scales.

Treatment for ADHD Children with ODD symptoms

As seen in Table 5, medication therapy was recommended to almost all (54/59) patients regardless of their ADHD/ODD diagnostic groupings. For 27 patients who did not have ODD, neither by VADRS nor CAP diagnosis, a slight majority (15) received recommendations for medication treatment alone, with most of the remainder receiving recommendations for combined treatment (medication and behavior therapy). In contrast, for the three other diagnostic groups with either a positive VADRS for ODD, a CAP diagnosis of ODD or both, the major treatment recommendation made by CAPs was for combined treatment. A single chi-square for the overall table was Pearson χ² = 22.8, df = 6, P = 0.0009.

Table 5 Psychiatric treatment recommendations for attention deficit hyperactivity disorder patients with vs without oppositional defiant disorder comorbidity (n = 59).

Diagnostic source	ODD	n	Medication therapy only	Behavior therapy only	Combined treatment
CAP	Negative	27	15 (25%)	1 (2%)	11 (19%)
VADRS	Negative
CAP	Negative	17	3 (5%)	0 (0%)	14 (24%)
VADRS	Positive
CAP	Positive	3	0 (0%)	1 (2%)	2 (3%)
VADRS	Negative
CAP	Positive	12	0 (0%)	3 (5%)	9 (15%)
VADRS	Positive
Total (%)			18 (30%)	5 (9%)	36 (61%)

ODD: Oppositional defiant disorder; CAP: Child and adolescent psychiatrist; VADRS: Vanderbilt ADHD Diagnostic Rating Scales.

To examine this relationship, we did a posthoc analysis using a 2 × 2 contingency table to examine the frequency of behavioral therapy recommendations (either alone or with combined treatment) as a function of the presence/absence of an ODD diagnosis; thus, 29 of 32 (91%) subjects with a possible ODD diagnosis received a treatment recommendation that included behavior therapy vs only 12 of 27 (44%) subjects without a possible ODD diagnosis (Pearson’s χ² = 14.7, P = 0.0001).

DISCUSSION

Our study examined the CAPs diagnostic practices and treatment recommendations for ADHD and its most common comorbidity, ODD, comparing their clinical diagnoses with the results of parent- and teacher-completed VADRS. We also examined CAPs’ treatment recommendations against current ADHD treatment guidelines. To our best knowledge, this is the first study to examine CAPs’ decision-making processes in a real-world, clinical practice setting where highest quality, specialized care for ADHD might be expected.

ADHD diagnostic findings

As anticipated, in this sample of children presenting to the ADHD clinic and diagnosed with ADHD by child psychiatrists, 91% of subjects had completed parent and teacher VADRSs available for scoring at the initial visit. Among them, the vast majority of subjects (92%, 58 of 63) met full ADHD diagnostic criteria in at least one setting (parent or teacher) on the rating scales (Table 1). Nearly half of the patients even met full ADHD diagnostic criteria in more than one setting (e.g., home and school). Of note, in the case of ADHD, five conditions must be met before a diagnosis should be made: (1) The child must manifest sufficient symptoms of inattention and/or hyperactivity-impulsivity; (2) several ADHD symptoms must be present in at least two settings (e.g., home, school, with peers, etc.); (3) the symptoms must be of sufficient duration (> 6 mo); (4) symptoms must begin during childhood (DSM-IV - before age 7, DSM-5 - before age 12); and (5) other likely or possible explanations for the symptoms must be ruled out during the evaluation (e.g., vision or hearing problems, child abuse, family chaos, etc.). Although the DSM diagnostic criteria require that a child meets the symptom criteria threshold (e.g., at least six of nine symptoms), they do not specify that the minimum six be met in more than one setting. Instead, the criteria require the presence of several ADHD symptoms in more than one setting. This two-setting requirement serves to eliminate “setting-specific” ADHD, which may be associated more with environmental factors rather than intrinsic factors.

To examine whether the 30 children evincing full ADHD symptoms in only one setting had some ADHD symptoms in one other setting, we examined the parents and teachers’ total ADHD scores, comparing those children with potentially “setting specific” ADHD to those meeting full criteria in both home and school settings, and finding comparably high symptom levels in all groups (Table 2). These findings provide supportive evidence for the concurrent validity of CAPs’ ADHD diagnoses. Furthermore, the fact that similar numbers of patients were identified as meeting full criteria by either parents (n = 17, 27%) or teachers (n = 13, 21%), may reflect equivalent referral pressures from both sources, and does not support uninformed claims as seen in the media that ADHD is due to single causes, such as bad parents or unskilled teachers. Furthermore, among these 30 subjects, parents or teachers reported ADHD symptom scores were associated with the scores of performance domains reported by the same informant. In ADHD subjects with both screenings positive on the parents and teachers’ VADRSs, their functions were severely impaired across the settings. Indeed, previous studies showed that ADHD symptoms were associated with executive function (EF) impairments, which are closely related to performance levels in daily activities in both home and school settings[26]. Yet, we should not ignore the variable behaviors reported by different sources in different settings, which are most likely due to interactions of intrinsic factors and environmental factors such as parent-child relationship qualities or expectation levels of parents or teachers. The considerations of these environmental factors could be more important when providers organize individualized treatment plans. Karpenko et al[27] concluded that without having significant treatment response to ADHD symptoms, some of the functional domains still improved reliably. This study suggests that treatment goals should focus not only on ADHD symptoms but also on better functioning in different settings[27].

Only 8% of the patients (n = 5) had both negative parent and teacher ADHD VADRS scores. Among these five cases, an in-depth review indicated that most of these patients were classified as ADHD, inattentive subtype, and tended to be comorbid with learning disorders or other chronic diseases. The parents of these five subjects reported relatively high impairments in their children’s functions at home.

ODD diagnostic findings

Our findings related to CAPs’ ODD diagnoses raise interesting questions: In this ADHD sample, CAPs diagnosed ODD in one-fourth of cases, substantially less than the expected 54% to 84% prevalence rates of ODD found within ADHD patients reported in other clinical sample studies[4]. In contrast to CAPs’ diagnoses, nearly half of our subjects had a positive ODD screening result in either the parent or teacher VADRS, consistent with the possible conclusion that CAPs under-diagnosed ODD in the patients referred to the clinic.

Further evaluation of the 17 cases with a positive VADRS ODD rating scale but not diagnosed with ODD by CAPs revealed that these cases had much higher ODD rating scale scores on the parent VADRS than on the teacher VADRS (Table 4). Although the teacher VADRS ODD scores were lower in these cases, they were nonetheless quite comparable to the average teacher scores reported in three cases where CAPs did make the diagnosis of ODD despite negative ODD screening results. Additionally, the impairment domains reported by parents and teachers were similarly high in all groups, which is an interesting finding considering the association of ADHD score results and performance levels.

In reviewing these findings, we considered the possibility that if only the parents experience disruptive symptoms in their children’s behaviors, CAPs might consider the disruptive symptoms to reflect problems in family functioning and parenting skills vs a disorder more intrinsic to the child. Another possible consideration that might lead to ODD “under-diagnosis” could be that CAPs do not follow ODD diagnostic criteria per se and are reluctant to make the diagnosis if teachers do not also highly rate ODD symptoms. The fact that most of the CAP-diagnosed, ODD-positive cases had both high parent and high teacher ODD symptom scores (Table 4) is consistent with these interpretations.

To further understand these findings, we considered the possibility that CAPs might tend to dismiss the possibility of an ODD diagnosis if children’s ODD symptoms were noted only (or principally) by parents rather than by teachers. To examine this possibility, we conducted a posthoc analysis to assess the likelihood of CAPs making an ODD diagnosis with vs without a positive teacher VADRS ODD score. Accordingly, these analyses indicated that CAPs made the ODD diagnosis in 57% (4 of 7) cases where the teacher VADRS ODD score was positive, in 36% (8 of 22) cases with only a positive parent VADRS ODD score, and in 10% (3 of 30) cases with neither parent nor teacher positive VADRS ODD scores (Pearson χ² = 8.9, df = 2, P = 0.01).

These results suggest that negative ODD screening results from the parent and teacher VADRS has high specificity (27/30 = 90%) in ruling out ODD among patients with ADHD, consistent with earlier reports[13]. However, in this study, CAPs do not appear to rely on parent VADRS ODD screening results to rule ODD in, but do appear to place significant weight on teachers’ ODD screening results - a possible decisional component that is NOT found in the DSM criteria. Moreover, the generally low prevalence rate of ODD, as diagnosed by CAPs in this study, raises questions for future research about how, when, and why CAPs make the diagnosis of ODD.

Treatment recommendation findings

In regard to treatment recommendations, for 27 children with no ODD diagnosis made by either CAP or VADRS, more than half of them were recommended for medication only, which can be seen as a conservative approach for an academic psychiatry center. In a previous study examining community-based pediatric care for ADHD, only 17% of children received behavior therapy[28]. In contrast, where CAPs made the diagnosis of ODD, or where the parent or teacher VADRS was positive for ODD, almost all of the patients received recommendations for combined therapy (medication plus behavior therapy), except for a few cases. In four cases, CAPs recommended behavior therapy only without medication for children with ODD symptoms - a group of patients consisting of younger children (mean age of 6.4 years, range 4 to 8 years).

In summary, our findings suggest that CAPs appear to follow diagnostic criteria for ADHD but not for ODD, given our results showing a relatively low prevalence rate of ODD diagnosis for ADHD, and discrepancies between CAPs’ ODD diagnoses, with positive ODD screening results on the VADRS. Despite CAPs’ apparent failure to make ODD diagnoses, for most of these children with ODD symptoms, whether detected by the CAPs or VADRS, CAPs nonetheless recommended behavior therapy. However, their failing to make ODD diagnoses when appropriate could lead others to underemphasize or even overlook the importance of the role of behavior therapy, e.g., neglecting the education of parents in understanding disruptive behaviors and learning necessary parenting skills to manage such behaviors.

Our findings suggest that the rating scales are important in real-world clinical practices as efficient and reliable means of obtaining valid information from both parents and teachers to assist CAPs in making better diagnoses of ADHD and its most common comorbidity, ODD.

We note several limitations intrinsic to this study. First, this was a retrospective study in which non-ADHD patients were excluded. Although we acquired highly completed parent and teacher VADRSs, collected at the initial evaluation in the ADHD clinic, and conducted rigorous medical-record evaluations to assess the accuracy of diagnoses in the ADHD patients, for all that, our results can only explain the accuracy of the diagnoses in the ADHD samples but not the reasons for which ADHD was not diagnosed in the non-ADHD samples. Second, our sample size was relatively small, and subjects were drawn from a single clinic, which may limit the generalizability of our findings. However, the fact that this study was conducted in an academic center psychiatric clinic, where practice standards presumably should be fairly uniform and high, suggests that future studies are needed across a broader range of clinics and clinical settings. Thus, additional research should further describe and evaluate CAP’s diagnoses and treatments for ADHD within larger samples and a wider range of community-based settings. Moreover, future studies should address these same issues within the outpatient practices of primary care providers, where even greater discrepancies might be noted.

CAPs’ ADHD diagnoses have strong concurrent validity against valid rating scales, but ADHD’s most common comorbid condition - ODD - may be under -recognized.

ACKNOWLEDGMENTS

Drs. Yuki and Bhagia have no conflicts to declare. Dr. Jensen has received research funding from NIMH, AHRQ, Marriott Foundation, Mayo Foundation, and REACH Institute (non-profit) and gained book royalties from Civic Research Institute, Guilford, APPI, Oxford, Free Spirit Publishing, and Random House. Dr. Jensen owns a stock in CATCH Services, Inc. Dr. Jensen has received charitable gifts from Shire, Inc.

COMMENTS

Background

The American Academy of Child and Adolescent Psychiatry and AAP guidelines for attention deficit hyperactivity disorders (ADHD) practices have emphasized the importance of accurately diagnosing ADHD and its comorbidities to provide appropriate treatment options for ADHD children. However, its most common comorbidity, oppositional defiant disorder (ODD), has been reported with a wide range of prevalence rates due to a heterogeneity of the research outcome measures of ODD symptoms. Because ADHD children comorbid with ODD are at greater risk for social and academic dysfunctions, diagnosing ODD within ADHD early on is critical to prevent the externalizing behaviors from progressing. Furthermore, implementations of the established diagnostic and treatment guidelines for ADHD practices have been examined in primary care settings but not in a setting of real-world child and adolescent psychiatric practice.

Research frontiers

To make the time-consuming diagnosing process more efficient and accurate, the parent- and teacher-reported Vanderbilt ADHD diagnostic rating scales, (VADRSs) known to have high concurrent validity with ADHD diagnoses (γ = 0.79), were introduced to ADHD clinical practices. It has been suggested that the current comorbidity-scoring system in the VADRSs is excellent at ruling out ODD/CD but not at ruling in ODD/CD. However, Becker et al[13] examined the utility of the VADRSs to assess ODD/CD in relation to ADHD and found that a total score > 10 of the 8 ODD items on the parent VADRS, which was not in the original scoring instruction, showed high sensitivity and specificity against C-DISC-IV diagnoses for ODD.

Innovations and breakthroughs

Given the previous studies’ results of high agreement of specific VADRSs thresholds against DISC-derived diagnoses, The authors compared the results of the parent and/or teacher VADRSs with clinicians’ final diagnoses of these conditions. The authors also reasoned the physicians’ treatment recommendations by comparing them with established treatment guidelines for ADHD and ODD practices. Theirs findings suggest that child and adolescent psychiatrists (CAPs) follow the guidelines for ADHD diagnoses and treatments but not entirely for ODD.

Applications

The VADRSs are an efficient tool to assist ADHD practices in diagnosing ADHD and ODD. CAPs and primary care physicians can gain more benefits through understanding the strengths and weaknesses of the VADRSs. Clinicians’ attentions to the ODD scorings will help to identify more ODD cases within ADHD. Further research to examine the best practice of ODD in ADHD children is required.

Terminology

ADHD: ADHD is a condition which has difficulties with attention, increased activity, and impulsivity; Cohen’s Kappa: Cohen’s Kappa is a statistic measure of the agreement between two groups who rate categorical data as agreed or disagreed, considering the agreement by chance alone; Concurrent validity: Concurrent validity is a measure of the extent to which a particular test correlates with a previously established measure; Confidence interval (CI): CI is a type of interval estimate of a population parameter; ODD: ODD is a condition which includes an ongoing pattern of defiant behavior toward authority figures that disturb the children’s daily functioning.

Peer-review

Very nice written paper with high clinical impact.