Minireviews Open Access
Copyright ©2014 Baishideng Publishing Group Inc. All rights reserved.
World J Psychiatr. Sep 22, 2014; 4(3): 49-55
Published online Sep 22, 2014. doi: 10.5498/wjp.v4.i3.49
Conducting clinical research in community mental health settings: Opportunities and challenges
Oleg V Tcheremissine, Whitney E Rossman, Manuel A Castro, Dineen R Gardner
Oleg V Tcheremissine, Whitney E Rossman, Manuel A Castro, Dineen R Gardner, Department of Psychiatry, Carolinas HealthCare System-Behavioral Health Center, Charlotte, NC 28203, United States
Author contributions: All the authors contributed to this paper.
Correspondence to: Oleg V Tcheremissine, MD, Department of Psychiatry, Carolinas HealthCare System-Behavioral Health Center, 501 Billingsley Road, Charlotte, NC 28203, United States.
Telephone: +1-704-4467504 Fax: +1-704-4467505
Received: March 20, 2014
Revised: June 24, 2014
Accepted: June 27, 2014
Published online: September 22, 2014


Tremendous progress has been made in the past decade surrounding the underlying mechanisms and treatment of neuropsychiatric disease. Technological advancements and a broadened research paradigm have contributed to the understanding of the neurochemistry, brain function and brain circuitry involved in neuropsychiatric disorders. The predominant area of unmet medical need in the United States is major psychiatric disorders, and major depressive disorder is the leading cause of disability for ages 15-44. Total spending on research and development by the pharmaceutical industry has grown exponentially during the past decade, but fewer new molecular entities (NME) for the treatment of major psychiatric disorders have received regulatory approvals compared to other therapeutic areas. Though significant expansion has occurred during the “decade of the brain”, the translation of clinical trials outcomes into the community mental health setting is deficient. Randomized controlled trials (RCTs) have been the standard approach to clinical evaluation of the safety and efficacy of NMEs for the past 60 years; however, there are significant barriers and skepticism in the implementation of evidence-based outcomes into clinical practice. Recruitment of patients, shortages of experienced clinical researchers, regulatory requirements and later translation of outcomes into clinical practice are ever growing problems faced by investigators. The community mental health setting presents particular barriers in the replication of therapeutic outcomes from RCTs. The diagnostic complexity of major psychiatric diseases and the highly selective patient populations involved in clinical trials lend to the gap in translation from the “bench to the bedside”. The community mental health setting lends to a diverse patient population with numerous co-morbidities and environmental factors that are unaccounted in the average RCT. While we acknowledge the enormous complexity in developing novel and innovative treatments for major psychiatric disorders, we must continue to improve the translatability of clinical trials to real world settings. Progress has been rather slow but as the gap in treatment effectiveness is reduced, so will costs and barriers in community mental health.

Key Words: Community mental health, Clinical trials in psychiatry, Study population

Core tip: Development of novel and effective pharmacological agents for the treatment of neuropsychiatric diseases is a long-standing challenge. Despite considerable investments into biomedical research from the pharmaceutical industry, academia and government organizations, the transformation of acquired knowledge from basic science and preclinical areas has been lacking. Randomized controlled trials (RCTs) remains the gold standard in drug development. Over the years, concerns have arisen regarding generalizability of RCT results into routine psychiatric care. Extending RCTs to clinical populations from the community mental health setting will support ecological validity and improve treatment outcomes.


The effective pharmacological treatment of neuropsychiatric diseases and development of new therapeutic entities has been a long-standing challenge. The greatest developments in understanding of brain circuitry, neurochemistry, and brain function gained during the “Decade of the Brain” from 1990 to 1999 provided new insights into underlying biological and physiological mechanisms of major neuropsychiatric disorders. Because of these achievements, considerable progress has been made over the past decade in broadening the treatment armamentarium in psychiatry by drastically increasing a number of psychotropic medications available to clinicians and their patients. Technological advancements in genomics, biomarkers, biosignatures, and imaging have contributed to a more detailed understanding of the etiological complexity of neuropsychiatric diseases but have yet impacted the early diagnosis, treatment, and prognosis. Furthermore, in most instances current diagnostic classifications do not address the complexity of neuropsychiatric diseases from the longitudinal perspectives (i.e., from the prodromal phase or subsyndromal state to the fully developed disease state), but rather define these heterogeneous diseases based on clusters of symptoms, often reducing these disorders to the level of monolithic constructs. The inherent limitation of this approach is further underscored by the fact that neuropsychiatric disorders are substantially more complicated than many other medical conditions. Although as many other diseases, such as hypertension or diabetes, neuropsychiatric diseases are epigenetic in nature, they are also context-driven and context-dependent phenomena. In essence, these diseases states, including depression, bipolar disorders, schizoaffective disorder, schizophrenia, anxiety disorders are the amalgam or the results of multilevel interactions between genetic predisposition or vulnerability in individuals interfaced with environmental factors and learning experiences. Given the complexity of these interactions and a lack of understanding the causes of these disorders, it is not surprising that the progress in the development of highly effective novel therapeutics was hindered over the years.

In the United States, major psychiatric disorders are the predominant area of unmet medical needs and the largest diagnostic category for Supplemental Security Income and Social Security Disability income with total cost of almost $25 billion per year[1]. In the sequenced treatment alternatives to relive depression (STAR*D) trial, over 60% of individuals treated with standard antidepressant (citalopram) did not achieved remission and only 30% of those receiving second-step augmentation achieved remission[2]. Similarly, in large national samples, only a third of individuals with schizophrenia achieved symptomatic and psychosocial remission[3,4]. Not surprisingly major depressive disorder (MDD) is the leading cause of disability in the United States for ages 15-44 and Schizophrenia is the third most common cause of disability for individuals age 15 and 45[5,6].

Despite considerable investments into biomedical research from the pharmaceutical industry, academia, and government organizations, the transformation of acquired knowledge from basic science and preclinical areas into drug development and clinical practice has been lacking. In the United States between the late 1990s and early 2000s, the budget of the National Institute of Health doubled, while the total spending of pharmaceutical industry on Research and Development grew from $2 billion in 1980 to $32 billion in 2002[7]. However, during the past decade, fewer new molecular entities (NME) for the treatment of major psychiatric disorders have received regulatory approvals compared to other therapeutic areas or disease stages[7]. In fact in the United States, in 2013, only 1 NME, vortioxetine, was approved by the Food and Drug Administration for the treatment of any of major psychiatric disorder[8]. New and improved technological abilities to isolate and study single molecules, cells and other components of biological systems have led to the dramatic growth of biological reductionism. This approach is guided by the hypothesis that a highly selective single-action molecular entity can produce desirable, clinically relevant outcomes through interactions with a single target in highly dynamic multifactorial neuronal networks of heterogeneous clinical populations of patients. This method, commonly utilized by the pharmaceutical industry for NME development, has been associated with limited results due to high attrition rates and poor predictive power[9].

The current situation in drug development is further confounded by rigorous oversight of governmental regulatory agencies, decreased revenues in pharmaceutical industry, unprecedented reduction of workforce across biotechnology and pharmaceutical companies, patent expirations, and globalization of clinical research. Most importantly, drug development programs are always conducted within the socioeconomic context and, consequently, could be influenced by geopolitical situations, economic incentives or disadvantages, business and marketing decisions, patent challenges and expirations, and successes or failures of competitors. On a practical level, there are growing concerns that advancements in neurosciences did not correspond to development of truly novel and innovative pharmacological approaches to the treatment of major neuropsychiatric disorders nor paved the way for the reduction of the socioeconomic burden of these disorders on patients, their families, communities, and the society at large. Arguably, the gap between “what we know” and “what we do” is getting wider[10].


For almost 60 years, randomized controlled trials (RCTs) in psychiatry have been considered the “gold standard” for evaluation of the safety, efficacy, and tolerability of NMEs[11]. Outcomes of RCTs are commonly employed to provide a scientific rationale for intended clinical indication utilized by regulatory agencies and serve as treatment guidelines and recommendations for evidence-based medicine. However, the way in which results from RTCs directly influence routine clinical care in psychiatry is complex. There are significant barriers in the implementation of outcomes from RCTs in evidence-based practice. These barriers are primarily related to an emergent skepticism about the overall probability of replicating therapeutic outcomes from RCTs in patients treated in the community mental health settings. The roots of this skepticism arise from the diagnostic complexity of major psychiatric diseases and the highly selective populations of patients involved in clinical trials. This fundamental difference between efficacy and effectiveness in clinical research represent the most delicate balancing act between the search for the ideal research conditions (internal validity) and real-world clinical settings (external validity). Broadly, validity is defined as truthfulness derived from research findings. Internal validity which extends the results of the experiment could reflect a causal relationship. The external validity pertains to the generalizability of the outcomes to populations, settings, and treatment valuables[12].

In recent years, concerns with regard to the generalizability of the outcomes from RTCs lead to the deliberate attempts to enhance external validity. Specifically through the conduct of ecological clinical trials, which apply inclusion/exclusion criteria, clinical settings, practice parameters and interventions that are more in line with real-world experiences of clinicians and patients.

In real-world clinical settings, patients with major psychiatric disorders have the tendency to meet multiple diagnostic criteria and often have a number of chronic medical conditions[13,14]. As a result, regardless of the mechanism of action of NME(s) and/or the intended indication(s), the clinical and commercial success of any drug development program heavily depends on the ability to clearly identify and recruit the most appropriate study population or subpopulation of patients.

In CNS, the target population for a specific RCT is defined by the diagnosis, severity of symptomatology, duration of illness, history of compliance with current and previous treatments, history of hospitalizations, co-morbid medical and neurological conditions and history of illicit drugs and alcohol abuse. Eligibility for study participation is based on inclusion and exclusion criteria developed to maximize the likelihood of detecting statistically significant differences between the NME and comparator, a standard therapeutic option for the intended indication and/or placebo. Such narrow criteria discards not only those affected by clinically relevant chronic medical conditions, such diabetes and hypertension, or those who are renally and hepatically impaired, but also those living in unstable environment and homeless, dual-diagnosed and current substance abusers, individuals with histories of poor adherence and treatment discontinuation, and personality disorders. From a sponsor perspective, a larger percentage of participants from a very specific population in the final cohort can increase the probability that the efficacy and safety signals would not be missed or misinterpreted. Conversely, in more heterogeneous sample, the efficacy signal could be under estimated or even overlooked[15]. This deliberate approach to enhance internal validity contradicts the regulatory requirements of new drugs to be studied in patient populations representative of full range of patients to whom the medication will be prescribed and subsequently interferes with external validity. To illustrate this problem, from a perspective of practicing psychiatrist, Zimmerman et al[16] applied the eligibility criteria of an efficacy trial of antidepressant to 599 psychiatric outpatients 18 years of age and older. Based on the exclusion criteria, the sample was divided in three groups: 123 depressed patients who qualified for the trial, 289 whose symptoms were not severe enough to qualify for the trial, and 187 who were excluded based on comorbid conditions such anxiety disorders, substance use disorders, and suicidal ideations. In other words, over 79% of patients did not qualify for the study of an antidepressant efficacy. Those who were excluded tend to have more episodes of depression, more personality pathology, and more social and functional impairment. In addition, it is has been long noted that research patients or clinical trial participants tend to be more complaint with their medications, appointments and recommendations; they also tend to be in better physical health, and have no on-going illicit drugs and/or alcohol use problems[17].

Recruitment of “professional subjects” has also been implicated as a hurdle in CNS drug development trials. The term “professional subjects” describes the heterogeneous group of patients involved in research studies primarily for financial gain and as a vehicle for ongoing access to care. It has been determined that some of these “subjects” utilize dedicated websites as a resource to guide them through study enrollment, eligibility and inclusion/exclusion criteria. Generally, “professional subjects” move from trial site to trial site collecting stipends, and interestingly, reliability of professional subjects has been shown to be greater than “local subjects”[18,19]. Commonly “subjects” are difficult to identify and subject duplication within a protocol is often not shared by pharmaceutical companies. In a study by Tishler et al[20], it was determined that the primary motivation for subject participation in a phase 1 clinical trial is “making money” and 40% of subjects claimed to have financial stress “often” or “almost all of the time”.

A number of methodological considerations have arisen over the past decade suggesting that the field of drug development will benefit from significant modifications to methodological approaches and design. For example, Post et al[21] advocated the utilization of an “off-on-off-on” design for RTCs in the therapeutic area of Bipolar Disorders. In this design, all patients initially started on placebo (off) and then receive the IP (on). Responders then enter another period of placebo treatment and their response once again re-confirmed during the second exposure to the IP. This approach more effectively addresses the heterogeneity of psychopathology and issues related to comorbidity, since patient’s response is compared to their own baseline. Study design also influences participant expectations of improvement, which can affect clinical outcomes[22].


Across all phases of clinical drug development, the recruitment of qualified patients is critical to the impact and generalizability of the study. It is often difficult to recruit patients that fall within eligibility criteria and many times this leads to extended recruitment periods and increases in overall study cost[23]. Struggles in recruitment rates can cost the pharmaceutical industry up to $1.3 million each day due to losses in prescription sales. Difficulties in recruitment can also result in reduced statistical power and even early study closure, which may leave important clinical, scientific or public health questions unanswered[24].

During the past decade, the common solution utilized by the pharmaceutical industry and contract research organizations to tackle shortages in appropriate patients for competitive clinical trials was to focus on recruitment in Eastern Europe, India, China, and Latin America. As greater numbers of clinical trials have been moved to these emerging markets, the competition for an access to patients in these areas has intensified presenting the industry with yet another set of challenges[25]. In 2012, almost 65% of the food and drug administration (FDA)-regulated clinical trials across all therapeutic areas were conducted outside the United States[26]. The overall impact of ethnic, social and cultural differences on clinically relevant outcomes from RTCs for CNS indications is still a matter of debate. The fact that the population of the United States became more diverse during the past decade, though there is still regional variability, this has lessened to some degree previous concerns regarding applicability of data from the large multi-center international trials[27].


Clinical data derived from highly selective patient populations has led to the development of an “efficacy-effectiveness gap”. In general, conventional RCTs have ignored upper and lower end dosing curves and instead establish average therapeutic doses for the average patient, which can impact patient safety and the “gap”[16,28]. A retrospective analysis conducted by Cross and Peck focused on dose changes in the post approval period and determined that the FDA approved 499 NMEs between 1980-1999. Out of the 499 NMEs, 354 were evaluable and they found that one in five had dosing changes; with 21% requiring an increase and even more critically, 79% of instances resulted in a decrease of the initially recommended dose. Dosing adjustments were three times higher for NMEs approved between 1995-1999 compared to drugs approved between 1980-1984, suggesting poorly confirmed pre-marketing dose optimization[29]. Disease progression models have been proposed as an effective tool to better evaluate disease progress and drug activity[30].

In our view, given the fact that there are significant differences between research and clinical populations, the practical solution, in terms of drug development in Psychiatry, is to include more patients from community mental health settings into RTCs. However, disparities in quality mental health services between urban and rural populations, as well as differences in socioeconomic status continue to be leading contributing factors to the significant under-representation of patients from community mental health centers in clinical trials. Patient geographic location has also been linked to outcome gaps due to disparities in access to novel and innovative care[31,32]. Correcting the current patterns of participant enrollment may favorably influence clinical trial design and outcomes by allowing more diverse population of patients, including those without previous research experience, to access RCTs. Importantly, it can ensure that patients from a lower socioeconomic status will continue to have an access to novel and innovative treatment options.


The Carolinas HealthCare System Behavioral Health Center is the largest provider of community mental health services in the region, offering a wide range of services to adults, adolescents and children. The free standing Center supports 71000 face to face encounters per year and provides around the clock access to care. With 45 physicians, 12 advanced care providers, 44 adult beds and 22 child/adolescent beds, the Center’s patient services include outpatient and partial hospitalization programs, medication management, school based services and electroconvulsive therapy. Subspecialty outpatient clinics serve a broad range of patients including those with severe and persistent mental illness. The Behavioral Health Research Section participates in clinical trials for a variety of indications including MDD, schizophrenia, adult ADHD, bipolar disorder, substance abuse and Alzheimer’s. The research site provides access to novel and innovative treatment to those in the community based mental health setting. Participants are identified by providers within the System and, patients are seamlessly transitioned into the research venue. Universal access to the electronic medical record serves as an effective tool to review and establish diagnosis, previous treatments, medical conditions and eligibility for trial participation. Upon completion of the study, patients are moved back into the clinical setting for routine care.


We must emphasize that we do not want to oversimplify a complex problem: conducting clinical research in community mental health centers can be challenging. Delivery of behavioral health services is in crisis due to the growing demand, lack of sufficient workforce, rising cost of care and complexity of major psychiatric disorders, and our Center is not an exception. Based on our experience of conducting RCTs in the therapeutic areas of schizophrenia, bipolar disorder, MDD, adult attention-deficit hyperactivity disorder, and alcohol dependency in community mental health setting, we have identified three major challenges:

Challenges related to a target population

Patients whose receive their treatment in community mental health centers encumber a variety of socioeconomic challenges, including financial difficulties, problems with stable housing and/or homelessness, lack of transportation and child care. Distrust of research and the healthcare system by the public is also a concern in participant enrollment[33]. Recent studies have also shown that minority patients, specifically African Americans and those that lack English-language proficiency, are more likely to distrust the medical research establishment[34,35] and therefore recruitment of these patient populations is often problematic. To a large extent, the primary motivator of patients most interested in clinical trial participation are their desire to help others[36], however, participants often lack awareness of the time commitment and responsibilities of participation. Responsibilities can include frequent and extended visits, multiple blood drawings, lengthy diagnostic interviews and numerous assessments. These requirements can negatively impact recruitment rates and commonly cause low retention and subsequently increased total cost. In addition to general struggles with participant recruitment, ensuring adequate representation of a diverse patient population is essential to the development of effective treatments. In our experience, there are particular challenges in recruiting and retaining of women in clinical trials. Although the regulatory agencies strongly advocate for the equal gender balance in terms of the women participation in clinical trials based on the assumption that women will represent at least fifty percent of those treated once the medicine is approved for marketing, the gender gap still exist. These specific challenges of recruiting female participants encountered in our practice are often related to the issues of birth control during the trial for reproductive age women, lack of child care, and reliable transportation as women continue to be typically poorer than men.

Improvements to ease the consent process and greater funding for trials that include non-English speaking individuals would enhance and encourage greater minority representation. Subject recruitment is a hurdle every investigator must overcome but the movement towards digital recruitment and electronic medical records support enhanced participant identification.

Challenges related to clinical research team

Critical shortages of psychiatrists and principal investigators with necessary research experience and expertise in conducting RTCs in the community mental health setting is yet another challenge in clinical research. This is a direct result of a decade-long trend viewing psychiatric clinical research as the primarily responsibility of academic institutions, pharmaceutical companies, contract research organizations and a limited number of investigators from privately owned and operated research sites. The negative impact of this problem on training and development of future researchers has been clearly identified[37]. In a recently conducted survey completed by over 700 investigators and focused on level of burden in the clinical trial process, it was found that the most “extremely burdensome” activities are contract and regulatory requirements (personal communication to OVT, 1/16/2014). In addition to an experienced principal investigator, a qualified research team consisting of sub-investigator(s), clinical research coordinator(s), rater(s), phlebotomist and office manager is necessary in order to successfully implement and complete the clinical protocol.

Challenges related to research and organizational environment

Clinical research is a complex endeavor, which is often time-consuming, labor-intense, expensive and difficult to be integrated into the daily, routine care provided by practitioners, particularly, in a public sector. There is an understandable reluctance to actively support recruitment of study participants by practitioners not directly involved in research. At the same time, a broader provider’s participation in clinical research not only enhances the collaborative matrix within the organization but ultimately enhances effective care by advancing the evidence-based approach to routine psychiatric care delivery.

None of the outlined challenges are easy to solve, particularly, since it is near impossible to estimate the magnitude of impact each set of challenges poses on clinical trial in the community mental health setting. While progress in developing novel and innovative approaches for the treatment of psychiatric disorders has been relatively slow, we must continue to evolve and improve the drug development process.


Despite recent advances in the understanding of the neurosciences, novel pharmacological treatments for psychiatric disorders have yet to be introduced. The break in translation of basic knowledge to enhanced treatments is in part due to the patient population, financial burden of clinical trials and rigorous administrative oversight. Implementation of clinical trial outcomes into community mental health centers is a multifaceted challenge due to the complexity of the patient population, requirements of the clinical trials team and struggles associated with drug development. According to reports from the Tufts Centre for the Study of Drug Development, a CNS investigational compound will spend 8.1 years in human testing, which is 2 years more than the average from other therapeutic areas, and requires an extra 1.9 years for the regulatory approval compared for 1.2 years for all other medicines[38]. Combining this time frame with 6 to 10 years of bench research, preclinical studies and preliminary testing, it can take up to 18 years for CNS compounds to complete their journey “from the bench to the bedside”. Even after the drug is on the market, the gap in translational effectiveness is great. With only 21% of patients in the community mental health setting meeting the stringent eligibility criteria set during the drug development phase, the gap is not narrowing[16]. A multitude of factors, as well as pricing pressures from generic competition and payers, force pharmaceutical and biotech companies to reexamine priorities and innovation in drug development[39,40].

Community mental health centers are important entry points for patients suffering from severe mental disorders and recruitment of a generalizable or well defined patient population is imperative to the translation of RCT’s into clinical practice[41]. Major psychiatric disorders are heterogeneous in nature and therefore identification of drug effectiveness across the disease continuum is difficult. Furthermore, the process of implanting new treatments into the community mental health setting can require significant human resources, financial support and dissemination of new interventions. The purpose of this commentary is to stimulate discussion on what we believe to be one of the most critical issues in drug development.


P- Reviewer: Huerta-Franco MR S- Editor: Song XX L- Editor: A E- Editor: Liu SQ

1.  Insel TR. Assessing the economic costs of serious mental illness. Am J Psychiatry. 2008;165:663-665.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 263]  [Cited by in F6Publishing: 323]  [Article Influence: 17.5]  [Reference Citation Analysis (0)]
2.  Trivedi MH, Fava M, Wisniewski SR, Thase ME, Quitkin F, Warden D, Ritz L, Nierenberg AA, Lebowitz BD, Biggs MM. Medication augmentation after the failure of SSRIs for depression. N Engl J Med. 2006;354:1243-1252.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 653]  [Cited by in F6Publishing: 709]  [Article Influence: 38.4]  [Reference Citation Analysis (0)]
3.  Barak Y, Aizenberg D. Clinical and psychosocial remission in schizophrenia: correlations with antipsychotic treatment. BMC Psychiatry. 2012;12:108.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 14]  [Cited by in F6Publishing: 16]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
4.  Haro JM, Novick D, Suarez D, Alonso J, Lépine JP, Ratcliffe M. Remission and relapse in the outpatient care of schizophrenia: three-year results from the Schizophrenia Outpatient Health Outcomes study. J Clin Psychopharmacol. 2006;26:571-578.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 94]  [Cited by in F6Publishing: 105]  [Article Influence: 5.9]  [Reference Citation Analysis (0)]
5.  Brundtland GH. From the World Health Organization. Mental health: new understanding, new hope. JAMA. 2001;286:2391.  [PubMed]  [DOI]  [Cited in This Article: ]
6.  World Health Organization Global health risks: mortality and burden of disease attributable to selected major risks. 2009;.  [PubMed]  [DOI]  [Cited in This Article: ]
7.  Conn PJ, Roth BL. Opportunities and challenges of psychiatric drug discovery: roles for scientists in academic, industry, and government settings. Neuropsychopharmacology. 2008;33:2048-2060.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 62]  [Cited by in F6Publishing: 67]  [Article Influence: 4.1]  [Reference Citation Analysis (0)]
8.  U.S. Food and Drug Administration New Molecular Entity Approvals for 2013. 2014;.  [PubMed]  [DOI]  [Cited in This Article: ]
9.  Kola I, Landis J. Can the pharmaceutical industry reduce attrition rates? Nat Rev Drug Discov. 2004;3:711-715.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2945]  [Cited by in F6Publishing: 3014]  [Article Influence: 155.0]  [Reference Citation Analysis (0)]
10.  Gould TD, Manji HK. The molecular medicine revolution and psychiatry: bridging the gap between basic neuroscience research and clinical psychiatry. J Clin Psychiatry. 2004;65:598-604.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 39]  [Cited by in F6Publishing: 25]  [Article Influence: 2.1]  [Reference Citation Analysis (0)]
11.  Cartwright ME, Cohen S, Fleishaker JC, Madani S, McLeod JF, Musser B, Williams SA. Proof of concept: a PhRMA position paper with recommendations for best practice. Clin Pharmacol Ther. 2010;87:278-285.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 44]  [Cited by in F6Publishing: 46]  [Article Influence: 3.4]  [Reference Citation Analysis (0)]
12.  Mitchell M, Jolly J.  Research Design Explained. 4th edn. 2001;.  [PubMed]  [DOI]  [Cited in This Article: ]
13.  Brady KT, Sinha R. Co-occurring mental and substance use disorders: the neurobiological effects of chronic stress. Am J Psychiatry. 2005;162:1483-1493.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 338]  [Cited by in F6Publishing: 367]  [Article Influence: 18.8]  [Reference Citation Analysis (0)]
14.  Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:617-627.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7240]  [Cited by in F6Publishing: 7925]  [Article Influence: 402.2]  [Reference Citation Analysis (0)]
15.  Zimmerman M, Posternak MA, Chelminski I. Symptom severity and exclusion from antidepressant efficacy trials. J Clin Psychopharmacol. 2002;22:610-614.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 40]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
16.  Zimmerman M, Chelminski I, Posternak MA. Generalizability of antidepressant efficacy trials: differences between depressed psychiatric outpatients who would or would not qualify for an efficacy trial. Am J Psychiatry. 2005;162:1370-1372.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 102]  [Cited by in F6Publishing: 111]  [Article Influence: 5.7]  [Reference Citation Analysis (0)]
17.  Seeman MV. Clinical trials in psychiatry: do results apply to practice? Can J Psychiatry. 2001;46:352-355.  [PubMed]  [DOI]  [Cited in This Article: ]
18.  Apseloff G. Reliability of professional volunteers recruited for clinical trials. J Clin Pharmacol. 2014;54:616-618.  [PubMed]  [DOI]  [Cited in This Article: ]
19.  Shiovitz TM, Zarrow ME, Shiovitz AM, Bystritsky AM. Failure rate and “professional subjects” in clinical trials of major depressive disorder. J Clin Psychiatry. 2011;72:1284; author reply 1284-1285.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 8]  [Article Influence: 0.7]  [Reference Citation Analysis (0)]
20.  Tishler CL, Apseloff G, Bartholomae S, Reiss NS, Rhodes AR, Singh A. Are normal healthy research volunteers psychologically healthy? A pilot investigation. Exp Clin Psychopharmacol. 2007;15:539-545.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 7]  [Article Influence: 0.5]  [Reference Citation Analysis (0)]
21.  Post RM, Luckenbaugh DA. Unique design issues in clinical trials of patients with bipolar affective disorder. J Psychiatr Res. 2003;37:61-73.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 19]  [Cited by in F6Publishing: 19]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
22.  Rutherford BR, Rose SA, Sneed JR, Roose SP. Study design affects participant expectations: a survey. J Clin Psychopharmacol. 2009;29:179-181.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 15]  [Article Influence: 0.9]  [Reference Citation Analysis (0)]
23.  Aitken L, Gallagher R, Madronio C. Principles of recruitment and retention in clinical trials. Int J Nurs Pract. 2003;9:338-346.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 50]  [Cited by in F6Publishing: 55]  [Article Influence: 2.6]  [Reference Citation Analysis (0)]
24.  Getz KA, de Bruin A. Breaking the Development Speed Barrier: Assessing Successful Practices of the Fastest Drug Development Companies. Drug InfJ. 2000;34:725-736.  [PubMed]  [DOI]  [Cited in This Article: ]
25.  Theirs FA, Sinskey SJ, Berndt ER. Trends in the globalization of clinical trials. Nat Rev Drug Discov. 2008;7:13-14.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 209]  [Cited by in F6Publishing: 209]  [Article Influence: 13.9]  [Reference Citation Analysis (0)]
26.  Khin NA, Chen YF, Yang Y, Yang P, Laughren TP. Exploratory analyses of efficacy data from major depressive disorder trials submitted to the US Food and Drug Administration in support of new drug applications. J Clin Psychiatry. 2011;72:464-472.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 144]  [Cited by in F6Publishing: 117]  [Article Influence: 12.0]  [Reference Citation Analysis (0)]
27.  Yasuda SU, Zhang L, Huang SM. The role of ethnicity in variability in response to drugs: focus on clinical pharmacology studies. Clin Pharmacol Ther. 2008;84:417-423.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 284]  [Cited by in F6Publishing: 298]  [Article Influence: 18.9]  [Reference Citation Analysis (0)]
28.  Weiss AP, Guidi J, Fava M. Closing the efficacy-effectiveness gap: translating both the what and the how from randomized controlled trials to clinical practice. J Clin Psychiatry. 2009;70:446-449.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 15]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
29.  Cross J, Lee H, Westelinck A, Nelson J, Grudzinskas C, Peck C. Postmarketing drug dosage changes of 499 FDA-approved new molecular entities, 1980-1999. Pharmacoepidemiol Drug Saf. 2002;11:439-446.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 105]  [Cited by in F6Publishing: 105]  [Article Influence: 5.3]  [Reference Citation Analysis (0)]
30.  Mould DR, Denman NG, Duffull S. Using disease progression models as a tool to detect drug effect. Clin Pharmacol Ther. 2007;82:81-86.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 35]  [Cited by in F6Publishing: 35]  [Article Influence: 2.2]  [Reference Citation Analysis (0)]
31.  Adler NE, Rehkopf DH. U.S. disparities in health: descriptions, causes, and mechanisms. Annu Rev Public Health. 2008;29:235-252.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 699]  [Cited by in F6Publishing: 796]  [Article Influence: 46.6]  [Reference Citation Analysis (0)]
32.  Raghavan D. Slow progress in cancer care disparities: HIPAA, PPACA, and CHEWBACCA... but we’re still not there! Oncologist. 2011;16:917-919.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 4]  [Article Influence: 0.4]  [Reference Citation Analysis (0)]
33.  Paskett ED, Reeves KW, McLaughlin JM, Katz ML, McAlearney AS, Ruffin MT, Halbert CH, Merete C, Davis F, Gehlert S. Recruitment of minority and underserved populations in the United States: the Centers for Population Health and Health Disparities experience. Contemp Clin Trials. 2008;29:847-861.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 124]  [Cited by in F6Publishing: 135]  [Article Influence: 8.3]  [Reference Citation Analysis (0)]
34.  Corbie-Smith G, Thomas SB, St George DM. Distrust, race, and research. Arch Intern Med. 2002;162:2458-2463.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 616]  [Cited by in F6Publishing: 681]  [Article Influence: 29.3]  [Reference Citation Analysis (0)]
35.  Woodall A, Morgan C, Sloan C, Howard L. Barriers to participation in mental health research: are there specific gender, ethnicity and age related barriers? BMC Psychiatry. 2010;10:103.  [PubMed]  [DOI]  [Cited in This Article: ]
36.  Bevan EG, Chee LC, McGhee SM, McInnes GT. Patients’ attitudes to participation in clinical trials. Br J Clin Pharmacol. 1993;35:204-207.  [PubMed]  [DOI]  [Cited in This Article: ]
37.  Bell J. Resuscitating clinical research in the United Kingdom. BMJ. 2003;327:1041-1043.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 47]  [Cited by in F6Publishing: 61]  [Article Influence: 2.4]  [Reference Citation Analysis (0)]
38.  Kaitin KI, DiMasi JA. Pharmaceutical innovation in the 21st century: new drug approvals in the first decade, 2000-2009. Clin Pharmacol Ther. 2011;89:183-188.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 240]  [Cited by in F6Publishing: 247]  [Article Influence: 18.5]  [Reference Citation Analysis (0)]
39.  Drews J, Ryser S. The role of innovation in drug development. Nat Biotechnol. 1997;15:1318-1319.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 120]  [Cited by in F6Publishing: 125]  [Article Influence: 4.8]  [Reference Citation Analysis (0)]
40.  Cowen PJ. Has psychopharmacology got a future? Br J Psychiatry. 2011;198:333-335.  [PubMed]  [DOI]  [Cited in This Article: ]
41.  Druss BG, Marcus SC, Campbell J, Cuffel B, Harnett J, Ingoglia C, Mauer B. Medical services for clients in community mental health centers: results from a national survey. Psychiatr Serv. 2008;59:917-920.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 14]  [Cited by in F6Publishing: 11]  [Article Influence: 0.9]  [Reference Citation Analysis (0)]