Minireviews Open Access
Copyright ©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Psychiatry. Mar 19, 2022; 12(3): 393-409
Published online Mar 19, 2022. doi: 10.5498/wjp.v12.i3.393
Digital phenotyping in depression diagnostics: Integrating psychiatric and engineering perspectives
Jayesh Kamath, Department of Psychiatry and Immunology, University of Connecticut School of Medicine, University of Connecticut Health Center, Farmington, CT 06030, United States
Jayesh Kamath, Roberto Leon Barriera, Neha Jain, Efraim Keisari, Department of Psychiatry, University of Connecticut School of Medicine, University of Connecticut Health Center, Farmington, CT 06032, United States
Bing Wang, Department of Computer Science and Engineering, University of Connecticut, Storrs, CT 06269, United States
ORCID number: Jayesh Kamath (0000-0002-6982-4302); Roberto Leon Barriera (0000-0002-6518-4758​); Neha Jain (0000-0003-1804-7758); Efraim Keisari (0000-0001-8089-3221); Bing Wang (0000-0002-7632-6512).
Author contributions: Kamath J and Wang B are the primary authors of this manuscript and are Co-Principal Investigators of the studies described in the manuscript; Barriera RL wrote the active ecological momentary assessments sections; Jain N wrote the telepsychiatry sections; Keisari E wrote the privacy, legal, and ethical challenges section; all three contributed to other sections of the manuscript.
Conflict-of-interest statement: The authors declare no conflicts of interest regarding this manuscript.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Jayesh Kamath, MD, PhD, Professor, Department of Psychiatry and Immunology, University of Connecticut School of Medicine, University of Connecticut Health Center, 263 Farmington Avenue, Farmington, CT 06030, United States. jkamath@uchc.edu
Received: June 30, 2021
Peer-review started: June 30, 2021
First decision: September 5, 2021
Revised: September 23, 2021
Accepted: February 12, 2022
Article in press: February 12, 2022
Published online: March 19, 2022
Processing time: 261 Days and 10.2 Hours

Abstract

Depression is a serious medical condition and is a leading cause of disability worldwide. Current depression diagnostics and assessment has significant limitations due to heterogeneity of clinical presentations, lack of objective assessments, and assessments that rely on patients' perceptions, memory, and recall. Digital phenotyping (DP), especially assessments conducted using mobile health technologies, has the potential to greatly improve accuracy of depression diagnostics by generating objectively measurable endophenotypes. DP includes two primary sources of digital data generated using ecological momentary assessments (EMA), assessments conducted in real-time, in subjects' natural environment. This includes active EMA, data that require active input by the subject, and passive EMA or passive sensing, data passively and automatically collected from subjects' personal digital devices. The raw data is then analyzed using machine learning algorithms to identify behavioral patterns that correlate with patients' clinical status. Preliminary investigations have also shown that linguistic and behavioral clues from social media data and data extracted from the electronic medical records can be used to predict depression status. These other sources of data and recent advances in telepsychiatry can further enhance DP of the depressed patients. Success of DP endeavors depends on critical contributions from both psychiatric and engineering disciplines. The current review integrates important perspectives from both disciplines and discusses parameters for successful interdisciplinary collaborations. A clinically-relevant model for incorporating DP in clinical setting is presented. This model, based on investigations conducted by our group, delineates development of a depression prediction system and its integration in clinical setting to enhance depression diagnostics and inform the clinical decision making process. Benefits, challenges, and opportunities pertaining to clinical integration of DP of depression diagnostics are discussed from interdisciplinary perspectives.

Key Words: Digital phenotyping; Depression; Ecological momentary assessment; Telepsychiatry; Passive sensing; Smart phone

Core Tip: There are systematic/quantitative reviews and meta-analyses of digital phenotyping (DP) in depression available in literature. These reviews are primarily published by engineering groups and provide limited psychiatric perspective, especially clinical relevance and clinical integration. The current review presents an overview of digital phenotyping of depression diagnostics and assessment from both psychiatric and engineering perspective. The overview includes major advances in the field of DP of depression diagnostics, including active and passive ecological momentary assessment, DP using data from social media, and DP using data from electronic medical records. We briefly discuss investigations conducted by our group and present a model for clinical integration of DP informed by those investigations conducted by our group. Finally, we discuss benefits, challenges, and opportunities pertaining to clinical integration of DP of depression diagnostics from an interdisciplinary perspective.



INTRODUCTION

Major depressive disorder (MDD) is a common, serious, and debilitating illness affecting all ages; children and adolescents, adults, and elderly[1]. It affects more than 264 million people worldwide and is associated with significant morbidity, increased mortality due to high suicide risk, diminished functioning, and poor quality of life[1,2].

In 2017, the worldwide prevalence of MDD was estimated to be at 4.4% globally[2]. The lifetime risk of depression was much higher (15%-18%)[2]. Consistent with this high risk, in terms of disease burden, MDD represented the third highest cause of Years Lived with Disability (YLD) globally[3]. In the United Sates (US), MDD accounted for 3.7% of all US adjusted disability years with significant economic burden and societal costs[4,5]. The National Survey on Drug and Health (NSDUH) conducted in 2017 found that an estimated 17.3 million or 7.1% of US adults have experienced at least one major depressive episode[4].

Similar to other fields of medicine, there has been a strong impetus in psychiatry to personalize depression assessment and treatment[6,7]. However, despite decades of research, few clinically relevant biomarkers, genetic variations or clinical characteristics have been identified that can aid in depression diagnosis and treatment[6,7]. Advances in digital technologies provide an exciting opportunity to personalize depression care[8]. Smart phones with their digital sensors and increasingly advanced computing capabilities have the potential to serve as “human sensors” by capturing granular changes in behavioral patterns[8,9]. Electronic medical records can gather large amounts of data across multiple disciplines of medicine, generate personalized patient reports, and seamlessly transfer data between large health care systems. Telepsychiatry can help us reach patients in real-time and conduct assessments in their natural settings. Integration and application of these technologies has the potential to significantly advance and personalize depression care.

Several recent systematic reviews of digital technologies and their application in depression care are available in literature[9-12]. These reviews are focused on either clinical or engineering/technical aspects of digital phenotyping technologies in depression care[9-12]. The objective of the current review is to integrate, evaluate, and synthesize evidence-informed literature from both clinical and engineering perspectives. The goal is to present a clinically-relevant, evidence informed review beneficial to clinicians, engineers, and researchers from diverse disciplines. Another goal is to help advance multidisciplinary collaborations with clear clinical objectives. We will summarize gaps, challenges, and opportunities from clinical, engineering, and legal perspectives. Finally, informed by investigations conducted by our research group[13-16], we will present a model for integration of digital phenotyping technologies in clinical setting to improve depression care.

DEPRESSION DIAGNOSIS AND ASSESSMENT: CURRENT STANDARD OF CARE

MDD is a heterogeneous disorder with potentially diverse and multifactorial presentations[17,18]. Decades of research has shown that depression is the result of a complex interplay between genetic and environmental vulnerabilities initiating a cascade of neurobiological changes in diverse bodily systems[19,20]. Diagnosis of MDD includes confirmation of symptomatic threshold, patient distress, and functional impairment as a result of depression symptoms[21]. Diagnosis also involves ruling out medical, psychiatric, and substance use disorders that may present with depression symptomatology[21]. Two major taxonomies available for diagnosing depressive disorders include American Psychiatric Association’s The Diagnostic and Statistical Manual of Mental Disorders (5th edition; [DSM-5]) and World Health Organization’s The International Statistical Classification of Diseases and Related Health Problems (11th edition; [ICD-11])[21,22]. Diagnostic criteria for MDD are same in both classifications. Depression is characterized by two primary symptoms; depressed mood and loss of pleasure or interest lasting at least 2 weeks[21,22]. To meet the threshold for a Major Depressive Episode (MDE), these core symptoms should be accompanied by at least four more symptoms (for a total of at least five) as noted in Table 1[21,22]. Additionally, significant distress and measurable negative impact on functioning are required for a depression diagnosis (Table 1)[21,22]). Symptoms of depression can be grouped into three major categories; psychological or emotional, neurovegetative, and neurocognitive (Figure 1)[23]. Psychological symptoms are primarily subjective in nature i.e., they depend on a patient’s experience and their perception of these symptoms. It can be argued that psychological symptoms (e.g., anhedonia/Lack of interest or pleasure) have behavioral consequences and lead to a change in functioning. Neurovegetative and neurocognitive symptoms are objective in nature and have measurable behavioral manifestations with subsequent impact on functioning. Patient reporting of subjective symptoms is inherently based on their experience and perception of these symptoms. This subjective vs. objective nature of depression symptoms with discussion of their direct or indirect behavioral manifestation and impact on functioning is critical to digital phenotyping in depression diagnostics. This distinction has a direct clinical relevance for application of digital phenotyping diagnostics in real-world clinical settings.

Figure 1
Figure 1 Depression symptomatology.
Table 1 Summary of major depressive disorder criteria.
Five (or more) of the following symptoms present for at least 2 wk period
Depressed mood
Anhedonia i.e., diminished interest or pleasure
Weight loss or weight gain
Sleep disturbances (insomnia or hypersomnia)
Psychomotor agitation or retardation
Fatigue
Feelings of worthlessness or excessive inappropriate guilt
Cognitive difficulties
Suicidal thoughts and/or behaviors
Other Criteria:
Symptoms cause clinically significant distress or functional impairment
Symptoms are not better explained by other psychiatric or medical diagnosis

Patient self-rated and clinician-rated depression questionnaires are frequently used in screening and diagnosis of MDD[24,25]. Commonly used patient self-rated instruments include the 9-item Patient Health Questionnaire (PHQ-9), the Beck Depression Inventory (BDI), the 16-item Quick Inventory of Depression Symptomatology-Self Rated (QIDS16-SR), and the Center for Epidemiologic Studies Depression Scale (CES-D)[24,26]. In real world clinical settings, self-rated instruments are used more frequently than clinician-administered instruments as they are easier to administer and demand fewer resources[27]. These instruments also play a critical role in the continuum of depression care and help personalize patient care.

Limitations of current depression diagnosis and assessment

The DSM of Mental Disorders (DSM-5) endeavors to categorize psychiatric symptomatology into specific disorders[21]. Despite evidence supporting such categorization, DSM-based diagnosis of depression remains subjective, as it relies upon patient report, clinician observation, and clinical judgment. In real world settings, clinicians struggle with the limitations of DSM-based diagnosis due to heterogeneity of patient presentations not fully captured by DSM criteria[28]. Limitations of DSM-based depression diagnosis and assessment are further exacerbated by challenges in clinical setting such as brief (15 to 20 minutes) patient visits with limited time for clinical assessments, and complexity of patient presentations with multiple comorbidities[29]. Administration of depression rating scales can add some objectivity to clinical assessments. However, evidence indicates that few clinicians use rating scales in their clinical practice[30]. This is due to several reasons, including lack of adequate resources to administer such scales[30]. Furthermore, the rating scales rely on a patient’s memory and capture a narrow spectrum of a patient’s overall mental state[31]. A major DSM criterion for depression diagnosis is two weeks of persistent symptomatology[21]. Evidence suggests that patient reports during clinical encounters may be largely influenced by their symptoms during the days leading up to the clinical encounter[31]. Due to their reliance on patient recall, clinical assessments may fail to fully capture the severity of the neurovegetative and neurocognitive symptoms of depression (e.g., fatigue, sleep disturbances, concentration)[31,32]. Current clinical assessments also fail to capture functional impact of depression, a core criterion (criterion B) for depression diagnosis[31,33]. These assessments provide a cross-sectional evaluation of a patient’s mental state as they are administered infrequently, usually every 4 to 6 weeks during the patient’s clinic visit.

DIGITAL PHENOTYPING IN DEPRESSION

Digital Phenotyping is defined as "moment-to-moment quantification of the individual-level human phenotype in situ using data collected from personal digital devices"[34,35]. DP has the potential to greatly improve the accuracy of depression diagnosis and assessment by adding much needed objectivity to the process. By generating objectively measurable endophenotypes, it can serve as a behavioral biomarker to personalize depression care[34,35]. The generated phenotype provides an ecological and continuous representation of a patient’s physical, emotional, behavioral, social, and cognitive activities in real-time[35,36]. At present, DP relies on two primary sources of data, active and passive data, generated by Ecological Momentary Assessments (EMA) conducted using personal digital devices. Active EMA consist of data reported directly by the user, and passive EMA consists of data automatically collected from digital devices and platforms[9,11,37]. The digital devices that currently serve as DP sources include smart phones, wearable sensors, and data collected from human-computer interactions[9,37]. DP in depression diagnostics involves a multistep process[38,39]. The first step involves obtaining signals from the digital devices to generate raw data. Once the data is collected, the goal is to find patterns that correlate with patient’s clinical status. This step involves use of machine learning algorithms to find predictive behavioral features from the raw data sets. The final step is to integrate the features and electronic self-reports (active EMA) to generate an ecological, continuous, and personalized digital phenotype of the patients that can enhance depression diagnostics and assessment in clinical setting[38,39].

ECOLOGICAL MOMENTARY ASSESSMENT IN DEPRESSION DIAGNOSTICS

EMA involve repeated sampling of an individual’s behaviors and experiences in real-time, in the person's natural environment[40]. EMA conducted digitally as part of DP in depression diagnostics strives to minimize recall bias seen with assessments conducted in clinical settings[9,11]. In addition, it seeks to maximize ecological validity and allows the investigation of processes that influence behavior in real world settings[9,11]. As mentioned earlier, EMA can be categorized into active and passive EMA[9,11]. Any data or assessments that need active input by participants falls under Active EMA (e.g., electronic assessments using depression questionnaires). Passive EMA includes any data or assessments collected passively (i.e., without participant’s active input)[9,11].

Table 2 delineates depression symptomatology and major categories of active and passive EMA used to measure these symptoms. 'Subjective symptoms' such as depressed mood, guilt/negative beliefs, and suicidality can be primarily measured using active EMA such as depression questionnaires. ‘Subjective symptoms with direct behavioral manifestations’ such as anhedonia and concentration difficulties can be measured using both active and passive EMA. Similarly, both active and passive EMA measurements play an important role in evaluation of ‘objective symptoms with subjective patient experiences’ such as psychomotor agitation or retardation and appetite. Finally, ‘objective symptoms with direct behavioral manifestations’ such as fatigue and sleep are primarily measured using passive EMA. As shown in Figure 2, active EMA such as self-report questionnaires can be used to measure depression symptoms, distress due to these symptoms, and their impact on functioning, while passive EMA can significantly contribute to the assessments of objective behavioral manifestations such as neurovegetative symptoms and impact on functioning.

Figure 2
Figure 2 Depression diagnosis and ecological momentary assessment.
Table 2 Depression symptoms, ecological momentary assessment active, and ecological momentary assessment passive.
Depression symptoms
Depressed mood. anhedonia
Fatigue, sleep disturbances (insomnia or hypersomnia)
Psychomotor agitation or retardation, cognitive difficulties
Appetite problems
Guilt/negative beliefs
Suicidal thoughts/behaviors
EMA active
Standardized assessments
Self-report depression questionnaires (e.g., PHQ-9)
Non-standardized assessments
Daily mood, anxiety, sleep ratings
Acoustic and paralinguistic information with audio sampling e.g., voice intonation
EMA passive (behavioral feature categories, features, and sensors used)
Physical activity and sleep
Activity time-accelerometer
Inactivity-accelerometer, GPS
Distance-accelerometer, GPS
Movement duration and speed-GPS
Sleep duration, latency, efficiency-fitbit, accelerometer
Location
Home stay-GPS
Location clusters and variance-GPS
Entropy-GPS
Circadian rhythm-GPS
Social communication
Call duration/frequency, missed calls, number of conversations-call log
Sms text (incoming and outgoing)-sms text message log
Device
Social media engagement, social media app usage
Screen active duration and frequency
Social media engagement duration/frequency-app usage
Response time notification
Computer-keyboard interactions
Active EMA

In active EMA, patients are prompted to enter information into their electronic devices at specific time intervals based on the type of assessment conducted[9,11]. A variety of standardized and non-standardized questionnaires can be used, allowing researchers to collect a varied amount of information from patients in real-time, in their natural environments[9,11].

Standardized assessments used in active EMA are generally self-report and self-administered questionnaires[9,11]. These assessments are validated to assess symptoms of depression[9,11]. Some examples of standardized assessments that have been used in EMA studies include: Patient Health Questionnaire (PHQ-9), Hamilton Depression Rating Scale (HDRS), Quick Inventory of Depressive Symptomatology (QIDS), and Beck Depression Inventory (BDI)[9,11]. While these depression assessment questionnaires are the same as those conducted in-person during a clinic visit, the major difference is that the active EMA are conducted in real-time, in participants’ natural environment, and can be conducted more frequently to minimize recall bias[9,11]. Active EMA can be used for screening or to guide treatments based on depression status[41]. When used with passive EMA (passive sensing), they are frequently used as ‘ground truth’ to develop machine learning models[11,14]. In mobile health (mHealth) studies, these are administered at baseline and then at specific intervals (e.g., PHQ-9 administered bi-weekly, QIDS administered weekly)[13,14].

Non-standardized assessments used in active EMA usually lack validation studies supporting their use in depression diagnosis or monitoring. However, they may provide important clinical information and leverage mHealth technology to conduct brief assessments in real-time and in the patients’ natural environment[11,13]. Examples include general questions about mood, anxiety, sleep time and quality, medication adherence, medication tolerability, and physical activity[11,13]. Information gathered using these assessments can be combined with passive EMA data to improve detection of depressive symptomatology[11,14]. For example, studies have shown negative correlation between self-reported mood and the amount of time the phone screen was on and the percentage of social and entertainment apps used by the participant[11]. These assessments can be used for daily monitoring of symptoms[11,13]. The frequency of their administration varies between studies depending on the assessment and the study objective[11,13].

Several studies have highlighted the issue of recall bias with self-report depression questionnaires conducted every 4 to 6 week during patients’ clinic visits[9,11]. Evidence indicates that patients with depression tend to judge their symptoms to be more severe or remember negative experiences more prominently when asked to recall them retrospectively[9,42]. Active EMA via mobile devices allows the collection of information in real-time, minimizing recall bias[9,11,42]. Obtaining this information in real-time also allows clinicians to put variations of mood in patients’ situational and social context. This may reveal subtle patterns of emotional expression that would otherwise be missed by traditional depression assessments[43]. Daily monitoring of mood may improve patients’ insight in their illness and allows them to become active participants in their treatment[11,43]. This may help them recognize patterns in their mood changes or negative feelings, triggers that lead to these changes, and help them examine if their coping strategies were effective[43]. Active EMA can also be used to monitor suicidal ideation, a critical aspect in depression management. One study found that 58% of their participants logged suicidal ideation during EMA assessment but denied it on retrospective review[44].

Active EMA includes alternate ways to assess affect and cognition using samples collected from patients[45]. Analyses of acoustic samples have identified acoustic cues that can predict individuals’ emotions and affective state[46]. This includes features such as prosodic features, spectral-based features, and glottal features[46].

Passive EMA

Passive sensing using smart phones and wearables can capture multiple dimensions of human behavior. Studies conducted in patients with depression have provided preliminary evidence of feasibility and efficacy of using passive sensing data for clinical inferences[9-11]. Passive sensing can capture and monitor behavioral correlates of all three clusters of depression symptomatology: psychological, neurovegetative, and neurocognitive. However, it is especially useful to capture the neurovegetative and neurocognitive symptoms (e.g., fatigue, sleep, concentration), as these symptoms have direct impact on behavior and functioning[13,14]. Several studies have shown consistent and statistically significant correlations between objective behavioral features collected via mobile phones and wearable devices and depressive symptomatology[9,10,14].

The process of passive sensing involves collecting raw and continuous data from multiple sensors present in mobile phones and wearable devices such as a Fitbit[13,14]. These include sensors such as the accelerometer, Global Positioning System sensor (GPS), light sensor, and microphone[9,10]. Data is also gathered on device analytics such as call logs, Short Message Service (SMS) texting patterns, and device activity[9,10]. Behavioral features are extracted from the raw data. The features are expected to capture behaviors, such as location clusters captured by GPS reflecting the number of locations visited by the individual. In patients with depression, behavioral features capture changes in behavior as a reflection of depression status and severity. The features are grouped into specific categories as correlates of depression symptomatology (e.g., reduced activity and decreased number of locations visited by the individual may be reflective of anhedonia and fatigue)[9,10].

Table 2 describes categories of behavioral features, their correlates in depression symptomatology, and features that have shown consistent and statistically significant correlations with depression symptoms[9,10]. In studies conducted in non-clinical samples, features home stay (more time at home) and screen active duration (longer phone usage) showed consistent positive correlations with depression symptomatology[9,10]. In the same sample, features that showed consistent negative correlations with depression symptoms include amount of vigorous activity, location variance, and distance covered[9,10]. In clinical samples, features that showed consistent positive correlations with mood symptoms include screen active duration and incoming call frequency and duration (amount of time spent by the individual on incoming calls)[9,10]. Features that showed consistent negative correlations with depression symptoms in clinical samples include the amount of visible cell towers (reflecting mobility), SMS text messages received, and outgoing call frequency and duration[9,10]. Recently, our group developed techniques to identify Internet usage sessions (i.e., time periods when a user is online)[15]. A novel set of features were extracted based on usage sessions from the Internet traffic meta-data[15]. Machine learning models developed using these features were successfully able to predict depression status of the participants[15].

In addition to the analyses of acoustic samples provided by patients, passively gathered acoustics samples (from patients’ digital devices) have also been used to predict patients’ affective state[45]. Studies have shown that participants’ affective state and cognitive traits can be predicted using alternate methods, such as language analyses and human-computer interactions[45,47].

Challenges and limitations of active and passive EMA

For both active and passive EMA, the degree of patients’ technical knowledge can be a critical factor affecting compliance. Technical problems and inappropriate operating systems have been cited as among the most common reasons for participant drop out in EMA studies[41,42]. For active EMA, this may include technical issues with data entry and uploading of data. For passive EMA, it usually involves uploading of passive sensory data to the servers[41,42].

Assessments conducted in active EMA can become inconvenient and burdensome for participants[11]. This can lead to non-compliance. Studies have found that patient compliance with assessments decreases with time depending on their content and frequency of administration[9,11]. The need for active data entry may deter patients from adopting active EMA[37]. The standardized assessments administered electronically on a weekly or bi-weekly basis (e.g., PHQ-9) can be conducted more frequently than in-office settings but still suffer from a similar recall-bias due to the duration they cover[9,11,42]. Although, one might argue that this recall bias is much less compared to their administration in office settings (usually every 4-6 week) due the higher frequency of their electronic administration. From a research perspective, daily mood monitoring can serve as a type of intervention, confounding the study design. Studies have shown that daily symptom recording, without any other direct treatment/intervention, improved symptoms of depression[11].

For passive EMA, other major technological challenges include battery drainage concerns reported by participants due to passive sensing on their mobile devices[48,49]. Studies have reported lack of sensor precision affecting data analyses (e.g., inaccurate location data)[48,49]. Another major issue is missing sensory data[15,50,51]. As an example, the energy management system on a phone may turn off GPS when the battery level is low. In addition, it is well known that GPS does not perform well in certain common environments (e.g., indoors), where it either fails to collect data or collects data with large errors[15]. Other challenges include heterogeneous data collection from different sensing devices[52,53]. As an example, because of the different operating systems and the specific sensors used by Android and iOS, the two predominant smart phone platforms, the methods of data collection on these two platforms differ substantially. Consequently, the behavioral parameters derived from the different sources of sensing data exhibit significant differences[52,53]. The large volume of collected data may present a challenge for secure storage, statistical analysis, and clinical application[48]. Other technological challenges include data security and privacy, in particular, when the data needs to be shared with clinician’s office[13,48].

Depression questionnaires and clinical interviews are used as ‘ground truth’ to find correlations with passive sensory data and to develop machine learning models[9,10]. A major limitation of this approach is the fact that the ‘ground truth’ (i.e., the questionnaires and interviews) is still subjective. This may change over time as we gather larger amounts of data leading to better machine learning models based on passive sensory data. However, what if there is a significant discrepancy between active EMA (i.e., patients’ perception of their symptoms) and passive EMA (i.e., objective behavioral data gathered by sensors on their mobile devices and analyzed using machine learning models)? In clinical settings, such a discrepancy may pose a challenge for clinicians with their decision-making process.

Privacy, legal, and ethical challenges

Digital phenotyping technologies have the potential to revolutionize mental health research and clinical care. However, they also present ethical, legal, privacy, and regulatory challenges[54]. A key initial consideration when developing and subsequently implementing digital depression assessment technologies is that of consent and, specifically, of informed consent, a key bioethics principal[55]. Participants agreeing to digital phenotyping in research or clinical settings should understand the risks and benefits of any monitoring hardware or software, or of any subsequent intervention. Ethical constituents of informed consent include sharing information with the patient, assessing decisional capacity of the patient, and examining a patient’s voluntarism[56]. For many of these technologies, a clinician must assess a participant’s understanding of the scope and granularity of data being collected. Since there is a broad range of technology literacy in the general public and few participants will have a full understanding of the data they are sharing or of its potential uses, the informed aspect of informed consent is ever more crucial[55,57]. One must also ensure that participants understand that consent is an ongoing process and can be withdrawn at any time.

Data privacy and protection are also key issues. When acquiring data, there must be adequate encryption to ensure data is securely transmitted from the source (e.g., a smartphone) to a storage device (e.g., servers). Once data is collected, there must be clear guidelines as to who can access this data and for what purpose. Storing data then becomes one of the biggest issues due to the scope and nature of data that is collected. Even with safeguards in place, data breaches are common in healthcare settings[55,57]. Another salient feature of data is that of ownership. Key questions to consider that largely remain unanswered are: Who owns the data created? What can be done with the data in the future? Who can profit from the data? As data collection moves from requiring user input (active EMA) to collecting passive data (passive EMA), the security and privacy challenge of bystanders, who do not provide consent, comes into play[57].

Once the ethical, security, and privacy concerns are managed, those who implement the various mHealth modalities must consider their liability. Liability can stem from failure to act on information (e.g., suicidal ideation), errors that stem from malfunction of apps, misunderstanding or misinterpretation of information by patients[58]. In the studies by our group[13,14], a study clinician is on call at all times to act on suicidal ideation that is entered into the study app when participants completed their weekly depression questionnaires. When these apps evolve to use more passive data and are ultimately predictive, what happens when the software predicts there is a risk of suicide? When must a clinician act? At what level would the risk of suicide have to be for the information to be actionable? Moving forward, these issues must be carefully addressed, both from patient safety and provider liability perspectives.

INTEGRATING ACTIVE AND PASSIVE EMA

Depression symptomatology includes both subjective and objective symptoms. Psychological symptoms such as depressed mood, guilt and negative beliefs, and suicidality are subjective in nature (i.e., these symptoms depend on patients’ subjective experience and perception of their status). Assessment of these symptoms requires clinical interview and/or use of depression questionnaires. Similarly, patient’s distress due to depression (criterion B), a required criterion, is also subjective and requires clinical assessment. Active EMA may be necessary to fully evaluate these subjective symptoms and criteria. One may argue that behavioral and functional consequences of these symptoms can be captured using passive EMA, providing a more comprehensive assessment of these symptoms.

Neurovegetative and neurocognitive symptoms such as fatigue, sleep disturbances, psychomotor agitation/retardation, and concentration difficulties are objective symptoms with direct behavioral manifestations. Active EMA using interview and depression questionnaires may provide assessment of these symptoms based on patient perception of these symptoms but may fail to capture the actual behavioral manifestations. Similarly, functional impairment, another essential criterion (criterion B) for depression diagnosis, can be more fully captured using passive EMA. Similar to the subjective symptoms, patients’ own assessment and perception of their status assessed using clinical interview and depression questionnaires (active EMA) can provide a more comprehensive assessment of objective symptoms. In summary, at present time, utilization of both active and passive EMA may be necessary to generate a more comprehensive digital phenotype of the patient[13].

LifeRhythm: Integration of active and passive EMA to predict depression symptomatology

Our group, in a 4-year project funded by the National Science Foundation, demonstrated successful prediction of depression symptomatology integrating active and passive EMA (Figure 3). The LifeRhythm project involved a two-phase study conducted in college age participants with depression, in comparison with a control group without depression diagnosis[14-16,52]. In Phase I of the project, a smart phone application, LifeRhythm, was developed to passively collect sensory data (location, activity, social interaction) for both Android and iOS, the two predominant smartphone platforms. Feature extraction techniques were developed to extract behavioral features from the sensory data as correlates of depression symptomatology and machine-learning models were developed to predict self-report depression questionnaire scores and depression status. These techniques and prediction models were then validated and refined in Phase II of the study. In Phase II, wristbands (Fitbit devices) were added to the sensory diagnostics for characterizing specific behavioral features (e.g., sleep disturbances and activity level). A total of 182 participants were recruited in this two-phase study and were followed over an 8 month study period. Three sets of data were collected during participant’s study participation: sensory data collected by the LifeRhythm app (EMA passive), self-report depression questionnaire completed electronically by the participant every two weeks (EMA active), and clinical assessments conducted by a study clinician. Study findings demonstrated that passive sensory data (EMA passive) predicted self-report depression scores and depression status per clinical interview conducted by the study clinician[14-16,52]. Notably, integration of passive sensing (EMA passive) and self-report depression scores (EMA active) showed better prediction power compared to passive or active EMA alone.

Figure 3
Figure 3 LifeRhythm: Integration of active and passive ecological momentary assessment to predict depression. Adapted from Ware et al[86] with permission from the Association for Computing Machinery (ACM) Citation: Ware S, Yue C, Morillo R, Lu J, Shang C, Kamath J, Bamis A, Bi J, Russell A, Wang B. Large-scale Automatic Depression Screening Using Meta-data from WiFi Infrastructure. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2018; 2: 1-27. Copyright © The Association for Computing Machinery (ACM).
DepWatch: Integrating active and passive EMA in clinical setting to predict treatment response

At present, we are investigating development of a depression prediction system, DepWatch, and its integration in clinical setting to inform the clinical decision making process (Figure 4). This 4-year project, funded by the National Institute of Mental Health, builds on the findings and insights gained from the LifeRhythm project[13]. It includes two study phases. The objective of Phase I is to develop machine learning models to predict response or lack of response to antidepressant treatment, when patients meeting a specific threshold for depression symptoms undergo adjustments to their antidepressant medication regimen. Similar to the LifeRhythm project, passive sensory data (EMA passive) is collected using the app developed by our team for both Android and iOS platforms. Active EMA conducted electronically include daily self-report mood and anxiety ratings, weekly self-report depression questionnaire, weekly self-report medication safety and tolerability assessments, and other clinical information collected at baseline. Participants also undergo monthly clinical assessments conducted by a study clinician to assess their depression status and their response/non-response to antidepressant treatment compared to their baseline status. A total of 250 participants meeting a specific threshold for depression severity and starting or adjusting antidepressant treatment are currently being enrolled in the Phase I. Machine learning models will be developed using passive and active EMA data. DepWatch, an automatic data collection, analytic, and prediction system will be developed based on the machine learning algorithms developed in Phase I and other relevant clinical information. In Phase II, the DepWatch prediction system will be investigated for its usefulness and applications as a clinical support system in the real-world clinical setting, compared to standard of care. Three clinicians will use DepWatch to support their clinical decision making process for their patients. A total of 128 participants under care of the three participating clinicians will be enrolled in Phase II[13].

Figure 4
Figure 4 DepWatch: Integrating active and passive ecological momentary assessment in clinical setting. Adapted from Kamath et al[13] an open access article distributed under the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/) with permission from the J Psychiatr Brain Sci (JPBS). Citation: Kamath J, Bi J, Russell A, Wang B. Grant Report on SCH: Personalized Depression Treatment Supported by Mobile Sensor Analytics. J Psychiatr Brain Sci 2020; 5: e200010. Copyright © The J Psychiatr Brain Sci (JPBS).
PREDICTING DEPRESSION STATUS USING OTHER DIGITAL TOOLS
Predicting depression status using social media

Preliminary investigations are exploring behavioral and linguistic cues from social media data to predict depression status. Data can be extracted from a variety of social platforms including popular sites such as Twitter, Facebook, Instagram, and Reddit[59,60]. These investigations have used several variables/features of interest in social media data that may predict depression status. These include Language analyses (e.g., length, characteristics of the posts), Emotion and Cognition analyses (e.g., affect and intensity of posts reflecting anxiety or anger), Behavior analyses (e.g., posting frequency, interaction with others on the platform), Demographics analyses (e.g., age, gender inferred using computational techniques), and Image analysis (e.g., visual information from the images posted)[59]. Machine learning and statistical modeling are applied to the extracted data to develop and validate algorithms to predict depression status[59,60]. At present, the major limitation of this promising area of research includes the “ground truth” definition of depression and the methods used to identify and operationalize depression status[59]. Some studies have demonstrated strong construct validity by using evidence-based and clinically-relevant practices to define depression (e.g., use of depression questionnaire or use of ICD-10 diagnostic codes)[59,61]. Despite these current limitations, data mining from social media has a promising future in digital phenotyping. This innovative tool, in conjunction with EMA, can be used to augment digital phenotyping in depression diagnostics.

Predicting depression status using EMR

Digital phenotyping of depression status can be enhanced by using extracted data from EMRs[60,62]. Studies conducted to date have primarily utilized features (extracted from EMR) interdependent with depression diagnosis to predict clinical depression. Such features include depression billing codes, medication information, and structured and unstructured notes containing explicit diagnostic information. Computational methods, such as natural language processing (NLP), have been developed to extract data from narrative clinical notes in EMR. NPL is an automated method of extracting and processing text into meaningful concepts based on a set of rules[63]. Recent studies have used non-psychiatric features in EMR and have applied machine learning approaches to the extracted data to predict depression status[62]. These EMR data extraction techniques can be used in conjunction with EMA to improve depression diagnostics as part of digital phenotyping strategy.

TELEPSYCHIATRY

The use of teleconferencing technology in psychiatry dates back to the 1950s, when the Nebraska Psychiatric Institute started using teleconferencing to provide group therapy, consultation-liaison services, and medical student training[64]. Initial research focused mainly on increasing access to care in remote geographical areas and comparing the efficacy of video visits with in-person visits[65]. Growth of telepsychiatry was slow and patchy until recently. This was primarily due to technological challenges and usability issues, lack of willingness among healthcare professionals to modify well-established routines (e.g., face to face interactions), lack of financial resources, and lack of organizational innovation[66,67]. For decades, telepsychiatry was considered effective and feasible, but not desirable.

With the COVID-19 pandemic of 2020, there was a paradigm shift. The personnel and financial barriers to the use of telepsychiatry were removed overnight, and practices across the United States transitioned to telehealth. The number of telehealth visits increased by 50% over the first quarter of 2020, compared with the same period in 2019[68].

The efficacy of telepsychiatry has been well established over the past few decades[69,70]. Multiple reviews have analyzed studies of various telepsychiatry outcomes, including feasibility, adherence, clinical outcomes, and cost. One review of 22 controlled studies concluded that telepsychiatry could adequately perform all functions of management of mental illness, including monitoring, surveillance, mental health promotion, mental illness prevention, and biopsychosocial treatment programs, more efficiently and as well as or more effectively than in-person care[71]. Other reviews have reported similar results[72,73].

Telepsychiatry: Challenges and opportunities

Challenges of widespread, successful adoption of telepsychiatry practice can be divided into systemic challenges and personnel challenges. Systemic challenges include federal and state licensure and reimbursement policies that restrict the use of telepsychiatry, platform and internet bandwidth issues, availability of leadership support, and the “digital divide”, which describes a lack of reliable device/internet access in underserved populations. Personnel challenges include a lack of clinician training and support, fear of technology amongst both patients and providers, physical and cognitive disabilities that limit the use of technology, patient safety issues, and provider concern that telepsychiatry does not provide the same range and depth of data that is provided in an in-person encounter[74,75].

One way to address this concern about the lack of personal interaction with the patient is to integrate EMA and DP based approaches with telepsychiatry visits. Incorporating both passive and active EMA data with the information available to the clinician might not only address the concern about the availability of “real time” patient data to the clinician, it may also augment and improve the clinician’s ability to accurately assess the neurovegetative symptoms of depression such as sleep and activity. In a study by Moore et al, sixty-seven older adults completed paper-and-pencil measures of mindfulness, depression, and anxiety along with two weeks of identical items reported during ambulatory monitoring via EMA before and after participation in a randomized trial of Mindfulness-Based Stress Reduction (MBSR). EMA measures of depression substantially outperformed paper-and-pencil measures with the same items[76].

Passive and active EMA may improve the clinician’s ability to predict and diagnose depression in underdiagnosed subgroups such as older adults[77]. Incorporating active EMA approaches more frequently may allow clinicians to increase engagement with an isolated, depressed patient. Combining EMA with telepsychiatry may improve access to care for patients with anergia/amotivation, and offers the opportunity to provide rapid interventions based on activity data[78].

CLINICAL INTEGRATION OF DIGITAL PHENOTYPING

The therapeutic alliance between patients and their provider is the cornerstone of depression care. It is well established that a strong therapeutic relationship is a robust predictor for treatment response across all therapeutic interventions, including pharmacological interventions[79]. The current model of clinical care has a significant negative impact on this therapeutic relationship due to brief medication management visits, fragmentation of care, limited contact between patients and their clinicians, and lack of meaningful monitoring in between patients’ clinic visits. One of the objectives of integrating DP into clinical care is to enhance the therapeutic relationship between patients and their providers[80]. A digital connection between patients and their providers and monitoring via active and passive EMA in between patients’ clinic visits can reinforce the therapeutic relationship[80]. The other major objective of using DP is to improve accuracy and clinical relevance of diagnostic assessment. As noted earlier, depression assessment should evaluate three major areas: depression symptoms, patient distress, and impact on functioning. Current clinical assessment focuses primarily on patient symptoms and distress. Digital data can enhance assessment of symptoms and distress (e.g., use of active EMA in-between visits). More importantly, digital data, specifically passive EMA, can greatly enhance clinical assessments by providing objective data on behavioral consequences of symptoms/distress with its impact on functioning. As shown in Figure 5, DP and other digital tools can be incorporated into clinical practice at multiple stages of depression diagnostics and management. Initial patient evaluation (in-person) can be improved using patient specific data gathered from EMR using machine learning algorithms. Active and passive EMA can provide continuous monitoring in between patient visits and inform patient-provider discussion and assessment during in-person or virtual visits. These digital and in-person interactions between patients and their providers can increase patients’ engagement in their care and support shared decision-making. Use of virtual telepsychiatry visits interspersed by in-person visits can help increase frequency of patient-provider contact, further strengthening the therapeutic relationship.

Figure 5
Figure 5 Hybrid clinical care model: Integration of in-person and digital care. EMA: Ecological momentray assessment; EMR: Electronic medical record.
MACHINE LEARNING AND FUTURE OF DIGITAL PHENOTYPING

Current diagnostic systems, DSM-5 and ICD-11, were originally conceived using careful observations of symptoms by expert clinicians[21]. These taxonomies are useful for grouping individuals into broad diagnostic categories but it is becoming increasingly evident that the diagnostic categories lack neurobiological validity as well as clinical predictability[81]. It is also becoming evident that these diagnostic categories are spectrum disorders with heterogeneous clinical presentations and diverse underlying etiological and pathophysiological factors[81]. The current ‘best-possible’ evidence-informed treatment choices are successful only in limited number of patients partially due to this heterogeneity of clinical presentations with diverse underlying pathophysiology[82]. To address this critical gap, the National Institute of Mental Health (NIMH) launched a research initiative called the Research Domain Criteria (RDoC) project[83]. The RDoC initiative, a translational program, intends to synergistically integrate self-reports, neuropsychological tests, brain measurements, and genetic profiles to create precision medicine in psychiatry[83]. Machine learning approaches offer a rich set of tools towards achieving the goal of endophenotype modelling proposed by the RDoC initiative[84]. Machine learning models developed for the field of psychiatry are typically supervised machine learning models that employ a two-step process: training and testing. The collected data is divided into training and testing datasets. A learning algorithm is first fitted on the training dataset to train the model. The ‘trained’ model is then empirically evaluated by testing it on the testing dataset[84]. This two-step approach is consistent with the ‘precision psychiatry’ objectives of the RDoC initiative[83,84]. Data gathered from diverse retrospective and prospective datasets (e.g., genetic profiles, neuroimaging, EMR, active and passive EMA data) can be integrated and analyzed using machine learning approaches to generate objectively measurable and clinically predictable endophenotypes. The models generated can then be validated in a new set of patients to predict clinical outcomes including treatment outcomes. The machine learning approaches can translate complex discoveries into clinically relevant predictions bringing us closer to the goal of precision psychiatry.

CONCLUSION

If we are to fulfill the promise of DP in depression diagnostics, it is critical that teams of psychiatric and engineering researchers work together to address the numerous challenges we have described. All investigations and digital tools under development should be scrutinized for their clinical relevance and real-world applicability. Investigations in the field of DP, to date, are spearheaded primarily by engineers with limited involvement of psychiatric researchers. This is problematic because, at present, clinical acumen of psychiatric clinicians play a central role in depression diagnosis, assessment, and management. The purportedly objective measures (e.g., depression questionnaires) are important tools, yet remain subjective in nature and play a limited secondary role in clinical settings. The field of DP needs to draw upon the experience and expertise of psychiatric clinicians as ‘ground truth’ combined with depression questionnaires. It is essential to include psychiatric investigators who have background and expertise in clinical care and clinical research into the research team. A major role of clinical investigators as part of the research team would be to assess clinical relevance of digital tools under development compared to the standard of clinical care.

Once the digital tools show promise in predicting depression status as assessed by the ‘ground truth’ (clinical judgment and depression questionnaires), the next step would be to challenge the subjectivity of the ‘ground truth’ by focusing on a different, objectively measurable outcome. As noted earlier, depression questionnaires and clinician interview are fundamentally subjective as they rely on patients’ memory/perception and on clinicians’ clinical judgment. In comparison, change in functioning with its behavioral manifestations may be a better and a more objective ‘ground truth’. In clinical setting, change in functioning is considered an important marker of depression status as it reflects depression symptoms, distress, and is associated with objective behavioral consequences. Furthermore, change in functioning with its behavioral consequences can be quantified objectively using DP tools. In the past decade, depression research has been striving towards ‘remission’ as an outcome[85,86]. This goal of achieving remission is directly related to patients’ functional improvement. DP may provide us with objective tools to measure both remission and functional improvement.

In conclusion, we live in a time when most of the global population carry smart phones in their pockets and broadband access is rapidly increasing even in remote areas. DP based on smart phones and other digital tools can significantly enhance depression diagnostics. Objective continuous measurement of behavioral manifestations of depression using patients’ own devices can provide clinically useful markers. Such ‘behavioral biomarkers’ can be used to refine diagnostic processes and management. These objective markers (passive EMA) combined with assessments conducted in patients’ milieu (active EMA) and strengthened therapeutic relationship and monitoring due to continuous digital connection between patients and their providers can help us move closer to the goal of personalized and patient-centered care.

Footnotes

Provenance and peer review: Invited article; Externally peer reviewed.

Peer-review model: Single blind

Specialty type: Psychiatry

Country/Territory of origin: United States

Peer-review report’s scientific quality classification

Grade A (Excellent): A, A

Grade B (Very good): 0

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Alkhatib AJ, Liu X S-Editor: Wang LL L-Editor: A P-Editor: Cai YX

References
1.  World Health Organization 2020  [cited 26 June 2021]. Available from: https://www.who.int/news-room/fact-sheets/detail/depression.  [PubMed]  [DOI]  [Cited in This Article: ]
2.  World Health Organization 2017  Depression and other Common Mental Health Disorders: Global Health Estimates. [cited 26 June 2021]. Available from: https://www.who.int/publications/i/item/depression-global-health-estimates.  [PubMed]  [DOI]  [Cited in This Article: ]
3.  GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392:1789-1858.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7855]  [Cited by in F6Publishing: 7199]  [Article Influence: 1199.8]  [Reference Citation Analysis (2)]
4.  National Institute of Mental Health  Major Depression. [cited 26 June 2021]. Available from: https://www.nimh.nih.gov/health/statistics/major-depression#part_155029.  [PubMed]  [DOI]  [Cited in This Article: ]
5.  Greenberg PE, Fournier AA, Sisitsky T, Pike CT, Kessler RC. The economic burden of adults with major depressive disorder in the United States (2005 and 2010). J Clin Psychiatry. 2015;76:155-162.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1093]  [Cited by in F6Publishing: 1085]  [Article Influence: 120.6]  [Reference Citation Analysis (0)]
6.  Maj M, Stein DJ, Parker G, Zimmerman M, Fava GA, De Hert M, Demyttenaere K, McIntyre RS, Widiger T, Wittchen HU. The clinical characterization of the adult patient with depression aimed at personalization of management. World Psychiatry. 2020;19:269-293.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 193]  [Cited by in F6Publishing: 173]  [Article Influence: 43.3]  [Reference Citation Analysis (0)]
7.  Cohen ZD, DeRubeis RJ. Treatment Selection in Depression. Annu Rev Clin Psychol. 2018;14:209-236.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 197]  [Cited by in F6Publishing: 210]  [Article Influence: 35.0]  [Reference Citation Analysis (0)]
8.  Insel TR. Digital Phenotyping: Technology for a New Science of Behavior. JAMA. 2017;318:1215-1216.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 447]  [Cited by in F6Publishing: 368]  [Article Influence: 52.6]  [Reference Citation Analysis (0)]
9.  Colombo D, Fernández-Álvarez J, Patané A, Semonella M, Kwiatkowska M, García-Palacios A, Cipresso P, Riva G, Botella C. Current State and Future Directions of Technology-Based Ecological Momentary Assessment and Intervention for Major Depressive Disorder: A Systematic Review. J Clin Med. 2019;8.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 123]  [Cited by in F6Publishing: 90]  [Article Influence: 18.0]  [Reference Citation Analysis (0)]
10.  Rohani DA, Faurholt-Jepsen M, Kessing LV, Bardram JE. Correlations Between Objective Behavioral Features Collected From Mobile and Wearable Devices and Depressive Mood Symptoms in Patients With Affective Disorders: Systematic Review. JMIR Mhealth Uhealth. 2018;6:e165.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 91]  [Cited by in F6Publishing: 86]  [Article Influence: 14.3]  [Reference Citation Analysis (8)]
11.  Dogan E, Sander C, Wagner X, Hegerl U, Kohls E. Smartphone-Based Monitoring of Objective and Subjective Data in Affective Disorders: Where Are We and Where Are We Going? J Med Internet Res. 2017;19:e262.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 118]  [Cited by in F6Publishing: 96]  [Article Influence: 13.7]  [Reference Citation Analysis (0)]
12.  Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, Sarris J. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017;16:287-298.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 696]  [Cited by in F6Publishing: 506]  [Article Influence: 72.3]  [Reference Citation Analysis (0)]
13.  Kamath J, Bi J, Russell A, Wang B. Grant Report on SCH: Personalized Depression Treatment Supported by Mobile Sensor Analytics. J Psychiatr Brain Sci. 2020;5.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3]  [Cited by in F6Publishing: 3]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
14.  Ware S, Yue C, Lu J, Chao S, Jinbo B, Kamath J, Russel A, Bamis A, Wang B.   Predicting Depressive Symptoms Using Smartphone Data. Smart Health 2020, 100093.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 30]  [Cited by in F6Publishing: 8]  [Article Influence: 2.0]  [Reference Citation Analysis (2)]
15.  Yue C, Ware S, Morillo R, Lu J, Shang C, Bi J, Kamath J, Russell A, Bamis A, Wang B. Automatic Depression Prediction Using Internet Traffic Characteristics on Smartphones. Smart Health (Amst). 2020;18.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 4]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
16.  Yue C, Ware S, Morillo R, Lu J, Shang C, Bi J, Kamath J, Russel A, Bamis A, Wang B.   Fusing Location Data for Depression Prediction In: 2017 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computed, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI); 2017 Aug 4-8; San Francisco, USA. New York (US): IEEE, 2018.  [PubMed]  [DOI]  [Cited in This Article: ]
17.  Zimmerman M, Ellison W, Young D, Chelminski I, Dalrymple K. How many different ways do patients meet the diagnostic criteria for major depressive disorder? Compr Psychiatry. 2015;56:29-34.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 160]  [Cited by in F6Publishing: 173]  [Article Influence: 19.2]  [Reference Citation Analysis (0)]
18.  Thase ME. The multifactorial presentation of depression in acute care. J Clin Psychiatry. 2013;74 Suppl 2:3-8.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 45]  [Cited by in F6Publishing: 42]  [Article Influence: 4.2]  [Reference Citation Analysis (0)]
19.  Pitsillou E, Bresnehan SM, Kagarakis EA, Wijoyo SJ, Liang J, Hung A, Karagiannis TC. The cellular and molecular basis of major depressive disorder: towards a unified model for understanding clinical depression. Mol Biol Rep. 2020;47:753-770.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 56]  [Cited by in F6Publishing: 85]  [Article Influence: 17.0]  [Reference Citation Analysis (0)]
20.  Malhi GS, Mann JJ. Depression. Lancet. 2018;392:2299-2312.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1255]  [Cited by in F6Publishing: 1965]  [Article Influence: 327.5]  [Reference Citation Analysis (0)]
21.  American Psychiatric Association  Diagnostic and statistical manual of mental disorders (5th edition), 2013. [cited 26 June 2021]. Available from: https://doi.org/10.1176/appi.books.9780890425596.  [PubMed]  [DOI]  [Cited in This Article: ]
22.  World Health Organization  International statistical classification of diseases and related health problems (11th edition), 2019. [cited 26 June 2021]. Available from: https://icd.who.int/.  [PubMed]  [DOI]  [Cited in This Article: ]
23.  Kendler KS. The Phenomenology of Major Depression and the Representativeness and Nature of DSM Criteria. Am J Psychiatry. 2016;173:771-780.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 115]  [Cited by in F6Publishing: 119]  [Article Influence: 14.9]  [Reference Citation Analysis (0)]
24.  Lakkis NA, Mahmassani DM. Screening instruments for depression in primary care: a concise review for clinicians. Postgrad Med. 2015;127:99-106.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 33]  [Cited by in F6Publishing: 29]  [Article Influence: 2.9]  [Reference Citation Analysis (0)]
25.  Uher R, Perlis RH, Placentino A, Dernovšek MZ, Henigsberg N, Mors O, Maier W, McGuffin P, Farmer A. Self-report and clinician-rated measures of depression severity: can one replace the other? Depress Anxiety. 2012;29:1043-1049.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 134]  [Cited by in F6Publishing: 155]  [Article Influence: 12.9]  [Reference Citation Analysis (0)]
26.  Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, Markowitz JC, Ninan PT, Kornstein S, Manber R, Thase ME, Kocsis JH, Keller MB. The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry. 2003;54:573-583.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2297]  [Cited by in F6Publishing: 2521]  [Article Influence: 120.0]  [Reference Citation Analysis (0)]
27.  Maurer DM, Raymond TJ, Davis BN. Depression: Screening and Diagnosis. Am Fam Physician. 2018;98:508-515.  [PubMed]  [DOI]  [Cited in This Article: ]
28.  Park SC, Kim JM, Jun TY, Lee MS, Kim JB, Yim HW, Park YC. How many different symptom combinations fulfil the diagnostic criteria for major depressive disorder? Nord J Psychiatry. 2017;71:217-222.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 22]  [Article Influence: 3.1]  [Reference Citation Analysis (0)]
29.  O'Dowd T. Depression and multimorbidity in psychiatry and primary care. J Clin Psychiatry. 2014;75:e1319-e1320.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 6]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
30.  Hong RH, Murphy JK, Michalak EE, Chakrabarty T, Wang Z, Parikh SV, Culpepper L, Yatham LN, Lam RW, Chen J. Implementing Measurement-Based Care for Depression: Practical Solutions for Psychiatrists and Primary Care Physicians. Neuropsychiatr Dis Treat. 2021;17:79-90.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 27]  [Cited by in F6Publishing: 22]  [Article Influence: 7.3]  [Reference Citation Analysis (0)]
31.  Robinson J, Khan N, Fusco L, Malpass A, Lewis G, Dowrick C. Why are there discrepancies between depressed patients' Global Rating of Change and scores on the Patient Health Questionnaire depression module? BMJ Open. 2017;7:e014519.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 17]  [Article Influence: 2.4]  [Reference Citation Analysis (0)]
32.  Hobbs C, Lewis G, Dowrick C, Kounali D, Peters TJ. Comparison between self-administered depression questionnaires and patients' own views of changes in their mood: a prospective cohort study in primary care. Psychol Med. 2021;51:853-860.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 10]  [Cited by in F6Publishing: 14]  [Article Influence: 4.7]  [Reference Citation Analysis (0)]
33.  Urban EJ, Charles ST, Levine LJ, Almeida DM. Depression history and memory bias for specific daily emotions. PLoS One. 2018;13:e0203574.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 41]  [Cited by in F6Publishing: 35]  [Article Influence: 5.8]  [Reference Citation Analysis (0)]
34.  Torous J, Staples P, Onnela JP. Realizing the potential of mobile mental health: new methods for new data in psychiatry. Curr Psychiatry Rep. 2015;17:602.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 128]  [Cited by in F6Publishing: 96]  [Article Influence: 10.7]  [Reference Citation Analysis (0)]
35.  Onnela JP, Rauch SL. Harnessing Smartphone-Based Digital Phenotyping to Enhance Behavioral and Mental Health. Neuropsychopharmacology. 2016;41:1691-1696.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 342]  [Cited by in F6Publishing: 309]  [Article Influence: 38.6]  [Reference Citation Analysis (0)]
36.  Huckvale K, Venkatesh S, Christensen H. Toward clinical digital phenotyping: a timely opportunity to consider purpose, quality, and safety. NPJ Digit Med. 2019;2:88.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 165]  [Cited by in F6Publishing: 141]  [Article Influence: 28.2]  [Reference Citation Analysis (0)]
37.  Yim SJ, Lui LMW, Lee Y, Rosenblat JD, Ragguett RM, Park C, Subramaniapillai M, Cao B, Zhou A, Rong C, Lin K, Ho RC, Coles AS, Majeed A, Wong ER, Phan L, Nasri F, McIntyre RS. The utility of smartphone-based, ecological momentary assessment for depressive symptoms. J Affect Disord. 2020;274:602-609.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 24]  [Cited by in F6Publishing: 31]  [Article Influence: 7.8]  [Reference Citation Analysis (0)]
38.  Hirschtritt ME, Insel TR. Digital Technologies in Psychiatry: Present and Future. Focus (Am Psychiatr Publ). 2018;16:251-258.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 25]  [Cited by in F6Publishing: 27]  [Article Influence: 4.5]  [Reference Citation Analysis (0)]
39.  Dwyer DB, Falkai P, Koutsouleris N. Machine Learning Approaches for Clinical Psychology and Psychiatry. Annu Rev Clin Psychol. 2018;14:91-118.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 341]  [Cited by in F6Publishing: 400]  [Article Influence: 66.7]  [Reference Citation Analysis (0)]
40.  Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1-32.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2845]  [Cited by in F6Publishing: 3122]  [Article Influence: 195.1]  [Reference Citation Analysis (0)]
41.  Schueller SM, Aguilera A, Mohr DC. Ecological momentary interventions for depression and anxiety. Depress Anxiety. 2017;34:540-545.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 88]  [Cited by in F6Publishing: 72]  [Article Influence: 10.3]  [Reference Citation Analysis (0)]
42.  Kim H, Kim S, Kong SS, Jeong Y-R, Kim H, Kim N. Possible Application of Ecological Momentary Assessment to Older Adults’ Daily Depressive Mood: Integrative Literature Review. JMIR Ment Heal. 2020;7:e13247.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 6]  [Cited by in F6Publishing: 5]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
43.  Wichers M, Simons CJP, Kramer IMA, et al Momentary assessment technology as a tool to help patients with depression help themselves.   Acta Psychiatr Scand 2011; 124: 262-272.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 93]  [Cited by in F6Publishing: 84]  [Article Influence: 6.5]  [Reference Citation Analysis (0)]
44.  Gratch I, Choo TH, Galfalvy H, Keilp JG, Itzhaky L, Mann JJ, Oquendo MA, Stanley B. Detecting suicidal thoughts: The power of ecological momentary assessment. Depress Anxiety. 2021;38:8-16.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 38]  [Cited by in F6Publishing: 52]  [Article Influence: 17.3]  [Reference Citation Analysis (0)]
45.  Liang Y, Zheng X, Zeng D. A survey on big data-driven digital phenotyping of mental health. Information Fusion. 2019;52:290-307.  [PubMed]  [DOI]  [Cited in This Article: ]
46.  Ayadi M, Kamel M, Kartay F. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition. 2011;44:572-587.  [PubMed]  [DOI]  [Cited in This Article: ]
47.  Dagum P. Digital biomarkers of cognitive function. NPJ Digit Med. 2018;1:10.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 89]  [Cited by in F6Publishing: 91]  [Article Influence: 15.2]  [Reference Citation Analysis (0)]
48.  Cornet VP, Holden RJ. Systematic review of smartphone-based passive sensing for health and wellbeing. J Biomed Inform. 2018;77:120-132.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 166]  [Cited by in F6Publishing: 147]  [Article Influence: 21.0]  [Reference Citation Analysis (0)]
49.  Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E, Mohr DC. Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res. 2011;13:e55.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 417]  [Cited by in F6Publishing: 289]  [Article Influence: 22.2]  [Reference Citation Analysis (0)]
50.  Palmius N, Tsanas A, Saunders KEA, Bilderbeck AC, Geddes JR, Goodwin GM, De Vos M. Detecting Bipolar Depression From Geographic Location Data. IEEE Trans Biomed Eng. 2017;64:1761-1771.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 80]  [Cited by in F6Publishing: 80]  [Article Influence: 10.0]  [Reference Citation Analysis (0)]
51.  Saeb S, Zhang M, Karr CJ, Schueller SM, Corden ME, Kording KP, Mohr DC. Mobile Phone Sensor Correlates of Depressive Symptom Severity in Daily-Life Behavior: An Exploratory Study. J Med Internet Res. 2015;17:e175.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 442]  [Cited by in F6Publishing: 318]  [Article Influence: 35.3]  [Reference Citation Analysis (0)]
52.  Farhan A, Yue C, Morillo R, Ware S, Lu J, Bi J, Kamath J, Russell A, Bamis A, Wang B. Behavior vs. introspection: Refining prediction of clinical depression via smartphone sensing data. Proc. of Wireless Health, 2016.  [PubMed]  [DOI]  [Cited in This Article: ]
53.  Lu J, Shang C, Yue C, Morillo R, Ware S, Kamath J, Russell A, Bamis A, Wang B, Bi J.   Joint Modeling of Heterogeneous Sensing Data for Depression Assessment via Multi-task Learning, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2018.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 32]  [Cited by in F6Publishing: 15]  [Article Influence: 2.5]  [Reference Citation Analysis (0)]
54.  Keisari EJ, Patel P, Wang B, Kamath J.   Investigation of mobile health (mHealth) technologies in the management of depression: Ethical, legal, and regulatory challenges. American Society of Clinical Psychopharmacology Annual Meeting, 2020.  [PubMed]  [DOI]  [Cited in This Article: ]
55.  Torous J, Roberts LW. The Ethical Use of Mobile Health Technology in Clinical Psychiatry. J Nerv Ment Dis. 2017;205:4-8.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 82]  [Cited by in F6Publishing: 100]  [Article Influence: 14.3]  [Reference Citation Analysis (0)]
56.  Roberts LW  A clinical guide to psychiatric ethics. Arlington, VA: American Psychiatric Publishing, Inc. 2016.  [PubMed]  [DOI]  [Cited in This Article: ]
57.  Cvrkel T. The ethics of mHealth: Moving forward. J Dent. 2018;74 Suppl 1:S15-S20.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 22]  [Article Influence: 4.4]  [Reference Citation Analysis (0)]
58.  Armontrout J, Torous J, Fisher M, Drogin E, Gutheil T. Mobile Mental Health: Navigating New Rules and Regulations for Digital Tools. Curr Psychiatry Rep. 2016;18:91.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 47]  [Cited by in F6Publishing: 36]  [Article Influence: 4.5]  [Reference Citation Analysis (0)]
59.  Chancellor S, De Choudhury M. Methods in predictive techniques for mental health status on social media: a critical review. NPJ Digit Med. 2020;3:43.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 194]  [Cited by in F6Publishing: 82]  [Article Influence: 20.5]  [Reference Citation Analysis (6)]
60.  Su C, Xu Z, Pathak J, Wang F. Deep learning in mental health outcome research: a scoping review. Transl Psychiatry. 2020;10:116.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 79]  [Cited by in F6Publishing: 70]  [Article Influence: 17.5]  [Reference Citation Analysis (10)]
61.  Eichstaedt JC, Smith RJ, Merchant RM, Ungar LH, Crutchley P, Preoţiuc-Pietro D, Asch DA, Schwartz HA. Facebook language predicts depression in medical records. Proc Natl Acad Sci U S A. 2018;115:11203-11208.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 344]  [Cited by in F6Publishing: 196]  [Article Influence: 32.7]  [Reference Citation Analysis (6)]
62.  Nemesure MD, Heinz MV, Huang R, Jacobson N. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep. 2021;11:1980.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 32]  [Cited by in F6Publishing: 47]  [Article Influence: 15.7]  [Reference Citation Analysis (0)]
63.  Perlis RH, Iosifescu DV, Castro VM, Murphy SN, Gainer VS, Minnier J, Cai T, Goryachev S, Zeng Q, Gallagher PJ, Fava M, Weilburg JB, Churchill SE, Kohane IS, Smoller JW. Using electronic medical records to enable large-scale studies in psychiatry: treatment resistant depression as a model. Psychol Med. 2012;42:41-50.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 127]  [Cited by in F6Publishing: 118]  [Article Influence: 9.8]  [Reference Citation Analysis (0)]
64.  Bashshur R, Shannon GW. History of telemedicine: evolution, context, and transformation. Healthc Inform Res. 2010;16:65-66.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 32]  [Article Influence: 2.3]  [Reference Citation Analysis (0)]
65.  Hilty DM, Marks SL, Urness D, Yellowlees PM, Nesbitt TS. Clinical and educational telepsychiatry applications: a review. Can J Psychiatry. 2004;49:12-23.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 148]  [Cited by in F6Publishing: 155]  [Article Influence: 7.8]  [Reference Citation Analysis (0)]
66.  Sanders C, Rogers A, Bowen R, Bower P, Hirani S, Cartwright M, Fitzpatrick R, Knapp M, Barlow J, Hendy J, Chrysanthaki T, Bardsley M, Newman SP. Exploring barriers to participation and adoption of telehealth and telecare within the Whole System Demonstrator trial: a qualitative study. BMC Health Serv Res. 2012;12:220.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 304]  [Cited by in F6Publishing: 269]  [Article Influence: 22.4]  [Reference Citation Analysis (0)]
67.  Reginatto B  Addressing barriers to wider adoption of telehealth in the homes of older people: An exploratory study in the Irish context. In The fourth International Conference on eHealth, Telemedicine and Social Medicine 2012: 175-183.  [PubMed]  [DOI]  [Cited in This Article: ]
68.  Koonin LM, Hoots B, Tsang CA, Leroy Z, Farris K, Jolly T, Antall P, McCabe B, Zelis CBR, Tong I, Harris AM. Trends in the Use of Telehealth During the Emergence of the COVID-19 Pandemic -United States, January-March 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1595-1599.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 360]  [Cited by in F6Publishing: 589]  [Article Influence: 147.3]  [Reference Citation Analysis (0)]
69.  De Las Cuevas C, Arredondo MT, Cabrera MF, Sulzenbacher H, Meise U. Randomized clinical trial of telepsychiatry through videoconference vs face-to-face conventional psychiatric treatment. Telemed J E Health. 2006;12:341-350.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 108]  [Cited by in F6Publishing: 105]  [Article Influence: 5.8]  [Reference Citation Analysis (0)]
70.  Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: a 2013 review. Telemed J E Health. 2013;19:444-454.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 585]  [Cited by in F6Publishing: 543]  [Article Influence: 54.3]  [Reference Citation Analysis (0)]
71.  Bashshur RL, Shannon GW, Bashshur N, Yellowlees PM. The Empirical Evidence for Telemedicine Interventions in Mental Disorders. Telemed J E Health. 2016;22:87-113.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 247]  [Cited by in F6Publishing: 211]  [Article Influence: 26.4]  [Reference Citation Analysis (0)]
72.  Hubley S, Lynch SB, Schneck C, Thomas M, Shore J. Review of key telepsychiatry outcomes. World J Psychiatry. 2016;6:269-282.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 352]  [Cited by in F6Publishing: 287]  [Article Influence: 35.9]  [Reference Citation Analysis (4)]
73.  Berryhill MB, Culmer N, Williams N, Halli-Tierney A, Betancourt A, Roberts H, King M. Videoconferencing Psychotherapy and Depression: A Systematic Review. Telemed J E Health. 2019;25:435-446.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 133]  [Cited by in F6Publishing: 139]  [Article Influence: 27.8]  [Reference Citation Analysis (0)]
74.  Wagnild G, Leenknecht C, Zauher J. Psychiatrists' satisfaction with telepsychiatry. Telemed J E Health. 2006;12:546-551.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 47]  [Cited by in F6Publishing: 43]  [Article Influence: 2.4]  [Reference Citation Analysis (0)]
75.  Brooks E, Turvey C, Augusterfer EF. Provider barriers to telemental health: obstacles overcome, obstacles remaining. Telemed J E Health. 2013;19:433-437.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 99]  [Cited by in F6Publishing: 106]  [Article Influence: 9.6]  [Reference Citation Analysis (0)]
76.  Moore RC, Depp CA, Wetherell JL, Lenze EJ. Ecological momentary assessment vs standard assessment instruments for measuring mindfulness, depressed mood, and anxiety among older adults. J Psychiatr Res. 2016;75:116-123.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 142]  [Cited by in F6Publishing: 145]  [Article Influence: 18.1]  [Reference Citation Analysis (0)]
77.  Kim H, Lee S, Hong S, Kang H, Kim N. Depression Prediction by Using Ecological Momentary Assessment, Actiwatch Data, and Machine Learning: Observational Study on Older Adults Living Alone. JMIR Mhealth Uhealth. 2019;7:e14149.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 47]  [Cited by in F6Publishing: 53]  [Article Influence: 10.6]  [Reference Citation Analysis (0)]
78.  Hevel DJ, Dunton GF, Maher JP. Acute Bidirectional Relations Between Affect, Physical Feeling States, and Activity-Related Behaviors Among Older Adults: An Ecological Momentary Assessment Study. Ann Behav Med. 2021;55:41-54.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 14]  [Article Influence: 4.7]  [Reference Citation Analysis (0)]
79.  Lambert M, Barley D. Research summary on the therapeutic relationship and psychotherapy outcome. Psychother: Theory, Res Pract Train. 2001;38:357-361.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 625]  [Cited by in F6Publishing: 673]  [Article Influence: 29.3]  [Reference Citation Analysis (0)]
80.  Torous J, Hsin H. Empowering the digital therapeutic relationship: virtual clinics for digital health interventions. NPJ Digit Med. 2018;1:16.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 58]  [Cited by in F6Publishing: 46]  [Article Influence: 7.7]  [Reference Citation Analysis (0)]
81.  Insel TR, Cuthbert BN. Medicine. Brain disorders? Science. 2015;348:499-500.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 426]  [Cited by in F6Publishing: 461]  [Article Influence: 51.2]  [Reference Citation Analysis (0)]
82.  Wong EH, Yocca F, Smith MA, Lee CM. Challenges and opportunities for drug discovery in psychiatric disorders: the drug hunters' perspective. Int J Neuropsychopharmacol. 2010;13:1269-1284.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 70]  [Cited by in F6Publishing: 100]  [Article Influence: 7.1]  [Reference Citation Analysis (0)]
83.  Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, Sanislow C, Wang P. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167:748-751.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4107]  [Cited by in F6Publishing: 4041]  [Article Influence: 288.6]  [Reference Citation Analysis (0)]
84.  Bzdok D, Meyer-Lindenberg A. Machine Learning for Precision Psychiatry: Opportunities and Challenges. Biol Psychiatry Cogn Neurosci Neuroimaging. 2018;3:223-230.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 174]  [Cited by in F6Publishing: 217]  [Article Influence: 31.0]  [Reference Citation Analysis (4)]
85.  Lam RW, McIntosh D, Wang J, Enns MW, Kolivakis T, Michalak EE, Sareen J, Song WY, Kennedy SH, MacQueen GM, Milev RV, Parikh SV, Ravindran AV; CANMAT Depression Work Group. Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder: Section 1. Disease Burden and Principles of Care. Can J Psychiatry. 2016;61:510-523.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 162]  [Cited by in F6Publishing: 179]  [Article Influence: 22.4]  [Reference Citation Analysis (0)]
86.  Ware S, Yue C, Morillo R, Lu J, Shang C, Kamath J, Bamis A, Bi J, Russell A, Wang B.   Large-scale Automatic Depression Screening Using Meta-data from WiFi Infrastructure. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2018; 195: 1-27.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 5]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]