Abstract:

Undergraduate students are often impacted by depression, anxiety, and stress. In this context, machine learning may support mental health assessment. Based on the following research question: “How do machine learning models perform in the detection of depression, anxiety, and stress among undergraduate students?”, we aimed to evaluate the performance of these models. PubMed, Embase, PsycINFO, and Web of Science databases were searched, aiming at studies meeting the following criteria: publication in English; targeting undergraduate university students; empirical studies; having been published in a scientific journal; and predicting anxiety, depression, or stress outcomes via machine learning. The certainty of evidence was analyzed using the GRADE. As of January 2024, 2,304 articles were found, and 48 studies met the inclusion criteria. Different types of data were identified, including behavioral, physiological, internet usage, neurocerebral, blood markers, mixed data, as well as demographic and mobility data. Among the 33 studies that provided accuracy assessment, 30 reported values that exceeded 70%. Accuracy in detecting stress ranged from 63% to 100%, anxiety from 53.69% to 97.9%, and depression from 73.5% to 99.1%. Although most models present adequate performance, it should be noted that 47 of them only performed internal validation, which may overstate the performance data. Moreover, the GRADE checklist suggested that the quality of the evidence was very low. These findings indicate that machine learning algorithms hold promise in Public Health; however, it is crucial to scrutinize their practical applicability. Further studies should invest mainly in external validation of the machine learning models.

Keywords:
Students; Machine Learning; Mental Health

Resumo:

Os alunos de graduação são frequentemente afetados por depressão, ansiedade e estresse. O aprendizado de máquina pode apoiar a avaliação da saúde mental. Com base na seguinte questão de pesquisa “Qual é o desempenho dos modelos de aprendizado de máquina na detecção de depressão, ansiedade e estresse entre estudantes de graduação?”, objetivou-se avaliar o desempenho desses modelos. As pesquisas foram realizadas no PubMed, Embase, PsycINFO e Web of Science. Foram pesquisados estudos que atendessem aos seguintes critérios: publicados em inglês, estudantes universitários de graduação como população alvo, empíricos, publicados em uma revista científica e que previssem resultados de ansiedade, depressão ou estresse via aprendizado de máquina. A qualidade das evidências foi analisada usando o GRADE. Em janeiro de 2024, foram encontrados 2.304 artigos, e 48 estudos atenderam aos critérios de inclusão. Foram identificados diferentes tipos de dados, incluindo dados comportamentais, fisiológicos, de uso da Internet, neurocerebrais, marcadores sanguíneos, dados mistos, demográficos e de mobilidade. Entre os 33 estudos que forneceram dados de precisão, 30 relataram valores superiores a 70%. A acurácia na detecção de estresse variou de 63% a 100%, ansiedade de 53,69% a 97,9% e depressão de 73,5% a 99,1%. Embora a maioria dos modelos apresente desempenho adequado, deve-se notar que 47 deles realizaram apenas validação interna, o que pode superestimar os dados de desempenho. Além disso, a avaliação GRADE indicou que a qualidade da evidência é muito baixa. Os resultados indicam que os algoritmos de aprendizado de máquina são promissores no campo da Saúde Pública; no entanto, é crucial examinar sua aplicabilidade prática. Estudos futuros devem investir principalmente na validação externa dos modelos de aprendizado de máquina.

Palavras-chave:
Estudantes; Aprendizado de Máquina; Saúde Mental

Resumen:

Los estudiantes de grado suelen verse afectados por la depresión, la ansiedad y el estrés. El aprendizaje automático puede respaldar la evaluación de la salud mental. Con base en la siguiente pregunta de investigación “¿Cuál es el rendimiento de los modelos de aprendizaje automático en la detección de depresión, ansiedad y estrés entre estudiantes universitarios?”, nuestro objetivo fue evaluar el rendimiento de estos modelos. Se realizaron búsquedas en PubMed, Embase, PsycINFO y Web of Science. Se buscaron estudios que cumplieran con los siguientes criterios: se hubieran publicado en inglés, tuvieran a estudiantes universitarios como población objetivo, fueran empíricos, publicados en una revista científica y que predijeran resultados de ansiedad, depresión o estrés mediante aprendizaje automático. La calidad de las evidencias se analizó mediante GRADE. En enero del 2024 se encontraron 2.304 artículos, y 48 estudios cumplieron con los criterios de inclusión. Se identificaron diferentes tipos de datos, incluidos datos conductuales, fisiológicos, de uso de internet, neurocerebrales, marcadores sanguíneos, datos mixtos, demográficos y de movilidad. Entre los 33 estudios que proporcionaron datos de precisión, 30 reportaron valores superiores al 70%. La precisión en la detección del estrés osciló entre el 63% y el 100%, la ansiedad del 53,69% al 97,9% y la depresión del 73,5% al 99,1%. Aunque la mayoría de los modelos presenta un rendimiento adecuado, cabe señalar que 47 de ellos realizaron únicamente validación interna, lo que puede sobrestimar los datos de rendimiento. Además, la evaluación GRADE indicó que la calidad de la evidencia es muy baja. Los resultados indican que los algoritmos de aprendizaje automático son prometedores en el campo de la Salud Pública; sin embargo, es crucial examinar su aplicabilidad práctica. Los estudios futuros deberían invertir principalmente en la validación externa de los modelos de aprendizaje automático.

Palabras-clave:
Estudiantes; Aprendizaje Automático; Salud Mental

Introduction

University students, such as undergraduate students, are widely affected by mental disorders and psychopathological symptoms, particularly those linked to depressive moods, anxiety, stress, and drug addiction ¹1. Duffy A, Saunders KEA, Malhi GS, Patten S, Cipriani A, McNevin SH, et al. Mental health care for university students: a way forward? Lancet Psychiatry 2019; 6:885-7.^,22. Lopes AR, Nihei OK. Depression, anxiety, and stress symptoms in Brazilian university students during the COVID-19 pandemic: predictors and association with life satisfaction, psychological well-being and coping strategies. PLoS One 2021; 16:e0258493.. Among university students, 12% to 46% experience some impairment in mental health in the first academic year ³3. Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. WHO World Mental Health Surveys International College Student project: prevalence and distribution of mental disorders. J Abnorm Psychol 2018; 127:623-38.. The most recent survey by the World Health Organization (WHO) on university students’ mental health, which included eight countries and approximately 14,000 participants, indicated that approximately 35% of participants presented mental health impairments related to mood (depressive or maniac), anxiety, and drug use, with anxiety being the most prominent ³3. Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. WHO World Mental Health Surveys International College Student project: prevalence and distribution of mental disorders. J Abnorm Psychol 2018; 127:623-38.. These mental health impairments have worsened since the emergence of the COVID-19 pandemic ⁴4. Elharake JA, Akbar F, Malik AA, Gilliam W, Omer SB. Mental health impact of COVID-19 among children and college students: a systematic review. Child Psychiatry Hum Dev 2023; 54:913-25.^,55. Kim H, Rackoff GN, Fitzsimmons-Craft EE, Shin KE, Zainal NH, Schwob JT, et al. College mental health before and during the COVID-19 pandemic: results from a nationwide survey. Cognit Ther Res 2021; 46:1-10.^,66. Li Y, Wang A, Wu Y, Han N, Huang H. Impact of the COVID-19 pandemic on the mental health of college students: a systematic review and meta-analysis. Front Psychol 2021; 14:12..

Several psychosocial stressors are associated with mental health problems such as pressure related to successful academic results, separation from family, and peer relationship problems. In addition, mental health disorders are linked to university dropout ⁷7. Bantjes J, Saal W, Gericke F, Lochner C, Roos J, Auerbach RP, et al. Mental health and academic failure among first-year university students in South Africa. S Afr J Psychol 2020; 51:396-408., drug use ⁸8. Kohls E, Baldofski S, Moeller R, Klemm S-L, Rummel-Kluge C. Mental health, social and emotional well-being, and perceived burdens of university students during COVID-19 pandemic lockdown in Germany. Front Psychiatry 2021; 12:643957., self-harm ⁹9. Russell K, Allan S, Beattie L, Bohan J, MacMahon K, Rasmussen S. Sleep problem, suicide and self-harm in university students: a systematic review. Sleep Med Rev 2019; 44:58-69., and in more severe cases, suicidal ideation and suicide ¹⁰10. Sheldon E, Simmonds-Buckley M, Bone C, Mascarenhas T, Chan N, Wincott M, et al. Prevalence and risk factors for mental health problems in university undergraduate students: a systematic review with meta-analysis. J Affect Disord 2021; 287:282-92.. Thus, the accurate detection of these disorders and symptoms can facilitate psychotherapeutic interventions, such as psychotherapies and pharmacological interventions, for preventing mental health problems and harmful psychopathological symptoms.

The detection of these symptoms and disorders is supported by psychological testing, which is a part of psychological assessment. Traditionally, psychological testing has been divided into psychometric self-report instruments and projective tests. Psychometric self-report tests measure psychological constructs ¹¹11. Rust J, Golombok S. Modern psychometrics. Abingdon: Routledge; 2018., whereas projective tests use the projection method to estimate psychological characteristics, such as personality and even psychopathological symptoms ¹²12. Lilienfeld SO, Wood JM, Garb HN. The scientific status of projective techniques. Psychol Sci Public Interest 2000; 1:27-66., such as the Rorschach test and the House-Tree-Person (HTP) test. However, both methods show certain limitations. Psychometric self-report instruments have measurement errors, are answered considering social desirability, and may even be time-consuming. Projective tests are frequently criticized for issues related to their scientific validity and reliability ¹²12. Lilienfeld SO, Wood JM, Garb HN. The scientific status of projective techniques. Psychol Sci Public Interest 2000; 1:27-66..

Machine learning algorithms have been established to provide real-time and accurate predictions and diagnoses and expending less time. machine learning is an intelligent system that debugs itself as it receives feedback to improve its predictive and classifying abilities ¹³13. Bi Q, Goodman KE, Kaminsky J, Lessler J. What is machine learning? A primer for the epidemiologist. Am J Epidemiol 2019; 188:2222-39.. machine learning involves the interaction of several fields such as Artificial Intelligence (AI), Computer Science, and Statistics ¹⁴14. Schultebraucks K, Galatzer-Levy IR. Machine learning for prediction of posttraumatic stress and resilience following trauma: an overview of basic concepts and recent advances. J Trauma Stress 2019; 32:215-25.. These predictions and classifications may involve variables with linear and nonlinear relationships, and unusual predictors may be used ¹⁵15. Orrù G, Monaro M, Conversano C, Gemignani A, Sartori G. Machine learning in psychometrics and psychological research. Front Psychol 2020; 10:10..

The increasing use of machine learning in psychological assessments has been observed on different fronts. For example, it has been used for the assessment of psychopathological variables, such as depression, anxiety, and stress ¹⁶16. Priya A, Garg S, Tigga NP. Predicting anxiety, depression and stress in modern life using machine learning algorithms. Procedia Comput Sci 2020; 167:1258-67.^,1717. Kumar P, Garg S, Garg A. Assessment of anxiety, depression and stress using machine learning models. Procedia Comput Sci 2020; 171:1989-98.^,1818. Tigga NP, Garg S. Efficacy of novel attention-based gated recurrent units transformer for depression detection using electroencephalogram signals. Health Inf Sci 2022; 11:1., personality evaluation ¹⁹19. Kosinski M, Stillwell D, Graepel T. Private traits and attributes are predictable from digital records of human behavior. Proc Natl Acad Sci U S A 2013; 110:5802-5., and positive psychological constructs, such as subjective well-being ²⁰20. Zhang N, Liu C, Chen Z, An L, Ren D, Yuan F, et al. Prediction of adolescent subjective well-being: a machine learning approach. Gen Psychiatry 2019; 32:e100096.. Different systematic reviews on the subject have indicated the potential of evaluating psychological constructs and mental disorders via machine learning ²¹21. Chung J, Teo J. Mental health prediction using machine learning: taxonomy, applications, and challenges. Applied Computational Intelligence and Soft Computing 2022; 2022:e9970363.^,2222. Lee Y, Ragguett R-M, Mansur RB, Boutilier JJ, Rosenblat JD, Trevizol A, et al. Applications of machine learning algorithms to predict therapeutic outcomes in depression: a meta-analysis and systematic review. J Affect Disord 2018; 241:519-32.^,2323. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72.^,2424. Thieme A, Belgrave D, Doherty G. Machine learning in mental health. ACM Trans Comput Hum Interact 2020; 27:1-53.^,2525. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16..

Thus, machine learning may be a promising tool for evaluating psychopathological symptoms in undergraduate students. Despite the systematic reviews that focused on machine learning for mental disorders and psychopathological symptoms such as stress, anxiety, and depression among the general population ²¹21. Chung J, Teo J. Mental health prediction using machine learning: taxonomy, applications, and challenges. Applied Computational Intelligence and Soft Computing 2022; 2022:e9970363.^,2222. Lee Y, Ragguett R-M, Mansur RB, Boutilier JJ, Rosenblat JD, Trevizol A, et al. Applications of machine learning algorithms to predict therapeutic outcomes in depression: a meta-analysis and systematic review. J Affect Disord 2018; 241:519-32.^,2323. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72.^,2424. Thieme A, Belgrave D, Doherty G. Machine learning in mental health. ACM Trans Comput Hum Interact 2020; 27:1-53.^,2525. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16., to the best of our knowledge, no review has focused on measuring psychological machine learning constructs among undergraduate students. Therefore, this systematic review aims to evaluate the performance of machine learning models in predicting and detecting depression, anxiety, and stress among undergraduate university students.

Method

This systematic review followed the reporting guidelines established by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for diagnostic test accuracy ²⁶26. McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM; PRISMA-DTA Group. Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies: the PRISMA-DTA statement. JAMA 2018; 319:388-96.. The research protocol was registered on the International Prospective Register of Systematic Reviews (PROSPERO) platform (registration n. CRD42022232335). All studies included in this systematic review were retrieved in January 2024.

Search strategy

The research question was: “How do machine learning models perform in the detection of depression, anxiety, and stress among undergraduate students?”. The search strategy was implemented by creating three strings using the population, intervention, comparison, and outcome (PICO) framework. First, the word “students” and correlates were used for the target population of undergraduate students (1). The expression “machine learning” was used for the intervention (in this study, the diagnostic method) (2). Finally, the descriptors of depression, anxiety, and stress were used for the outcomes (3).

The combination of these descriptors generated the following general search strategy: (depression OR anxiety OR stress OR mental health) AND (machine learning OR artificial intelligence OR supervised learning OR unsupervised learning OR big data OR transfer learning OR machine intelligence) AND (students OR college students OR university students), which was applied to the consulted databases. The full search strategy combined natural language terms with controlled vocabulary terms (e.g., MeSH Terms, APA Thesaurus, and Emtree) from the consulted databases in titles and abstracts sections. The full search strategy for each of the databases is presented in Supplementary Material (Box S1; https://cadernos.ensp.fiocruz.br/static//arquivo/suppl-e00029323_4593.pdf).

Articles were searched in PubMed, Embase, Web of Science, and PsycINFO databases. Titles and abstracts were screened and made available on Rayyan platform (https://www.rayyan.ai/) ²⁷27. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan - a web and mobile app for systematic reviews. Syst Rev 2016; 5:210.. Then, two independent reviewers (B.L.S. and P.Ü.C.) accepted or rejected the articles following the inclusion and exclusion criteria. A third researcher (S.C.C.) analyzed the reports that generated disagreements. This procedure was supervised by two seniors researchers (C.T.R. and A.T.S.) with experience in systematic review methodology.

Eligibility criteria

The inclusion criteria for articles were as follows: (a) published in English; (b) targeted undergraduate university students; (c) empirical study; (d) published in a scientific journal; and (e) predicted anxiety, depression, or stress outcomes via machine learning.

All articles included were read thoroughly. Studies that did not meet the eligibility criteria were excluded from the analysis. Subsequently, the data of interest were extracted via a document in DOC format developed exclusively for this study. The variables evaluated included the authors, country of study, sample characteristics, studies designs, type of data, outcome measure, goals, machine learning algorithms, model’s performance, and data about model’s validation.

Certainty of evidence assessment

The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) was employed for test accuracy studies to assess the certainty of evidence - also called quality of the evidence ²⁸28. Schünemann HJ, Mustafa RA, Brozek J, Steingart KR, Leeflang M, Murad MH, et al. GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy. J Clin Epidemiol 2020; 122:129-41.^,2929. Schünemann HJ, Mustafa RA, Brozek J, Steingart KR, Leeflang M, Murad MH, et al. GRADE guidelines: 21 part 2. Test accuracy: inconsistency, imprecision, publication bias, and other domains for rating the certainty of evidence and presenting it in evidence profiles and summary of findings tables. J Clin Epidemiol 2020; 122:142-52.. GRADE assesses the certainty of evidence based on five domains: risk of bias, indirectness, inconsistency, imprecision, and publication bias. GRADE provides a judgment on the certainty of evidence, classifying it as very low, low, moderate, or high. The general evidence assessment considers the “high” classification as baseline, decreasing depending on the judgment of each of the five domains.

To ensure a homogeneous assessment of the certainty of evidence, the studies were categorized based on the performance metrics they reported. Initially, the quality of evidence was evaluated in the 33 studies that provided accuracy data. For those studies that did not report accuracy specifically, sensitivity or specificity scores were considered (5 studies). When neither accuracy nor sensitivity and specificity were available, the evidence was grouped by the area under the curve (AUC) (3 studies) and positive predictive value (PPV) (2 studies). Finally, all remaining studies that did not report any of the aforementioned metrics were integrated (5 studies).

Quality of machine learning models

To assess the quality of the included articles, the instrument proposed by Ramos-Lima et al. ²³23. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72. was employed after receiving formal authorization. The tool was built to evaluate the quality of machine learning studies, given the lack of applications within this scope, and is under validation. The instrument was used to evaluate nine criteria: (1) sample representativeness (if the study represents target population heterogeneity), (2) control of the confounding variables (if the study controls for potential confounding variables), (3) assessment of the outcome (how the outcome variable was assessed), (4) use of an machine learning technique (if an machine learning technique was mentioned and employed), (5) presentation of performance statistics (if the performance was reported), (6) management of missing data (how missing data were managed), (7) test unseen (separation of data between test and validation), (8) class imbalance (if the authors address the balance of cases), and (9) feature selection (if the authors address feature selection in the dataset).

Data analysis

The data was organized and presented via a narrative synthesis of the main results. Due to the wide heterogeneity of the studies, it was not possible to perform a meta-analysis.

Results

Selection of relevant articles

After applying the search strategy, 2,304 potential studies, dating from 1988 to 2024, were retrieved from the databases. Of these studies, 412 were from PubMed, 1,071 from Web of Science, 569 from Embase, and 252 from PsycINFO. In total, 85 articles were selected after screening and reading. From these, 48 articles met the inclusion criteria. Figure 1 illustrates the process of selection and exclusion of studies. The list of 37 articles excluded with reasons after full reading is presented in Supplementary Material (Box S2; https://cadernos.ensp.fiocruz.br/static//arquivo/suppl-e00029323_4593.pdf).

General characteristics of the selected studies

Box 1 outlines the main features of the studies, including the country where it was conducted, the sample size and characteristics, and the study design (e.g., cross-sectional or longitudinal). Most studies were conducted in China (n = 10; 20.83%), European countries (n = 11; 22.91%), or the United States (n = 8; 16.67%). The sample sizes ranged from 24 to 4,184 participants, with ages typically ranging from 17 to 67 years, and a predominant female majority. In total, 36 studies (75%) employed a cross-sectional design.

Machine learning models and performance

Box 2 presents machine learning models organized according to their employed data types. They were grouped into eight main categories: physiological data, behavioral data, neurocerebral data, blood markers, internet usage data, mixed data, mobility data, and demographic data. For each of these models, the illustration presents their primary goals, machine learning algorithms employed, performance parameters reported, methodology for evaluating the outcomes, and whether the model underwent validation. These data are summarized as follows.

Models employing physiological data

This subsection encompasses seven distinct machine learning models exclusively employing physiological data. These data encompass parameters such as breathing, skin conductance, skin temperature, blood pressure, heart rate, and another physiological signal derived from electrocardiograms, electromyograms, and electroencephalograms (EEG).

Amalraj et al. ³⁰30. Amalraj JDI, Bojan VK, Murugasamy K. Detection of stress level based on sweat from Gen-Z students using ANN and GA algorithms. Int J Clin Exp Med 2023; 16:260-74. used physiological data such as body temperature, skin conductance, sweat rate, sweat pH, and acceleration to evaluate different levels of stress among university students (high stress, medium stress, and low stress). The Artificial Neural Network (ANN) with a genetic algorithm achieved a 99% accuracy rate in detecting stress levels.

Jiao et al. ³¹31. Jiao Y, Wang X, Liu C, Du G, Zhao L, Dong H, et al. Feasibility study for detection of mental stress and depression using pulse rate variability metrics via various durations. Biomed Signal Process Control 2023; 79:104145. employed pulse rate variability metrics to detect depression and stress among university students. They achieved a 95.26% accuracy in detecting depression and 98.46% in detecting stress.

Pal et al. ³²32. Pal R, Adhikari D, Heyat MBB, Guragai B, Lipari V, Brito Ballester J, et al. A novel smart belt for anxiety detection, classification, and reduction using IIoMT on students' cardiac signal and MSY. Bioengineering 2022; 9:793. aimed to classify students with and without anxiety considering information from cardiac signals. Their Random Forest (RF) algorithm achieved an accuracy of 80%.

Pourmohammadi & Maleki ³³33. Pourmohammadi S, Maleki A. Stress detection using ECG and EMG signals: a comprehensive study. Comput Methods Programs Biomed 2020; 193:105482. aimed to classify stress levels in university students by combining physiological signals from electrocardiograms and electromyograms. Stress was induced in the laboratory via experiments such as the Stroop color and word test and mental arithmetic. The study employed a Support Vector Machine (SVM) algorithm, achieving a stress classification 100% accuracy for two levels, 97.6% for three levels, and 96.2% for four levels.

Sharma et al. ³⁴34. Sharma V, Prakash NR, Kalra P. Depression status identification using autoencoder neural network. Biomed Signal Process Control 2022; 75:103568. used electrodermal data such as skin conductance to identify students with and without depression following an experiment involving sound stimuli to evoke emotions. They achieved an accuracy of 95.2% using the Autoencoder Neural Network.

Silva et al. ³⁵35. Silva E, Aguiar J, Reis LP, Sá JO, Gonçalves J, Carvalho V. Stress among Portuguese medical students: the EuStress Solution. J Med Syst 2020; 44:45. sought to predict stress in university students based on heart rate and heart rate variability data. The Neural Network (NN) algorithm exhibited the best performance, with a specificity of 74.2% and a sensitivity of 78.1%.

Tiwari & Agarwal ³⁶36. Tiwari S, Agarwal S. A shrewd artificial neural network-based hybrid model for pervasive stress detection of students using galvanic skin response and electrocardiogram signals. Big Data 2021; 9:427-42. developed an machine learning model to assess four distinct mental states: relaxation, stress, partial stress, and happiness. Data sources included parameters such as skin conductivity, heart rate, and blood pressure. These mental states were induced via experimental tasks in the laboratory. The ANN algorithm demonstrated a 99.4% accuracy in detecting these mental states.

Models employing behavioral data

This subsection encompasses 13 machine learning models constructed from behavioral data obtained via self-report instruments. These models were developed using various data sources, including psychopathological symptoms (e.g., anxiety, paranoia, and anger), personality traits, cognitive beliefs, daily activities, and self-concept information.

Anand et al. ³⁷37. Anand RV, Md AQ, Urooj S, Mohan S, Alawad MA. Enhancing diagnostic decision-making: ensemble learning techniques for reliable stress level classification. Diagnostics 2023; 13:3455. assessed various levels of stress (high stress, manageable stress, and no stress) based on students’ behavioral habits during graduation, including sleep duration, productive time, and completion of academic tasks. They employed a combination of Decision Trees (DT), RF, and AdaBoost algorithms, achieving a 93.48% accuracy.

Balli et al. ³⁸38. Balli M, Okan A, Gürsan NÖ, Gülgöz S, Eser HY. Comparison of machine learning algorithms for Beck Depression Inventory measured depression status classification. Eur Psychiatry 2023; 66 Suppl 1:S419. developed an algorithm to detect individuals with depression and without depression based on psychopathological symptoms, including variables such as anxiety, stress, and childhood trauma. A 89.6% accuracy was attained using an XGBoost algorithm.

Daza et al. ³⁹39. Daza A, Bobadilla J, Apaza O, Pinto J. Stacking ensemble learning model for predict anxiety level in university students using balancing methods. Inform Med Unlocked 2023; 42:101340. developed a model based on anxiety symptoms to predict different levels of anxiety (no anxiety, mild, moderate, or severe). The K-Nearest Neighbors (KNN) algorithm demonstrated a 97.83% accuracy.

Estabragh et al. ⁴⁰40. Estabragh ZS, Kashani MM, Moghaddam F, Sari S, Taherifar Z, Moradi Moosavy S, et al. Bayesian network modeling for diagnosis of social anxiety using some cognitive-behavioral factors. Netw Model Anal Health Inform Bioinform 2013; 2:257-65. developed an algorithm for assessing social anxiety based on cognitive and behavioral factors, including self-efficacy, attachment patterns, behavioral inhibition, and shyness. The Bayesian Network (BN) algorithm demonstrated an AUC of 89.8%.

Herbert et al. ⁴¹41. Herbert C, El Bolock A, Abdennadher S. How do you feel during the COVID-19 pandemic? A survey using psychological and linguistic self-report measures, and machine learning to investigate mental health, subjective experience, personality, and behaviour during the COVID-19 pandemic among university students. BMC Psychology 2021; 9:90. evaluated university students’ trait anxiety, measured by the State-Trait Anxiety Inventory (STAI), at the outset of the COVID-19 pandemic. They integrated a range of psychological data, encompassing personality traits, mental health indicators, self-concept information, and health beliefs. The Support Vector Regression (SVR) algorithm yielded an root mean square error (RMSE) of 0.90 with 15.4% variation.

Ge et al. ⁴²42. Ge F, Zhang D, Wu L, Mu H. Predicting psychological state among Chinese undergraduate students in the COVID-19 epidemic: a longitudinal study using a machine learning. Neuropsychiatr Dis Treat 2020; 16:2111-8. developed a machine learning model for predicting anxiety in university students. The model was constructed using mental health data, including variables related to suicidal ideation, relationship issues, anxiety levels, and sleeping difficulties. The XGBoost algorithm demonstrated a 97.3% accuracy in predicting anxiety, with a 97.3% sensitivity and a 96.3% specificity.

Gil et al. ⁴³43. Gil M, Kim SS, Min EJ. Machine learning models for predicting risk of depression in Korean college students: identifying family and individual factors. Front Public Health 2022; 10:1023010. aimed to predict the risk of depression among university students using family and individual behavioral data, including family adaptation and cohesion, family bonds, marital satisfaction, personality, health habits, among others. The RF algorithm achieved a 86.27% accuracy.

Maitre et al. ⁴⁴44. Maitre J, Bergeron-Leclerc C, Maltais D, Gaboury S. Investigating anxiety levels in the Quebec university community during the COVID-19 pandemic using machine learning and data exploration techniques. Multimed Tools Appl 2023; 82:46109-27. explored anxiety level among university students using behavioral data. The CatBoost algorithm yielded an R² value of 0.56.

Morales-Rodríguez et al. ⁴⁵45. Morales-Rodríguez FM, Martínez-Ramón JP, Méndez I, Ruiz-Esteban C. Stress, coping, and resilience before and after COVID-19: a predictive model based on artificial intelligence in the university environment. Front Psychol 2021; 12:647964. predicted stress levels using information on the resilience and coping strategies of university students. The ANN algorithm achieved an AUC of 74.8%.

Ren et al. ⁴⁶46. Ren Z, Xin Y, Ge J, Liu D, Ho CS. Psychological impact of COVID-19 on college students after school reopening: a cross-sectional study based on machine learning. Front Psychol 2021; 12:641806. aimed to assess the anxiety and depression levels of students during the COVID-19 pandemic using behavioral factors associated with the disease, such as mask-wearing, quarantine status, presence of infected friends, and frequent fever measurements. The RF algorithm achieved a 73.5% accuracy for depression and 81.42% for anxiety.

Upadhyay et al. ⁴⁷47. Upadhyay DK, Mohapatra S, Singh NK. An early assessment of Persistent Depression Disorder using machine learning algorithm. Multimed Tools Appl 2023; 83:49149-71. developed a model based on behavioral data to detect persistent depression disorder among university students. The SVM algorithm achieved an accuracy of 89.4%.

Vergaray et al. ⁴⁸48. Vergaray A, Miranda JCH, Cornelio JB, Carranza ARL, Sánchez CFP. Predicting the depression in university students using stacking ensemble techniques over oversampling method. Inform Med Unlocked 2023; 41:101295. used symptoms of depression to identify students with depression. The SVM algorithm demonstrated a 94.69% accuracy.

Wang et al. ⁴⁹49. Wang C, Zhao H, Zhang H. Chinese college students have higher anxiety in new semester of online learning during COVID-19: a machine learning approach. Front Psychol 2020; 11:587413. aimed to assess anxiety levels among university students, measured by the Self-Rating Anxiety Scale (SAS), both at the beginning of the academic semester and one month after the commencement of the academic semester, which coincided with the onset of the COVID-19 lockdown. The most effective machine learning model consisted of 20 SAS items and used an XGBoost algorithm, which achieved a 82.1% accuracy in predicting anxiety and a 84.38% accuracy in predicting changes in anxiety levels.

Models employing neurocerebral data

This subsection encompasses five machine learning models that employed neurocerebral data, including neuroimaging data revealing brain regions activated during specific activities, such as the prefrontal cortex, amygdala, and temporal lobe. AlShorman et al. ⁵⁰50. AlShorman O, Masadeh M, Heyat MBB, Akhtar F, Almahasneh H, Ashraf GM, et al. Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection. J Integr Neurosci 2022; 21:20. introduced a model for stress classification among university students employing brain EEG signals. Their SVM model with radial basis function (RBF) kernel demonstrated an 81.4% accuracy in stress detection.

He et al. ⁵¹51. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40. developed a machine learning model to assess depression and anxiety in university students. The model employed neuroimages derived from the connectome. The Bayesian logistic regression (BLR) machine learning model achieved a 68.72% accuracy in distinguishing anxious university students from healthy controls and 53.68% accuracy in distinguishing anxiety from depression.

Li et al. ⁵²52. Li X, Hu B, Shen J, Xu T, Retcliffe M. Mild depression detection of college students: an EEG-based solution with free viewing tasks. J Med Syst 2015; 39:187. employed data from the EEG during a free viewing task to differentiate between students with depression and those without. Their KNN algorithm demonstrated a 99.1% accuracy in correctly classifying individuals with depression.

Modinos et al. ⁵³53. Modinos G, Mechelli A, Pettersson-Yeo W, Allen P, McGuire P, Aleman A. Pattern classification of brain activation during emotional processing in subclinical depression: psychosis proneness as potential confounding factor. PeerJ 2013; 1:e42. also constructed a machine learning model using neuroimaging data, with the objective of accurately classifying students with and without depression. The SVM algorithm showed a 77% accuracy in classifying depression, along with a 71% sensitivity and 82% specificity.

Zhang et al. ⁵⁴54. Zhang W, Shen Q, Song J, Zhou R. Classification of test-anxious individuals using Event-Related Potentials (ERPs): the effectiveness of machine learning algorithms. Acta Psychologica Sinica 2019; 51:1116-27. aimed to accurately identify students with and without anxiety using EEG data acquired during an emotional Stroop test. They achieved an 86.5% accuracy using a Convolutional Neural Network (CNN).

Models employing blood markers

This subsection discusses two machine learning models that employ data associated with blood markers, including indicators of blood stasis (poor blood circulation or blockage of blood flow in the body) and biomarkers, such as the chromatin of neutrophils in peripheral blood.

Liu et al. ⁵⁵55. Liu M, Xu Y, Wu H, Wang X, Ye B. Blood stasis constitution and depression among Chinese female college students: a longitudinal moderation model. Int J Ment Health Addict 2023; 21:929-43. developed a model based on the constitution of blood stasis to predict depression in female university students, measured by the Center for Epidemiologic Studies Depression (CES-D) scale, over a 1-year period. The SVM algorithm was employed. The constitution of blood stasis successfully predicted depression over the course of one year (r = 0.81; p < 0.01).

Topalovic et al. ⁵⁶56. Topalovic N, Mazic S, Nesic D, Vukovic O, Cumic J, Laketic D, et al. Association between chromatin structural organization of peripheral blood neutrophils and self-perceived mental stress: gray-level co-occurrence matrix analysis. Microsc Microanal 2021; 27:1202-8. constructed a model based on the organization of peripheral blood neutrophils to forecast an increase in stress among university students. The BLR algorithm achieved a 70% accuracy.

Models employing internet usage data

This subsection covers five machine learning models that were constructed using data sourced from the internet. Examples of these data sources include patterns of social network usage (text interactions and engagement with other users) and browsing activities on web browsers.

Ding et al. ⁵⁷57. Ding Y, Chen X, Fu Q, Zhong S. A depression recognition method for college students using deep integrated support vector algorithm. IEEE Access 2020; 8:75616-29. developed a machine learning model for classifying depression among university students based on user interaction data from a Chinese social network called Sina Weibo (https://weibo.com). This data included elements such as the words used, likes, and emojis. The Deep Integrated Support Vector Machine (DISVM) algorithm showed the best performance, achieving an 86% accuracy in classifying students with depression.

Dehghan-Bonari et al. ⁵⁸58. Dehghan-Bonari M, Alipour-Vaezi M, Nasiri MM, Aghsami A. A diagnostic analytics model for managing post-disaster symptoms of depression and anxiety among students using a novel data-driven optimization approach. Healthc Anal 2023; 4:100238. employed sentiment analysis of texts and interactions on social networks to classify students with severe, moderate, and mild depression. The RF algorithm achieved a 94% accuracy.

Siraji et al. ⁵⁹59. Siraji MI, Rahman AA, Nishat MM, Al Mamun MA, Faisal F, Khalid LI. Impact of mobile connectivity on students' wellbeing: detecting learners' depression using machine learning algorithms. PLoS One 2023; 18:e0294803. aimed to evaluate students with depression using internet connectivity data. The SVM algorithm demonstrated an 85.14% accuracy.

Zhang et al. ⁶⁰60. Zhang B, Zaman A, Silenzio V, Kautz H, Hoque E. The relationships of deteriorating depression and anxiety with longitudinal behavioral changes in Google and YouTube use during COVID-19: observational study. JMIR Ment Health 2020; 7:e24012. constructed a machine learning model to assess the exacerbation of depression and anxiety in university students during the COVID-19 social isolation. This model was based on search data from Google Search (https://www.google.com/) and YouTube (https://www.youtube.com/) and used an ordinary least square (OLS) algorithm. Temporal aspects of platform usage, including search times, proved to be the most effective predictors of the exacerbation of depression (mean squared error - MSE = 2.37; R² = 0.84) and anxiety (MSE = 2.48; R² = 0.81).

Ware et al. ⁶¹61. Ware S, Yue C, Morillo R, Lu J, Shang C, Bi J, et al. Predicting depressive symptoms using smartphone data. Smart Health (Amst) 2020; 15:100093. developed two machine learning models to assess different depression symptoms, including physical, affective, and cognitive aspects. The models used smartphone usage data, with one based on a local app (Model 1) and the other on data obtained via the wireless network (Model 2). Both models were evaluated using an SVM algorithm with an RBF kernel. Model 1 achieved 67% accuracy in identifying lethargy, whereas Model 2 achieved 72% accuracy in identifying sleep problems.

Models employing mixed data

In this subsection, we encompass 13 machine learning models constructed using mixed data. Here, models employed some of the previously mentioned data types, such as physiological, psychological, and internet usage patterns, but in conjunction with data not previously discussed, including smartphone activity, geolocation, mobility, among others.

Aalbers et al. ⁶²62. Aalbers G, Hendrickson AT, Vanden Abeele MM, Keijsers L. Smartphone-tracked digital markers of momentary subjective stress in college students: idiographic machine learning analysis. JMIR Mhealth Uhealth 2023; 11:e37469. developed a model based on digital markers such as smartphone login data, messages, and sleep inferences to assess stress among students. The RF algorithm yielded a mean absolute error (MAE) of 0.84.

Acikmese & Alptekin ⁶³63. Acikmese Y, Alptekin SE. Prediction of stress levels with LSTM and passive mobile sensors. Procedia Comput Sci 2019; 159:658-67. employed a machine learning model to classify stress levels in university students, which were assessed via qualitative feedback (indicating whether or not they were feeling stressed). The model primarily relied on smartphone usage data, including light sensor data, audio usage, call conversations, and wi-fi data, as well as geolocation and physical activity. The Long Short-Term Memory (LSTM) algorithm achieved a 63% accuracy in detecting stressed university students.

Ahmed & Ahmed ⁶⁴64. Ahmed MS, Ahmed N. A fast and minimal system to identify depression using smartphones: explainable machine learning-based approach. JMIR Form Res 2023; 7:e28848. assessed students with and without depression using digital marks captured by an app on their smartphones. The BFS algorithm was 78% accurate in identifying students with and without depression.

Chikersal et al. ⁶⁵65. Chikersal P, Doryab A, Tumminia M, Villalba DK, Dutcher JM, Liu X, et al. Detecting depression and predicting its onset using longitudinal symptoms captured by passive sensing. ACM Trans Comput Hum Interact 2021; 28:3. developed a model that incorporated geolocation and movement data, as well as smartphone usage patterns, conversations, audio inferences, and contacts. The model aimed to classify students with depression at the end of the academic semester, as well as to predict the worsening of these symptoms. The AdaBoost algorithm successfully identified 85.7% of students with depression at the end of the semester and 88.1% of those with worsening depression symptoms.

Guerrero et al. ⁶⁶66. Guerrero G, Avila D, da Silva FJM, Pereira A, Fernández-Caballero A. Internet-based identification of anxiety in university students using text and facial emotion analysis. Internet Interv 2023; 34:100679. constructed two models to identify students with anxiety: one based on facial expressions (Model 1) and another based on emotional expressions in Facebook (https://www.facebook.com/) posts (Model 2). Model 1 achieved a PPV of 86.84%, whereas Model 2 achieved a PPV of 84.21%.

Mahalingam et al. ⁶⁷67. Mahalingam M, Jammal M, Hoteit R, Ayna D, Romani M, Hijazi S, et al. A machine learning study to predict anxiety on campuses in Lebanon. Stud Health Technol Inform 2023; 29:85-8. constructed a model employing demographic information including gender, income, and age, as well as health habits such as diet, sleep, and alcohol and cigarette use. The SVM algorithm demonstrated an accuracy of 69.7% in identifying students with anxiety.

Meda et al. ⁶⁸68. Meda N, Pardini S, Rigobello P, Visioli F, Novara C. Frequency and machine learning predictors of severe depressive symptoms and suicidal ideation among university students. Epidemiol Psychiatr Sci 2023; 7:e42. employed demographic and behavioral data, including income, location, diet, and suicidal ideation, to predict the worsening of depression among university students over six months. The RF algorithm exhibited a PPV of 77%.

Nemesure et al. ⁶⁹69. Nemesure MD, Heinz MV, Huang R, Jacobson NC. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep 2021; 11:1980. developed a machine learning model using physiological data (such as blood pressure and heart rate), body data (height and weight), psychological data (life satisfaction), and health habits (smoking, diet, physical activity) to classify major depressive disorder and generalized anxiety disorder among university students. The XGBoost algorithm achieved an AUC of 73% in the classification of generalized anxiety disorder and an AUC of 67% in the classification of major depressive disorder. Bhadra & Kumar ⁷⁰70. Bhadra S, Kumar CJ. Enhancing the efficacy of depression detection system using optimal feature selection from EHR. Comput Methods Biomech Biomed Engin 2024; 27:222-36. reanalyzed the same dataset and achieved an 88.46% accuracy in detecting depression using a RF algorithm.

Rois et al. ⁷¹71. Rois R, Ray M, Rahman A, Roy SK. Prevalence and predicting factors of perceived stress among Bangladeshi university students using machine learning algorithms. J Health Popul Nutr 2021; 40:50. constructed a machine learning model that integrated physiological metrics, including blood pressure and pulse rate, along with health-related habits data such as body mass index, sleep patterns, and physical activity, for the purpose of categorizing stress levels among university students. The results were assessed based on qualitative feedback from the participants, in which they indicated whether they felt stressed or not. The RF algorithm exhibited an 89% accuracy in stress identification.

Sano et al. ⁷²72. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res 2018; 20:e9410. developed an machine learning model to assess stress in university students. The model was composed of different types of data, such as physiological data (skin conductance and temperature), geolocation data, mobility, cell phone usage patterns (including calls and messages), and social network usage, among others. The SVM with RBF kernel algorithm demonstrated an 81.5% accuracy in classifying university students with high stress and low stress over a period of one month.

Ware et al. ⁷³73. Ware S, Yue C, Morillo R, Shang C, Bi J, Kamath J, et al. Automatic depression screening using social interaction data on smartphones. Smart Health (Amst) 2022; 26:100356. employed social interaction data from smartphones, including messages and calls, to distinguish between students with and without depression. The XGBoost algorithm achieved an F1 score of 82%.

Xu et al. ⁷⁴74. Xu X, Chikersal P, Dutcher JM, Sefidgar YS, Seo W, Tumminia MJ, et al. Leveraging collaborative-filtering for personalized behavior modeling: a case study of depression detection among college students. Proc ACM Interact Mob Wearable Ubiquitous Technol 2021; 5:41. developed a model that incorporated information extracted from cell phone use, such as calls and location data, as well as step and sleep data obtained via a wearable sensor, to detect students with and without depression throughout the academic semester. The developed algorithm demonstrated a 79.1% accuracy.

Yue et al. ⁷⁵75. Yue C, Ware S, Morillo R, Lu J, Shang C, Bi J, et al. Fusing location data for depression prediction. IEEE Trans Big Data 2021; 7:355-70. developed a model integrating geographic location data and wi-fi access information from smartphones to detect university students with depression. The SVM with RBF kernel algorithm achieved an F1 score of 79%.

Models employing mobility data

Only one study used only mobility data. Müller et al. ⁷⁶76. Müller SR, Chen XL, Peters H, Chaintreau A, Matz SC. Depression predictions from GPS-based mobility do not generalize well to large demographically heterogeneous samples. Sci Rep 2021; 11:14007. classified students with and without depression based on GPS mobility data. The RF algorithm presented an AUC of 82%.

Models employing demographic data

In a single study focusing on demographic data, Nayan et al. ⁷⁷77. Nayan MIH, Uddin MSG, Hossain MI, Alam MM, Zinnia MA, Haq I, et al. Comparison of the performance of machine learning-based algorithms for predicting depression and anxiety among university students in Bangladesh: a result of the first wave of the COVID-19 pandemic. Asian J Soc Health Behav 2022; 5:75-84. aimed to identify students with and without depression, as well as those with and without anxiety by employing variables such as gender, education, professional occupation, and years of study. Their KNN algorithm achieved an accuracy of 88.28% in detecting depression, whereas the RF algorithm demonstrated an accuracy of 91.49% in detecting anxiety.

Certainty of evidence of the selected studies

The GRADE assessment revealed very low quality of evidence in all studies. Serious risks of bias were found, mainly due to issues in the assessment of outcomes. Furthermore, the indirectness dimension was also scored as serious, given that few studies employed the assessment of clinical professionals in diagnosing outcomes. Additionally, the imprecision dimension was also classified as “serious” since most datasets do not seem to adequately represent the college students population. On the other hand, the inconsistency was considered “not serious,” as the variability of performance scores and instruments used reflect particular characteristics of the studies such as the type of sample, being was already expected ²⁹29. Schünemann HJ, Mustafa RA, Brozek J, Steingart KR, Leeflang M, Murad MH, et al. GRADE guidelines: 21 part 2. Test accuracy: inconsistency, imprecision, publication bias, and other domains for rating the certainty of evidence and presenting it in evidence profiles and summary of findings tables. J Clin Epidemiol 2020; 122:142-52.. Finally, no publication bias was identified. Box 3 presents this information in detail.

Quality assessment of machine learning models

Box 4 summarizes the quality assessment data. The included articles presented adequate methodological attributes and limitations of the evaluated items. In total, 29 of the 48 (60.41%) articles showed consistent data of sample representativeness, but only four indicated the control of confounding variables (8.33%). All studies used machine learning algorithms and included model performance data (100%). A total of 45 studies consistently reported the assessment of outcomes (93.75%). Moreover, 18 articles addressed the handling of missing data (37.5%).

Regarding the specific characteristics of machine learning models, 44 studies specified the sample split between testing and validation (91.66%). In total, nine articles addressed the resolution of the class imbalance issue (18.75%). Finally, 21 studies commented on feature selection from the dataset (43.75%).

Discussion

The current systematic review aims to assess the performance of various machine learning models in predicting and detecting depression, anxiety, and stress in college students. A diverse range of models were examined among the 48 studies, including physiological, behavioral, internet usage, neurocerebral, blood markers, mixed, mobility, and demographic data. Overall, these machine learning models demonstrated satisfactory performance in predicting and classifying the intended outcomes.

Out of all the studies assessed, 33 of them ³⁰30. Amalraj JDI, Bojan VK, Murugasamy K. Detection of stress level based on sweat from Gen-Z students using ANN and GA algorithms. Int J Clin Exp Med 2023; 16:260-74.^,3131. Jiao Y, Wang X, Liu C, Du G, Zhao L, Dong H, et al. Feasibility study for detection of mental stress and depression using pulse rate variability metrics via various durations. Biomed Signal Process Control 2023; 79:104145.^,3232. Pal R, Adhikari D, Heyat MBB, Guragai B, Lipari V, Brito Ballester J, et al. A novel smart belt for anxiety detection, classification, and reduction using IIoMT on students' cardiac signal and MSY. Bioengineering 2022; 9:793.^,3333. Pourmohammadi S, Maleki A. Stress detection using ECG and EMG signals: a comprehensive study. Comput Methods Programs Biomed 2020; 193:105482.^,3434. Sharma V, Prakash NR, Kalra P. Depression status identification using autoencoder neural network. Biomed Signal Process Control 2022; 75:103568.^,3636. Tiwari S, Agarwal S. A shrewd artificial neural network-based hybrid model for pervasive stress detection of students using galvanic skin response and electrocardiogram signals. Big Data 2021; 9:427-42.^,3737. Anand RV, Md AQ, Urooj S, Mohan S, Alawad MA. Enhancing diagnostic decision-making: ensemble learning techniques for reliable stress level classification. Diagnostics 2023; 13:3455.^,3838. Balli M, Okan A, Gürsan NÖ, Gülgöz S, Eser HY. Comparison of machine learning algorithms for Beck Depression Inventory measured depression status classification. Eur Psychiatry 2023; 66 Suppl 1:S419.^,3939. Daza A, Bobadilla J, Apaza O, Pinto J. Stacking ensemble learning model for predict anxiety level in university students using balancing methods. Inform Med Unlocked 2023; 42:101340.^,4242. Ge F, Zhang D, Wu L, Mu H. Predicting psychological state among Chinese undergraduate students in the COVID-19 epidemic: a longitudinal study using a machine learning. Neuropsychiatr Dis Treat 2020; 16:2111-8.^,4343. Gil M, Kim SS, Min EJ. Machine learning models for predicting risk of depression in Korean college students: identifying family and individual factors. Front Public Health 2022; 10:1023010.^,4646. Ren Z, Xin Y, Ge J, Liu D, Ho CS. Psychological impact of COVID-19 on college students after school reopening: a cross-sectional study based on machine learning. Front Psychol 2021; 12:641806.^,4747. Upadhyay DK, Mohapatra S, Singh NK. An early assessment of Persistent Depression Disorder using machine learning algorithm. Multimed Tools Appl 2023; 83:49149-71.^,4848. Vergaray A, Miranda JCH, Cornelio JB, Carranza ARL, Sánchez CFP. Predicting the depression in university students using stacking ensemble techniques over oversampling method. Inform Med Unlocked 2023; 41:101295.^,4949. Wang C, Zhao H, Zhang H. Chinese college students have higher anxiety in new semester of online learning during COVID-19: a machine learning approach. Front Psychol 2020; 11:587413.^,5050. AlShorman O, Masadeh M, Heyat MBB, Akhtar F, Almahasneh H, Ashraf GM, et al. Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection. J Integr Neurosci 2022; 21:20.^,5151. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40.^,5252. Li X, Hu B, Shen J, Xu T, Retcliffe M. Mild depression detection of college students: an EEG-based solution with free viewing tasks. J Med Syst 2015; 39:187.^,5353. Modinos G, Mechelli A, Pettersson-Yeo W, Allen P, McGuire P, Aleman A. Pattern classification of brain activation during emotional processing in subclinical depression: psychosis proneness as potential confounding factor. PeerJ 2013; 1:e42.^,5454. Zhang W, Shen Q, Song J, Zhou R. Classification of test-anxious individuals using Event-Related Potentials (ERPs): the effectiveness of machine learning algorithms. Acta Psychologica Sinica 2019; 51:1116-27.^,5656. Topalovic N, Mazic S, Nesic D, Vukovic O, Cumic J, Laketic D, et al. Association between chromatin structural organization of peripheral blood neutrophils and self-perceived mental stress: gray-level co-occurrence matrix analysis. Microsc Microanal 2021; 27:1202-8.^,5757. Ding Y, Chen X, Fu Q, Zhong S. A depression recognition method for college students using deep integrated support vector algorithm. IEEE Access 2020; 8:75616-29.^,5858. Dehghan-Bonari M, Alipour-Vaezi M, Nasiri MM, Aghsami A. A diagnostic analytics model for managing post-disaster symptoms of depression and anxiety among students using a novel data-driven optimization approach. Healthc Anal 2023; 4:100238.^,5959. Siraji MI, Rahman AA, Nishat MM, Al Mamun MA, Faisal F, Khalid LI. Impact of mobile connectivity on students' wellbeing: detecting learners' depression using machine learning algorithms. PLoS One 2023; 18:e0294803.^,6363. Acikmese Y, Alptekin SE. Prediction of stress levels with LSTM and passive mobile sensors. Procedia Comput Sci 2019; 159:658-67.^,6464. Ahmed MS, Ahmed N. A fast and minimal system to identify depression using smartphones: explainable machine learning-based approach. JMIR Form Res 2023; 7:e28848.^,6565. Chikersal P, Doryab A, Tumminia M, Villalba DK, Dutcher JM, Liu X, et al. Detecting depression and predicting its onset using longitudinal symptoms captured by passive sensing. ACM Trans Comput Hum Interact 2021; 28:3.^,6767. Mahalingam M, Jammal M, Hoteit R, Ayna D, Romani M, Hijazi S, et al. A machine learning study to predict anxiety on campuses in Lebanon. Stud Health Technol Inform 2023; 29:85-8.^,7070. Bhadra S, Kumar CJ. Enhancing the efficacy of depression detection system using optimal feature selection from EHR. Comput Methods Biomech Biomed Engin 2024; 27:222-36.^,7171. Rois R, Ray M, Rahman A, Roy SK. Prevalence and predicting factors of perceived stress among Bangladeshi university students using machine learning algorithms. J Health Popul Nutr 2021; 40:50.^,7272. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res 2018; 20:e9410.^,7474. Xu X, Chikersal P, Dutcher JM, Sefidgar YS, Seo W, Tumminia MJ, et al. Leveraging collaborative-filtering for personalized behavior modeling: a case study of depression detection among college students. Proc ACM Interact Mob Wearable Ubiquitous Technol 2021; 5:41.^,7777. Nayan MIH, Uddin MSG, Hossain MI, Alam MM, Zinnia MA, Haq I, et al. Comparison of the performance of machine learning-based algorithms for predicting depression and anxiety among university students in Bangladesh: a result of the first wave of the COVID-19 pandemic. Asian J Soc Health Behav 2022; 5:75-84. reported at least one accuracy score, whereas ten studies ³⁵35. Silva E, Aguiar J, Reis LP, Sá JO, Gonçalves J, Carvalho V. Stress among Portuguese medical students: the EuStress Solution. J Med Syst 2020; 44:45.^,4040. Estabragh ZS, Kashani MM, Moghaddam F, Sari S, Taherifar Z, Moradi Moosavy S, et al. Bayesian network modeling for diagnosis of social anxiety using some cognitive-behavioral factors. Netw Model Anal Health Inform Bioinform 2013; 2:257-65.^,4545. Morales-Rodríguez FM, Martínez-Ramón JP, Méndez I, Ruiz-Esteban C. Stress, coping, and resilience before and after COVID-19: a predictive model based on artificial intelligence in the university environment. Front Psychol 2021; 12:647964.^,6161. Ware S, Yue C, Morillo R, Lu J, Shang C, Bi J, et al. Predicting depressive symptoms using smartphone data. Smart Health (Amst) 2020; 15:100093.^,6666. Guerrero G, Avila D, da Silva FJM, Pereira A, Fernández-Caballero A. Internet-based identification of anxiety in university students using text and facial emotion analysis. Internet Interv 2023; 34:100679.^,6868. Meda N, Pardini S, Rigobello P, Visioli F, Novara C. Frequency and machine learning predictors of severe depressive symptoms and suicidal ideation among university students. Epidemiol Psychiatr Sci 2023; 7:e42.^,6969. Nemesure MD, Heinz MV, Huang R, Jacobson NC. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep 2021; 11:1980.^,7373. Ware S, Yue C, Morillo R, Shang C, Bi J, Kamath J, et al. Automatic depression screening using social interaction data on smartphones. Smart Health (Amst) 2022; 26:100356.^,7575. Yue C, Ware S, Morillo R, Lu J, Shang C, Bi J, et al. Fusing location data for depression prediction. IEEE Trans Big Data 2021; 7:355-70.^,7676. Müller SR, Chen XL, Peters H, Chaintreau A, Matz SC. Depression predictions from GPS-based mobility do not generalize well to large demographically heterogeneous samples. Sci Rep 2021; 11:14007. relied solely on any metrics among F1, AUC, PPV, sensitivity, and specificity and five ⁴¹41. Herbert C, El Bolock A, Abdennadher S. How do you feel during the COVID-19 pandemic? A survey using psychological and linguistic self-report measures, and machine learning to investigate mental health, subjective experience, personality, and behaviour during the COVID-19 pandemic among university students. BMC Psychology 2021; 9:90.^,4444. Maitre J, Bergeron-Leclerc C, Maltais D, Gaboury S. Investigating anxiety levels in the Quebec university community during the COVID-19 pandemic using machine learning and data exploration techniques. Multimed Tools Appl 2023; 82:46109-27.^,5555. Liu M, Xu Y, Wu H, Wang X, Ye B. Blood stasis constitution and depression among Chinese female college students: a longitudinal moderation model. Int J Ment Health Addict 2023; 21:929-43.^,6060. Zhang B, Zaman A, Silenzio V, Kautz H, Hoque E. The relationships of deteriorating depression and anxiety with longitudinal behavioral changes in Google and YouTube use during COVID-19: observational study. JMIR Ment Health 2020; 7:e24012.^,6262. Aalbers G, Hendrickson AT, Vanden Abeele MM, Keijsers L. Smartphone-tracked digital markers of momentary subjective stress in college students: idiographic machine learning analysis. JMIR Mhealth Uhealth 2023; 11:e37469. studies presented other metrics, such as regression or correlation coefficients. All models exhibited at least one acceptable performance score, that is, above 0.5. Stress detection accuracy ranged from 63% to 100%, anxiety detection accuracy ranged from 53.68% to 97.9%, and depression detection accuracy ranged from 73.5% to 99.1%. These results raise the hypothesis that models targeting stress detection may exhibit subtly higher accuracy compared to those for anxiety and depression. However, further investigation with more homogeneous and comprehensive data is essential to test this hypothesis.

Regarding accuracy specifically, 30 out of these 33 studies (90.9%) ³⁰30. Amalraj JDI, Bojan VK, Murugasamy K. Detection of stress level based on sweat from Gen-Z students using ANN and GA algorithms. Int J Clin Exp Med 2023; 16:260-74.^,3131. Jiao Y, Wang X, Liu C, Du G, Zhao L, Dong H, et al. Feasibility study for detection of mental stress and depression using pulse rate variability metrics via various durations. Biomed Signal Process Control 2023; 79:104145.^,3232. Pal R, Adhikari D, Heyat MBB, Guragai B, Lipari V, Brito Ballester J, et al. A novel smart belt for anxiety detection, classification, and reduction using IIoMT on students' cardiac signal and MSY. Bioengineering 2022; 9:793.^,3333. Pourmohammadi S, Maleki A. Stress detection using ECG and EMG signals: a comprehensive study. Comput Methods Programs Biomed 2020; 193:105482.^,3434. Sharma V, Prakash NR, Kalra P. Depression status identification using autoencoder neural network. Biomed Signal Process Control 2022; 75:103568.^,3636. Tiwari S, Agarwal S. A shrewd artificial neural network-based hybrid model for pervasive stress detection of students using galvanic skin response and electrocardiogram signals. Big Data 2021; 9:427-42.^,3737. Anand RV, Md AQ, Urooj S, Mohan S, Alawad MA. Enhancing diagnostic decision-making: ensemble learning techniques for reliable stress level classification. Diagnostics 2023; 13:3455.^,3838. Balli M, Okan A, Gürsan NÖ, Gülgöz S, Eser HY. Comparison of machine learning algorithms for Beck Depression Inventory measured depression status classification. Eur Psychiatry 2023; 66 Suppl 1:S419.^,3939. Daza A, Bobadilla J, Apaza O, Pinto J. Stacking ensemble learning model for predict anxiety level in university students using balancing methods. Inform Med Unlocked 2023; 42:101340.^,4242. Ge F, Zhang D, Wu L, Mu H. Predicting psychological state among Chinese undergraduate students in the COVID-19 epidemic: a longitudinal study using a machine learning. Neuropsychiatr Dis Treat 2020; 16:2111-8.^,4343. Gil M, Kim SS, Min EJ. Machine learning models for predicting risk of depression in Korean college students: identifying family and individual factors. Front Public Health 2022; 10:1023010.^,4646. Ren Z, Xin Y, Ge J, Liu D, Ho CS. Psychological impact of COVID-19 on college students after school reopening: a cross-sectional study based on machine learning. Front Psychol 2021; 12:641806.^,4747. Upadhyay DK, Mohapatra S, Singh NK. An early assessment of Persistent Depression Disorder using machine learning algorithm. Multimed Tools Appl 2023; 83:49149-71.^,4848. Vergaray A, Miranda JCH, Cornelio JB, Carranza ARL, Sánchez CFP. Predicting the depression in university students using stacking ensemble techniques over oversampling method. Inform Med Unlocked 2023; 41:101295.^,4949. Wang C, Zhao H, Zhang H. Chinese college students have higher anxiety in new semester of online learning during COVID-19: a machine learning approach. Front Psychol 2020; 11:587413.^,5050. AlShorman O, Masadeh M, Heyat MBB, Akhtar F, Almahasneh H, Ashraf GM, et al. Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection. J Integr Neurosci 2022; 21:20.^,5252. Li X, Hu B, Shen J, Xu T, Retcliffe M. Mild depression detection of college students: an EEG-based solution with free viewing tasks. J Med Syst 2015; 39:187.^,5353. Modinos G, Mechelli A, Pettersson-Yeo W, Allen P, McGuire P, Aleman A. Pattern classification of brain activation during emotional processing in subclinical depression: psychosis proneness as potential confounding factor. PeerJ 2013; 1:e42.^,5454. Zhang W, Shen Q, Song J, Zhou R. Classification of test-anxious individuals using Event-Related Potentials (ERPs): the effectiveness of machine learning algorithms. Acta Psychologica Sinica 2019; 51:1116-27.^,5656. Topalovic N, Mazic S, Nesic D, Vukovic O, Cumic J, Laketic D, et al. Association between chromatin structural organization of peripheral blood neutrophils and self-perceived mental stress: gray-level co-occurrence matrix analysis. Microsc Microanal 2021; 27:1202-8.^,5757. Ding Y, Chen X, Fu Q, Zhong S. A depression recognition method for college students using deep integrated support vector algorithm. IEEE Access 2020; 8:75616-29.^,5858. Dehghan-Bonari M, Alipour-Vaezi M, Nasiri MM, Aghsami A. A diagnostic analytics model for managing post-disaster symptoms of depression and anxiety among students using a novel data-driven optimization approach. Healthc Anal 2023; 4:100238.^,5959. Siraji MI, Rahman AA, Nishat MM, Al Mamun MA, Faisal F, Khalid LI. Impact of mobile connectivity on students' wellbeing: detecting learners' depression using machine learning algorithms. PLoS One 2023; 18:e0294803.^,6464. Ahmed MS, Ahmed N. A fast and minimal system to identify depression using smartphones: explainable machine learning-based approach. JMIR Form Res 2023; 7:e28848.^,6565. Chikersal P, Doryab A, Tumminia M, Villalba DK, Dutcher JM, Liu X, et al. Detecting depression and predicting its onset using longitudinal symptoms captured by passive sensing. ACM Trans Comput Hum Interact 2021; 28:3.^,7070. Bhadra S, Kumar CJ. Enhancing the efficacy of depression detection system using optimal feature selection from EHR. Comput Methods Biomech Biomed Engin 2024; 27:222-36.^,7171. Rois R, Ray M, Rahman A, Roy SK. Prevalence and predicting factors of perceived stress among Bangladeshi university students using machine learning algorithms. J Health Popul Nutr 2021; 40:50.^,7272. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res 2018; 20:e9410.^,7474. Xu X, Chikersal P, Dutcher JM, Sefidgar YS, Seo W, Tumminia MJ, et al. Leveraging collaborative-filtering for personalized behavior modeling: a case study of depression detection among college students. Proc ACM Interact Mob Wearable Ubiquitous Technol 2021; 5:41.^,7777. Nayan MIH, Uddin MSG, Hossain MI, Alam MM, Zinnia MA, Haq I, et al. Comparison of the performance of machine learning-based algorithms for predicting depression and anxiety among university students in Bangladesh: a result of the first wave of the COVID-19 pandemic. Asian J Soc Health Behav 2022; 5:75-84. reported at least one accuracy score above 70%, categorizing them as achieving good accuracy ²³23. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72.. Additionally, 26 out of these 33 studies (78.78%) ³⁰30. Amalraj JDI, Bojan VK, Murugasamy K. Detection of stress level based on sweat from Gen-Z students using ANN and GA algorithms. Int J Clin Exp Med 2023; 16:260-74.^,3131. Jiao Y, Wang X, Liu C, Du G, Zhao L, Dong H, et al. Feasibility study for detection of mental stress and depression using pulse rate variability metrics via various durations. Biomed Signal Process Control 2023; 79:104145.^,3232. Pal R, Adhikari D, Heyat MBB, Guragai B, Lipari V, Brito Ballester J, et al. A novel smart belt for anxiety detection, classification, and reduction using IIoMT on students' cardiac signal and MSY. Bioengineering 2022; 9:793.^,3333. Pourmohammadi S, Maleki A. Stress detection using ECG and EMG signals: a comprehensive study. Comput Methods Programs Biomed 2020; 193:105482.^,3434. Sharma V, Prakash NR, Kalra P. Depression status identification using autoencoder neural network. Biomed Signal Process Control 2022; 75:103568.^,3636. Tiwari S, Agarwal S. A shrewd artificial neural network-based hybrid model for pervasive stress detection of students using galvanic skin response and electrocardiogram signals. Big Data 2021; 9:427-42.^,3737. Anand RV, Md AQ, Urooj S, Mohan S, Alawad MA. Enhancing diagnostic decision-making: ensemble learning techniques for reliable stress level classification. Diagnostics 2023; 13:3455.^,3838. Balli M, Okan A, Gürsan NÖ, Gülgöz S, Eser HY. Comparison of machine learning algorithms for Beck Depression Inventory measured depression status classification. Eur Psychiatry 2023; 66 Suppl 1:S419.^,3939. Daza A, Bobadilla J, Apaza O, Pinto J. Stacking ensemble learning model for predict anxiety level in university students using balancing methods. Inform Med Unlocked 2023; 42:101340.^,4242. Ge F, Zhang D, Wu L, Mu H. Predicting psychological state among Chinese undergraduate students in the COVID-19 epidemic: a longitudinal study using a machine learning. Neuropsychiatr Dis Treat 2020; 16:2111-8.^,4343. Gil M, Kim SS, Min EJ. Machine learning models for predicting risk of depression in Korean college students: identifying family and individual factors. Front Public Health 2022; 10:1023010.^,4646. Ren Z, Xin Y, Ge J, Liu D, Ho CS. Psychological impact of COVID-19 on college students after school reopening: a cross-sectional study based on machine learning. Front Psychol 2021; 12:641806.^,4747. Upadhyay DK, Mohapatra S, Singh NK. An early assessment of Persistent Depression Disorder using machine learning algorithm. Multimed Tools Appl 2023; 83:49149-71.^,4848. Vergaray A, Miranda JCH, Cornelio JB, Carranza ARL, Sánchez CFP. Predicting the depression in university students using stacking ensemble techniques over oversampling method. Inform Med Unlocked 2023; 41:101295.^,4949. Wang C, Zhao H, Zhang H. Chinese college students have higher anxiety in new semester of online learning during COVID-19: a machine learning approach. Front Psychol 2020; 11:587413.^,5050. AlShorman O, Masadeh M, Heyat MBB, Akhtar F, Almahasneh H, Ashraf GM, et al. Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection. J Integr Neurosci 2022; 21:20.^,5252. Li X, Hu B, Shen J, Xu T, Retcliffe M. Mild depression detection of college students: an EEG-based solution with free viewing tasks. J Med Syst 2015; 39:187.^,5454. Zhang W, Shen Q, Song J, Zhou R. Classification of test-anxious individuals using Event-Related Potentials (ERPs): the effectiveness of machine learning algorithms. Acta Psychologica Sinica 2019; 51:1116-27.^,5757. Ding Y, Chen X, Fu Q, Zhong S. A depression recognition method for college students using deep integrated support vector algorithm. IEEE Access 2020; 8:75616-29.^,5858. Dehghan-Bonari M, Alipour-Vaezi M, Nasiri MM, Aghsami A. A diagnostic analytics model for managing post-disaster symptoms of depression and anxiety among students using a novel data-driven optimization approach. Healthc Anal 2023; 4:100238.^,5959. Siraji MI, Rahman AA, Nishat MM, Al Mamun MA, Faisal F, Khalid LI. Impact of mobile connectivity on students' wellbeing: detecting learners' depression using machine learning algorithms. PLoS One 2023; 18:e0294803.^,6565. Chikersal P, Doryab A, Tumminia M, Villalba DK, Dutcher JM, Liu X, et al. Detecting depression and predicting its onset using longitudinal symptoms captured by passive sensing. ACM Trans Comput Hum Interact 2021; 28:3.^,7070. Bhadra S, Kumar CJ. Enhancing the efficacy of depression detection system using optimal feature selection from EHR. Comput Methods Biomech Biomed Engin 2024; 27:222-36.^,7171. Rois R, Ray M, Rahman A, Roy SK. Prevalence and predicting factors of perceived stress among Bangladeshi university students using machine learning algorithms. J Health Popul Nutr 2021; 40:50.^,7272. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res 2018; 20:e9410.^,7777. Nayan MIH, Uddin MSG, Hossain MI, Alam MM, Zinnia MA, Haq I, et al. Comparison of the performance of machine learning-based algorithms for predicting depression and anxiety among university students in Bangladesh: a result of the first wave of the COVID-19 pandemic. Asian J Soc Health Behav 2022; 5:75-84. achieved at least one accuracy score above 80%, which can be classified as excellent accuracy ²³23. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72.. These findings align with other systematic reviews in the field of mental health, which also identified satisfactory performance in most models that assessed conditions such as post-traumatic stress, depression, suicidal ideation, and anxiety ²¹21. Chung J, Teo J. Mental health prediction using machine learning: taxonomy, applications, and challenges. Applied Computational Intelligence and Soft Computing 2022; 2022:e9970363.^,2222. Lee Y, Ragguett R-M, Mansur RB, Boutilier JJ, Rosenblat JD, Trevizol A, et al. Applications of machine learning algorithms to predict therapeutic outcomes in depression: a meta-analysis and systematic review. J Affect Disord 2018; 241:519-32.^,2323. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72.^,2424. Thieme A, Belgrave D, Doherty G. Machine learning in mental health. ACM Trans Comput Hum Interact 2020; 27:1-53.^,2525. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16.^,7878. Lejeune A, Le Glaz A, Perron PA, Sebti J, Baca-Garcia E, Walter M, et al. Artificial intelligence and suicide prevention: a systematic review. Eur Psychiatry 2022; 65:5929.. It is plausible that these models could exhibit enhanced accuracy by accounting for the influence of potential comorbidities, given that the presence of other psychopathological symptoms may impact the precision of machine learning models ²⁵25. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16..

The studies that demonstrated the best model performance employed physiological data and showed stress as an outcome. Pourmohammadi & Maleki ³³33. Pourmohammadi S, Maleki A. Stress detection using ECG and EMG signals: a comprehensive study. Comput Methods Programs Biomed 2020; 193:105482. and Tiwari & Agarwal ³⁶36. Tiwari S, Agarwal S. A shrewd artificial neural network-based hybrid model for pervasive stress detection of students using galvanic skin response and electrocardiogram signals. Big Data 2021; 9:427-42. developed models with accuracies of 100% and 99.4%, respectively. A possible explanation is that machine learning models based on data correlated with the outcome tend to perform better ²⁵25. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16.. The association between stress variables and parameters such as blood pressure, skin conductivity, and heart rate are well-established and can account for these positive results ⁷⁹79. Noushad S, Ahmed S, Ansari B, Mustafa UH, Saleem Y, Hazrat H. Physiological biomarkers of chronic stress: a systematic review. Int J Health Sci (Qassim) 2021; 15:46-59.. However, we highlight that both studies induced stress via a laboratory experiment, which differs from the stress experienced in an academic context.

Conversely, the two studies ⁵¹51. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40.^,6969. Nemesure MD, Heinz MV, Huang R, Jacobson NC. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep 2021; 11:1980. that exhibited the lowest models performances were based on neuroimaging ⁵¹51. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40. and mixed data ⁶⁹69. Nemesure MD, Heinz MV, Huang R, Jacobson NC. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep 2021; 11:1980.. He et al. ⁵¹51. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40. found a specificity of 32.88% in distinguishing individuals with anxiety from those with symptoms of schizophrenia. This observation can be partially attributed to the linear relationship between anxiety and psychosis variables, possibly implicating the activation of overlapping brain regions ⁵¹51. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40.. On the other hand, Nemesure et al. ⁶⁹69. Nemesure MD, Heinz MV, Huang R, Jacobson NC. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep 2021; 11:1980. reported a sensitivity of merely 55% in identifying major depression among university students.

When examining the performance of machine learning algorithms, it is not possible to definitively assert the superiority of any specific technique. Algorithmic performance is contingent upon specific factors, including objectives, data volume and type, case distribution, outlier, noise management, among others. Consequently, the presence of a diverse array of algorithms in the evaluated studies is expected, given the variations in objectives, data types, and dataset characteristics. In this systematic review, SVM algorithms and their variations predominate, accounting for 35.41% of cases, a trend also observed in other systematic reviews within the field of mental health ²³23. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72.^,2525. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16.. This could be attributed to the fact that SVM algorithms excel in processing structured data, particularly in binary outcome classifications.

If, on the one hand, the performance data is promising, on the other hand, it is important to highlight that only one study ⁵¹51. He L, Wei D, Yang F, Zhang J, Cheng W, Feng J, et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am J Psychiatr 2021; 178:530-40. indicated external validation of the machine learning model. machine learning models that only rely on internal validation may overestimate their performance. Further studies must perform external validation of their machine learning models to disseminate them among the population.

Despite the adequate results, it should be noted that the quality of evidence from all studies was considered very low after GRADE assessment. These results suggest the importance of conducting studies that improve the assessment of outcomes and use larger and more representative samples. Issues pertaining to the construction of machine learning models were also identified. Only nine ³⁵35. Silva E, Aguiar J, Reis LP, Sá JO, Gonçalves J, Carvalho V. Stress among Portuguese medical students: the EuStress Solution. J Med Syst 2020; 44:45.^,3737. Anand RV, Md AQ, Urooj S, Mohan S, Alawad MA. Enhancing diagnostic decision-making: ensemble learning techniques for reliable stress level classification. Diagnostics 2023; 13:3455.^,3939. Daza A, Bobadilla J, Apaza O, Pinto J. Stacking ensemble learning model for predict anxiety level in university students using balancing methods. Inform Med Unlocked 2023; 42:101340.^,4646. Ren Z, Xin Y, Ge J, Liu D, Ho CS. Psychological impact of COVID-19 on college students after school reopening: a cross-sectional study based on machine learning. Front Psychol 2021; 12:641806.^,4848. Vergaray A, Miranda JCH, Cornelio JB, Carranza ARL, Sánchez CFP. Predicting the depression in university students using stacking ensemble techniques over oversampling method. Inform Med Unlocked 2023; 41:101295.^,5252. Li X, Hu B, Shen J, Xu T, Retcliffe M. Mild depression detection of college students: an EEG-based solution with free viewing tasks. J Med Syst 2015; 39:187.^,6161. Ware S, Yue C, Morillo R, Lu J, Shang C, Bi J, et al. Predicting depressive symptoms using smartphone data. Smart Health (Amst) 2020; 15:100093.^,6363. Acikmese Y, Alptekin SE. Prediction of stress levels with LSTM and passive mobile sensors. Procedia Comput Sci 2019; 159:658-67.^,7272. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res 2018; 20:e9410. studies outlined measures to address class imbalance. Moreover, several studies featured a sample size of fewer than 55 participants, which is considered small ⁸⁰80. Abd-Alrazaq A, AlSaad R, Alhuwail D, Ahmed A, Healy PM, Latifi S, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ 2023; 9:e48291..

Most models may inherit limitations from the diagnostic process itself. Psychometric instruments, such as the Beck Depression Inventory-II, inherently possess measurement errors that can be replicated in these models. Moreover, these instruments can be influenced by respondents’ tendencies towards socially desirable responses. It is essential that mental health diagnoses stem from a triangulation of diverse sources of evidence ⁸⁰80. Abd-Alrazaq A, AlSaad R, Alhuwail D, Ahmed A, Healy PM, Latifi S, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ 2023; 9:e48291., including qualitative and exploratory data, clinical interviews, observational data, and self-report instruments. Notably, only seven studies ⁴⁷47. Upadhyay DK, Mohapatra S, Singh NK. An early assessment of Persistent Depression Disorder using machine learning algorithm. Multimed Tools Appl 2023; 83:49149-71.^,5454. Zhang W, Shen Q, Song J, Zhou R. Classification of test-anxious individuals using Event-Related Potentials (ERPs): the effectiveness of machine learning algorithms. Acta Psychologica Sinica 2019; 51:1116-27.^,6161. Ware S, Yue C, Morillo R, Lu J, Shang C, Bi J, et al. Predicting depressive symptoms using smartphone data. Smart Health (Amst) 2020; 15:100093.^,6969. Nemesure MD, Heinz MV, Huang R, Jacobson NC. Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Sci Rep 2021; 11:1980.^,7070. Bhadra S, Kumar CJ. Enhancing the efficacy of depression detection system using optimal feature selection from EHR. Comput Methods Biomech Biomed Engin 2024; 27:222-36.^,7373. Ware S, Yue C, Morillo R, Shang C, Bi J, Kamath J, et al. Automatic depression screening using social interaction data on smartphones. Smart Health (Amst) 2022; 26:100356.^,7575. Yue C, Ware S, Morillo R, Lu J, Shang C, Bi J, et al. Fusing location data for depression prediction. IEEE Trans Big Data 2021; 7:355-70. constructed models after evaluation by healthcare professionals. However, the literature points that involving trained clinicians in this process can be more resource-intensive ⁸⁰80. Abd-Alrazaq A, AlSaad R, Alhuwail D, Ahmed A, Healy PM, Latifi S, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ 2023; 9:e48291..

The findings of this systematic review offer promise from a public health perspective, indicating that machine learning algorithms may serve as valuable tools for the detection of depression, anxiety, and stress among university students using various types of data. Consequently, they show potential to enhance mental health support for university students, particularly those in remote or rural areas. These algorithms can aid identifying students at risk or flagging cases of depression, anxiety, and stress. Moreover, this study aligns with previous research endorsing the application of machine learning in mental healthcare ⁸¹81. Shatte ABR, Hutchinson DM, Teague SJ. Machine learning in mental health: a scoping review of methods and applications. Psychol Med 2019; 49:1426-48.. Although some machine learning initiatives have been under development in other regions, we highlight that most assessed studies are concentrated in European countries, China, and the United States. Expanding machine learning research and implementation in developing countries could significantly contribute to the advancement of mental healthcare worldwide.

Finally, a potential challenge to the widespread adoption of machine learning models in public health is the type of data they depend on. Models that rely on neuroimaging and physiological data collected via electromyograms and electroencephalograms demand specialized data collection and can present practical challenges for real-world implementation ²⁵25. Wu Y, Mao K, Dennett L, Zhang Y, Chen J. Systematic review of machine learning in PTSD studies for automated diagnosis evaluation. NPJ Ment Health Res 2023; 2:16.. In contrast, models that are built employing behavioral data gathered from research or even linguistic interactions on social networks may offer a more practical and feasible approach. Therefore, it is crucial to assess the potential challenges and advantages associated with each model when applied to real-world contexts. Additionally, the consideration of ethical issues is of significance.

Ethical issues

The use of machine learning has sparked ethical discussions, particularly regarding the privacy of personal data and the purpose of these models. Interestingly, only a few studies ⁴⁰40. Estabragh ZS, Kashani MM, Moghaddam F, Sari S, Taherifar Z, Moradi Moosavy S, et al. Bayesian network modeling for diagnosis of social anxiety using some cognitive-behavioral factors. Netw Model Anal Health Inform Bioinform 2013; 2:257-65.^,4141. Herbert C, El Bolock A, Abdennadher S. How do you feel during the COVID-19 pandemic? A survey using psychological and linguistic self-report measures, and machine learning to investigate mental health, subjective experience, personality, and behaviour during the COVID-19 pandemic among university students. BMC Psychology 2021; 9:90. in this review mentioned ethical issues related to machine learning. Models should prioritize the protection of personal data, especially when dealing with sensitive content, such as language patterns and interactions on social networks, as well as smartphones messages and calls. Moreover, these algorithms must solely aim at identifying mental health issues for prevention and promotion of mental well-being, protecting sensitive data from vested interests.

Limitations

This systematic review shows methodological limitations that suggest caution in interpreting and generalizing the results. These limitations refer to the inclusion and exclusion criteria and quality of the machine learning models.

Firstly, the inclusion criteria may have limited the number of articles. To refine the quality of the articles, we decided to exclude those published in gray literature. Thus, book chapters and articles from conference proceedings and references were excluded. Secondly, based on previous research, we found no validated instrument to assess the quality of machine learning articles. Therefore, an instrument that has not yet been validated ²³23. Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-72. was employed to assess the articles.

Conclusion

The findings of this review suggest that most machine learning models demonstrate adequate performance in assessing the intended outcomes, particularly stress. Various types of data were employed in these machine learning models, indicating that depression, anxiety, and stress may be predicted or classified using various approaches, although concerns persist regarding the certainty of evidence of models, which may be considered very low. These results hold promise for the application of machine learning in public health, as it can assist in identifying students at risk of mental illness or those experiencing depression, anxiety, and stress.

Machine learning algorithms show the potential to significantly enhance the accessibility of mental health services by enabling accurate real-time assessments, often remotely, even with non-linear data. This capacity is especially valuable for improving mental healthcare in rural or underserved areas with limited access to traditional mental health services. Thus, we suggest further development of machine learning models, with a particular focus on incorporating various sources of evidence for classifying outcomes, beyond solely relying on self-report instruments. It is essential that future studies also perform external validation of machine learning models to obtain more consistent and realistic performance data. Wider dissemination of these studies can facilitate the adoption of more rigorous statistical techniques, including meta-analysis, which can offer more conclusive insights into the performance and practical utility of these models.

Acknowledgments

B. L. Schaab and P. Ü. Calvetti were supported by a grant from the Brazilian Coordination for the Improvement of Higher Education Personnel (CAPES). C. T. Reppold was supported by a grant from the Brazilian National Research Council (CNPq).

References

Publication Dates

History