Enhancing Student Anxiety Detection: A Multimodal Transformer Approach to Video-Based Screening
DOI:
https://doi.org/10.56988/chiprof.v5i1.158Keywords:
Anxiety Screening, Facial Expressions, Multimodal Transformer, Speech Transcripts, Student AnxietyAbstract
This study developed and applied a multimodal Transformer model for student anxiety screening through video analysis of short interviews that included facial expressions, speech, and numerical data. Student anxiety is a problem that often affects mental health and academic performance, so early detection is important. The model combines three main data sources: facial expression features, speech analysis (including speech speed, intonation, and negative word count), and demographic information. The data used came from 500 students who participated in interviews lasting 20-40 seconds. The multimodal Transformer model was trained to classify anxiety levels into low, medium, and high categories, with evaluation using accuracy, precision, and recall metrics. The results showed that this model had a prediction accuracy of 88%, with a significant correlation between facial expressions and negative word counts on anxiety levels. Compared to the linear regression model used for comparison, the multimodal Transformer model shows better performance in detecting anxiety. These findings indicate that a multimodal approach using AI technology can improve accuracy and efficiency in student anxiety screening. This research opens up opportunities for the development of a more objective, non-invasive, and efficient video-based automated screening system, with potential applications in the field of mental health in higher education.
Downloads
References
G. Barbayannis, M. Bandari, X. Zheng, and H. Baquerizo, “Academic Stress and Mental Well-Being in College Students : Correlations , Affected Groups , and,” vol. 13, no. May, pp. 1–10, 2022, doi: 10.3389/fpsyg.2022.886344.
E. D. Nugraini, “Digital evolutions : Analysis of self-talk transformation among Generation Z and its impact on counseling services,” 2026.
U. Haruna, A. Mohammed, and M. Braimah, “Understanding the burden of depression , anxiety and stress among first-year undergraduate students,” 2025.
T. Literatur, K. Psikologis, and P. Kanker, “Berajah Journal,” pp. 129–142, 2018.
Peer-Led Support Groups : Overview of the Empirical Research and Implications for Individuals Who Have Experienced Trafficking and Substance Use Disorder.
E. J. Liebling, J. J. S. Perez, M. M. Litterer, and C. Greene, “Implementing hospital-based peer recovery support services for substance use disorder,” Am. J. Drug Alcohol Abuse, vol. 47, no. 2, pp. 229–237, 2021, doi: 10.1080/00952990.2020.1841218.
A. Framework, “Aberrated Multidimensional EEG Characteristics in Patients with Generalized Anxiety Disorder : A Machine-Learning Based,” 2022.
W. Barnett, J. Garon-bissonnette, C. Carrow, H. A. Piersiak, and L. G. Bailes, “Reading the mind in infant eyes test : A measure of the recognition of infant emotion,” pp. 1–10, 2025, doi: 10.1017/S0954579425000185.
Y. N. Kunang and W. P. Mentari, “Analysis of the Impact of Vectorization Methods on Machine Learning-Based Sentiment Analysis of Tweets Regarding Readiness for Offline Learning,” vol. 11, no. 2, pp. 271–280, 2023.
“Improving ChatGPT’s emotional intelligence through prompt engineering,” no. November, 2023.
H. Shin and H. Kim, “Explainable vision transformer for automatic visual sleep staging on multimodal PSG signals,” npj Digit. Med., pp. 1–14, 2025, doi: 10.1038/s41746-024-01378-0.
Y. Byeon et al., “Interpretable multimodal transformer for prediction of molecular subtypes and grades in adult-type diffuse gliomas,” npj Digit. Med., pp. 1–10, 2025, doi: 10.1038/s41746-025-01530-4.
J. Li et al., “Next-generation AI framework for comprehensive oral leukoplakia evaluation and management,” pp. 1–10, 2025.
L. Wang et al., “AI-assisted multi-modal information for the screening of depression : a systematic review and meta-analysis,” pp. 1–14, 2025.
N. Gour et al., “Transformers for autonomous recognition of psychiatric dysfunction via raw and imbalanced EEG signals,” Brain Informatics, 2023, doi: 10.1186/s40708-023-00201-y.
J. July, D. Santos, and A. Faro, “Predictive Model of Anxious and Depressive Symptoms Based on the Relationship Between Hardiness and Academic Adaptation Modelo Preditivo dos Sintomas Ansiosos e Depressivos com Base na Relação Entre Hardiness e Adaptação Acadêmica,” pp. 1–11, 2025.
T. M. Alanazi, “Multimodal feature distinguishing and deep learning approach to detect lung disease from MRI images,” pp. 1–16, 2025.
B. Kurniawan, R. Wahyuni, Y. Irawan, and M. H. Yuhandri, “Multimodal Deep Learning and IoT Sensor Fusion for Real-Time Beef Freshness Detection,” vol. 6, no. 4, pp. 2921–2937, 2025.
C. Li, Y. Zhao, Y. Bai, B. Zhao, and Y. O. Tola, “Unveiling the Potential of Large Language Models in Transforming Chronic Disease Management : Mixed Methods Systematic Review Corresponding Author :,” vol. 27, 2025, doi: 10.2196/70535.
F. Damla and M. Cetin, “AI-driven early diagnosis of specific mental disorders : a comprehensive study,” vol. 0123456789, 2025, doi: 10.1007/s11571-025-10253-x.
M. D. Hamanrora, Y. N. Kunang, and I. Z. Yadi, “Image segmentation of Komering script using bounding box,” vol. 35, no. 3, pp. 1565–1578, 2024, doi: 10.11591/ijeecs.v35.i3.pp1565-1578.
A. Muzakir, K. Adi, and R. Kusumaningrum, “Revue d ’ Intelligence Artificielle Short Text Classification Based on Hybrid Semantic Expansion and Bidirectional GRU ( BiGRU ) Based Method to Improve Hate Speech Detection,” vol. 37, no. 6, pp. 1471–1481, 2023.
A. Muzakir, R. M. N. Halim, and A. Wijaya, “Perangkat Lunak Pencarian Apotek Menggunakan Metode Item Based Collaborative Filltering Dan Algoritma Floyd Warshall,” vol. 1, no. 3, pp. 122–133, 2020.
S. Li and H. Li, “Feature fusion based transformer for,” pp. 1–23, 2025, doi: 10.1371/journal.pone.0333416.
Q. Zhang, Z. Song, D. Wang, Y. Cai, M. Bi, and M. Zuo, “Named entity recognition and coreference resolution using prompt-based generative multimodal,” vol. 123, 2026.
V. Calderón-fajardo and I. Rodríguez-rodríguez, “From words to visuals : a transformer-based multi-modal framework for emotion-driven tourism analytics,” pp. 939–979, 2025.
C. Bormpotsis, M. Anagnostouli, M. Sedky, and E. Jelastopulu, “Mobile Mental Health Screening in EmotiZen via the Novel,” pp. 1–33, 2025.
E. Maekawa et al., “Bayesian Networks for Prescreening in Depression: Algorithm Development and Validation,” JMIR Ment. Heal., vol. 11, pp. 1–18, 2024, doi: 10.2196/52045.
L. Zhang, “Applied Mathematics and Nonlinear Sciences,” vol. 9, no. 1, pp. 1–14, 2024.
Yunike, Jawiah, and K. N. Febrianti, “Storytelling Dolls Reduce Children ’ s Anxiety During Hospitalization,” vol. 1, no. 1, pp. 19–24, 2024.
B. Grimm, B. Talbot, and L. Larsen, “applied sciences PHQ-V / GAD-V : Assessments to Identify Signals of Depression and Anxiety from Patient Video Responses,” 2022.
A. Mazur, “Evaluation of an AI-Based Voice Biomarker Tool to Detect Signals Consistent With Moderate to Severe Depression,” pp. 60–65, 2025.
J. Park and N. Moon, “Design and Implementation of Attention Depression Detection Model Based on Multimodal Analysis,” 2022.
O. Pierrès, A. Darvishy, and M. Christen, “Exploring the role of generative AI in higher education : Semi ‑ structured interviews with students with disabilities,” Educ. Inf. Technol., vol. 30, no. 7, pp. 8923–8952, 2025, doi: 10.1007/s10639-024-13134-8.
N. Ahmad, “Perceived Impact of Procrastination on Academic Performance Among Students and the Role of AI Tools,” Libri, vol. 75, no. 4, pp. 355–373, 2025, doi: 10.1515/libri-2025-0093.
C. Singh and A. Malespina, “Test anxiety , self-efficacy , and gender : A quest for equitable assessment practices in physics,” pp. 390–395, 2021.
L. H. Zeng, Y. Hao, J. Chao, H. Jhen, and N. Ye, “The relationship between teacher support and bullying in schools : the mediating role of emotional self ‑ efficacy,” Curr. Psychol., vol. 42, no. 36, pp. 31853–31862, 2023, doi: 10.1007/s12144-022-04052-4.
C. Bergeron-leclerc and S. Gaboury, “Exploring Anxiety of Québec University Community during COVID-19 Pandemic via Machine Learning,” pp. 55–60, doi: 10.1145/3524458.3547236.
M. Shoryabi, A. Hajipour, A. Shoeibi, and A. Foroutannia, “Recognition of anxiety and depression using gait data recorded by the kinect sensor : a machine learning approach with data augmentation,” pp. 1–16, 2025.
X. L. Liao, Y. Y. Deng, and Y. Kang, “Value of procalcitonin in diagnosing ventilator-associated pneumonia: A systematic review,” Chinese J. Evidence-Based Med., vol. 10, no. 8, pp. 910–915, 2010.
I. Kusumawaty, “Trust in digitalization and artificial intelligence : Insights from qualitative research on online parenting programs,” vol. 12, no. 5, 2024.
E. G. Colato, “Artificial intelligence ( AI ) -Enabled behavioral health application for college students : Pilot study protocol,” pp. 1–12, 2025, doi: 10.1371/journal.pone.0335847.
R. B. Bist, “A Novel YOLOv6 Object Detector for Monitoring Piling Behavior of Cage-Free Laying Hens,” pp. 905–923, 2023.
L. S. Id, W. Y. Id, and L. Y. Id, “How can industrial intelligence change the employment structure of the floating population ?,” pp. 1–17, 2024, doi: 10.1371/journal.pone.0297266.
J. Liu, “Exploring the impact of artificial intelligence-enhanced language learning on youths’ intercultural communication competence,” 2025.
P. Dhiman, “AI ’ s Impact on Social Integrity , Well-being and Academic Performance of International Students,” vol. 9, no. 2, pp. 1–23, 2025.
Y. Novaria, S. Nurmaini, D. Stiawan, and B. Yudho, “Journal of Information Security and Applications Attack classification of an intrusion detection system using deep learning and hyperparameter optimization,” J. Inf. Secur. Appl., vol. 58, no. March, p. 102804, 2021, doi: 10.1016/j.jisa.2021.102804.
T. Sutabri, “Implementation of YOLO Algorithm in Adolescent Suicide Ideation Monitoring System Based on Real-Time Data Analysis,” vol. 4, no. 1, pp. 334–344, 2025.
A. Algarni, “CareAssist GPT improves patient user experience with a patient centered approach to computer aided diagnosis,” pp. 1–28, 2025.
Y. Yang, F. Mohammadzadeh, M. Khishe, A. N. Ahmed, M. M. Abualhaj, and T. M. Ghazal, “Deep learning for sports motion recognition with a high-precision framework for performance enhancement,” pp. 1–26, 2025.
G. K. M et al., “Hybrid optimization driven fake news detection using reinforced transformer models,” pp. 1–16, 2025.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Yunike, Yesi Novaria Kunang, Ari Muzakir, Ira Kusumawaty
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
