Depression Classification Using Bert Embedding Model on Social Media Posts
Nareshkumar R
Department of Networking and Communications, School of Computing, College of Engineering and Technology (CET), SRM Institute of Science and Technology, Kattankulathur, Chennai, Tamil Nadu, India.
Department of Networking and Communications, School of Computing, College of Engineering and Technology (CET), SRM Institute of Science and Technology, Kattankulathur, Chennai, Tamil Nadu, India.
Developing sophisticated methods to precisely detect health-related concerns on social media, including identifying sadness and anxiety, has become essential due to the expansion of the Internet. These systems focus on using machine learning techniques to determine the meaning and structure of writings shared by users on social media. Social media users’ data is confusing and inconsistent. Novel methods using deep learning and social networking platforms data to detect health issues. Provide just a little bit of information and understanding on the various texts individuals provide. This investigation introduces an innovative approach utilizing BERT to accurately and specifically detect posts related to sadness and anxiety. This approach preserves the contextual and semantic importance of words across the collection. The researcher employed word2vec, fasttext, BERT and Enhanced Grey Wolf Optimizer (E-GWO) and Deep Learning technologies to promptly analyze and detect indications of anxiety and melancholy in social media messages. Our solution surpasses previous advanced methods and incorporates the knowledge distillation methodology to achieve an accuracy of 95.9%.
Keywords
Depression, Social Media Posts, Natural Language Processing, LSTM, Deep Learning, RNN, BERT.
C. Su, Z. Xu, J. Pathak, and F. Wang, “Deep learning in mental health outcome research: a scoping review,” Translational Psychiatry, vol. 10, no. 1, Apr. 2020, doi: 10.1038/s41398-020-0780-3.
S. R. M, A. R. L, and A. A, “Detection of Depression among Social Media Users with Machine Learning,” Webology, vol. 19, no. 1, pp. 250–257, Jan. 2022, doi: 10.14704/web/v19i1/web19019.
H. Kulkarni, S. Macavaney, N. Goharian, O. Frieder, “TBD3: A Thresholding-Based Dynamic Depression Detection from Social Media for Low-Resource Users,” Proceedings of the Thirteenth Language Resources and Evaluation Conference, PP. 2157-2165, 2022.
X. Wang, Y. Sui, K. Zheng, Y. Shi, and S. Cao, “Personality Classification of Social Users Based on Feature Fusion,” Sensors, vol. 21, no. 20, p. 6758, Oct. 2021, doi: 10.3390/s21206758.
T. Zhang, K. Yang, S. Ji, and S. Ananiadou, “Emotion fusion for mental illness detection from social media: A survey,” Information Fusion, vol. 92, pp. 231–246, Apr. 2023, doi: 10.1016/j.inffus.2022.11.031.
H. Zogan, I. Razzak, S. Jameel, and G. Xu, “DepressionNet: Learning Multi-modalities with User Post Summarization for Depression Detection on Social Media,” Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 133–142, Jul. 2021, doi: 10.1145/3404835.3462938.
N. V. Babu and E. G. M. Kanaga, “Sentiment Analysis in Social Media Data for Depression Detection Using Artificial Intelligence: A Review,” SN Computer Science, vol. 3, no. 1, Nov. 2021, doi: 10.1007/s42979-021-00958-1.
J. A. García-Díaz, S. M. Jiménez-Zafra, M. A. García-Cumbreras, and R. Valencia-García, “Evaluating feature combination strategies for hate-speech detection in Spanish using linguistic features and transformers,” Complex & Intelligent Systems, vol. 9, no. 3, pp. 2893–2914, Feb. 2022, doi: 10.1007/s40747-022-00693-x.
T. Nijhawan, G. Attigeri, and T. Ananthakrishna, “Stress detection using natural language processing and machine learning over social interactions,” Journal of Big Data, vol. 9, no. 1, Mar. 2022, doi: 10.1186/s40537-022-00575-6.
WHO, WHO. Depression and other common mental disorders: global health estimates ,Depress Other Common Ment Disord Glob Heal Estim (2017).
B. Guo, C. Zhang, J. Liu, and X. Ma, “Improving text classification with weighted word embeddings via a multi-channel TextCNN model,” Neurocomputing, vol. 363, pp. 366–374, Oct. 2019, doi: 10.1016/j.neucom.2019.07.052.
T. Zhang, A. M. Schoene, S. Ji, and S. Ananiadou, “Natural language processing applied to mental illness detection: a narrative review,” npj Digital Medicine, vol. 5, no. 1, Apr. 2022, doi: 10.1038/s41746-022-00589-7.
T. Nijhawan, G. Attigeri, and T. Ananthakrishna, “Stress detection using natural language processing and machine learning over social interactions,” Journal of Big Data, vol. 9, no. 1, Mar. 2022, doi: 10.1186/s40537-022-00575-6.
V. Sanh, L. Debut, J. Chaumond and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” 2019, arXiv preprint arXiv:1910.01108.
Q. Nisa and R. Muhammad, „Towards transfer learning using BERT for early detection of self-harm of social media users,” Proceedings of the Working Notes of CLEF, pp. 21-24, 2021.
H. Zogan, X. Wang, S. Jameel, G. Xu, “Depression Detection with Multi Modalities Using a Hybrid Deep Learning Model on Social Media,” CoRR,02847 (2020).
T. Chen, R. Xu, Y. He, and X. Wang, “Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN,” Expert Systems with Applications, vol. 72, pp. 221–230, Apr. 2017, doi: 10.1016/j.eswa.2016.10.065.
H. Metzler, H. Baginski, T. Niederkrotenthaler, and D. Garcia, “Detecting Potentially Harmful and Protective Suicide-Related Content on Twitter: Machine Learning Approach,” Journal of Medical Internet Research, vol. 24, no. 8, p. e34705, Aug. 2022, doi: 10.2196/34705.'
A. Kumar, A. Sharma, A. Arora, “Anxious depression prediction in real-time social data,” 2019, arXiv preprint arXiv:1903.10222.
M. M. Tadesse, H. Lin, B. Xu, and L. Yang, “Detection of Depression-Related Posts in Reddit Social Media Forum,” IEEE Access, vol. 7, pp. 44883–44893, 2019, doi: 10.1109/access.2019.2909180.
H. S. ALSAGRI and M. YKHLEF, “Machine Learning-Based Approach for Depression Detection in Twitter Using Content and Activity Features,” IEICE Transactions on Information and Systems, vol. E103.D, no. 8, pp. 1825–1832, Aug. 2020, doi: 10.1587/transinf.2020edp7023.
M. Dong, Y. Li, X. Tang, J. Xu, S. Bi, and Y. Cai, “Variable Convolution and Pooling Convolutional Neural Network for Text Sentiment Classification,” IEEE Access, vol. 8, pp. 16174–16186, 2020, doi: 10.1109/access.2020.2966726.
J. Kim, J. Lee, E. Park, and J. Han, “A deep learning model for detecting mental illness from user content on social media,” Scientific Reports, vol. 10, no. 1, Jul. 2020, doi: 10.1038/s41598-020-68764-y.
R. Nareshkumar and K. Nimala, “An Exploration of Intelligent Deep Learning Models for Fine Grained Aspect-Based Opinion Mining,” 2022 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES), pp. 1–7, Jul. 2022, doi: 10.1109/icses55317.2022.9914094.
J. Kim, J. Lee, E. Park, and J. Han, “A deep learning model for detecting mental illness from user content on social media,” Scientific Reports, vol. 10, no. 1, Jul. 2020, doi: 10.1038/s41598-020-68764-y.
R. Nareshkumar and K. Nimala, “Interactive Deep Neural Network for Aspect-Level Sentiment Analysis,” 2023 International Conference on Artificial Intelligence and Knowledge Discovery in Concurrent Engineering (ICECONF), pp. 1–8, Jan. 2023, doi: 10.1109/iceconf57129.2023.10083812.
S. Akyol, M. Yildirim, and B. Alatas, “Multi-feature fusion and improved BO and IGWO metaheuristics based models for automatically diagnosing the sleep disorders from sleep sounds,” Computers in Biology and Medicine, vol. 157, p. 106768, May 2023, doi: 10.1016/j.compbiomed.2023.106768.
Y. Hou, H. Gao, Z. Wang, and C. Du, “Improved Grey Wolf Optimization Algorithm and Application,” Sensors, vol. 22, no. 10, p. 3810, May 2022, doi: 10.3390/s22103810.
H. A and B. S. P, “Deep Learning with Crested Porcupine Optimizer for Detection and Classification of Paddy Leaf Diseases for Sustainable Agriculture,” Journal of Machine and Computing, pp. 1018–1031, Oct. 2024, doi: 10.53759/7669/jmc202404095.
S. Nithya and S. U. -, “MOOC Dropout Prediction using FIAR-ANN Model based on Learner Behavioral Features,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 9, 2022, doi: 10.14569/ijacsa.2022.0130972.
P. V, U. S, S. B, R. V, M. Thangaraju, and U. M. P, “Advanced Explainable AI: Self Attention Deep Neural Network of Text Classification,” Journal of Machine and Computing, pp. 586–593, Jul. 2024, doi: 10.53759/7669/jmc202404056.
Acknowledgements
Author(s) thanks to Dr. Nirmalrani V for this research completion and support.
Funding
No funding was received to assist with the preparation of this manuscript.
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Availability of data and materials
Data sharing is not applicable to this article as no new data were created or analysed in this study.
Author information
Contributions
All authors have equal contribution in the paper and all authors have read and agreed to the published version of the manuscript.
Corresponding author
Nimala K
Department of Networking and Communications, School of Computing, College of Engineering and Technology (CET), SRM Institute of Science and Technology, Kattankulathur, Chennai, Tamil Nadu, India.
Open Access This article is licensed under a Creative Commons Attribution NoDerivs is a more restrictive license. It allows you to redistribute the material commercially or non-commercially but the user cannot make any changes whatsoever to the original, i.e. no derivatives of the original work. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
Cite this article
Nareshkumar R and Nimala K, “Depression Classification Using Bert Embedding Model on Social Media Posts”, Journal of Machine and Computing. doi: 10.53759/7669/jmc202505037.
