Joint Representation Learning from Speech Foundation Models and Acoustic Features for Multilingual Depression Detection

Automated Depression Detection (ADD) from speech aims to estimate an individual's depression-related attributes using artificial intelligence applied to spoken signals. However, existing speech-based ADD methods rely on handcrafted acoustic features and high-resource language, which fail to capture clinically grounded speech cues and limit their effectiveness in low-resource multilingual settings. In addition, the integration of these features to overcome individual heterogeneity remains a significant challenge. In this regard, we propose a novel, joint representation learning framework specifically designed to support multilingual ADD. The design of simple yet effective cross-attention mechanisms to fuse the embeddings of speech foundation models with acoustic features offers the ability to enhance contextual information. Notably, this study introduces a new Indic-Bengali dataset on depression speech, collected through standardized and comprehensive elicitation tasks, with diagnostic labels provided by a psychiatrist. Using the proposed framework, extensive subject-independent experiments were conducted on the DAIC-WOZ, EDAIC, Androids corpus, and our Indic-Bengali dataset. We demonstrate that our method outperforms existing state-of-the-art speech-based ADD approaches on classification tasks, while also exhibiting improved cross-corpus transferability in multilingual settings. Finally, we believe that the proposed dataset and framework may support future research on speech-based depression assessment.