Tilbage til forskningsbiblioteket
Original publikation
Akustiske og naturligt-sproglige markører for bipolar lidelse: Et pilot, mHealth tværsnitsstudie
Studiet undersøger, om tale- og sproganalyse kan bruges til at vurdere symptomer hos personer med bipolar lidelse. Forskerne analyserer tale- og skriftmønstre ved hjælp af naturligt-sproglig og akustisk signalbehandling inden for en mobil sundhedsramme. Formålet er at støtte løbende vurdering af bipolar lidelse.
Titel og resumé er skrevet på dansk med AI ud fra den originale publikation. Originalteksten står nedenfor.
Crocamo C, Cioni RM, Canestro A, Nasti C, Palpella D, Piacenti S, Bartoccetti A, Re M, Simonetti V, Barattieri di San Pietro C, Bulgheroni M, Bartoli F, Carrà G. · JMIR formative research · 2025
AI-billedeCC BY 4.0 · © Cristina Crocamo, Riccardo Matteo Cioni, Aurelia Canestro, Christian Nasti, Dario Palpella, Susanna Piacenti, Alessandra Bartoccetti, Martina Re, Valentina Simonetti, Chiara Barattieri di San Pietro, Maria Bulgheroni, Francesco Bartoli, Giuseppe Carrà. Originally published in JMIR Formative Research
Teksten oversættes automatisk og kan indeholde fejl. Originalteksten på engelsk er den gældende. Oversættelsen er ikke medicinsk rådgivning.
Acoustic and Natural Language Markers for Bipolar Disorder: A Pilot, mHealth Cross-Sectional Study
Original dokumenttekst
Abstract
Background
Monitoring symptoms of bipolar disorder (BD) is a challenge faced by mental health services. Speech patterns are crucial in assessing the current experiences, emotions, and thought patterns of people with BD. Natural language processing (NLP) and acoustic signal processing may support ongoing BD assessment within a mobile health (mHealth) framework.
Objective
Using both acoustic and NLP-based features from the speech of people with BD, we built an app-based tool and tested its feasibility and performance to remotely assess the individual clinical status.
Methods
We carried out a pilot, observational study, sampling adults diagnosed with BD from the caseload of the Nord Milano Mental Health Trust (Italy) to explore the relationship between selected speech features and symptom severity and to test their potential to remotely assess mental health status. Symptom severity assessment was based on clinician ratings, using the Young Mania Rating Scale (YMRS) and Montgomery-Åsberg Depression Rating Scale (MADRS) for manic and depressive symptoms, respectively. Leveraging a digital health tool embedded in a mobile app, which records and processes speech, participants self-administered verbal performance tasks. Both NLP-based and acoustic features were extracted, testing associations with mood states and exploiting machine learning approaches based on random forest models.
Results
We included 32 subjects (mean [SD] age 49.6 [14.3] years; 50% [16/32] females) with a MADRS median (IQR) score of 13 (21) and a YMRS median (IQR) score of 5 (16). Participants freely managed the digital environment of the app, without perceiving it as intrusive and reporting an acceptable system usability level (average score 73.5, SD 19.7). Small-to-moderate correlations between speech features and symptom severity were uncovered, with sex-based differences in predictive capability. Higher latency time (ρ=0.152), increased silences (ρ=0.416), and vocal perturbations correlated with depressive symptomatology. Pressure of speech based on the mean intraword time (ρ=–0.343) and lower voice instability based on jitter-related parameters (ρ ranging from –0.19 to –0.27) were detected for manic symptoms. However, a higher contribution of NLP-based and conversational features, rather than acoustic features, was uncovered, especially for predictive models for depressive symptom severity (NLP-based: R2=0.25, mean squared error [MSE]=110.07, mean absolute error [MAE]=8.17; acoustics: R2=0.11, MSE=133.75, MAE=8.86; combined: R2=0.16; MSE=118.53, MAE=8.68).
Conclusions
Remotely collected speech patterns, including both linguistic and acoustic features, are associated with symptom severity levels and may help differentiate clinical conditions in individuals with BD during their mood state assessments. In the future, multimodal, smartphone-integrated digital ecological momentary assessments could serve as a powerful tool for clinical purposes, remotely complementing standard, in-person mental health evaluations.
Introduction
Bipolar disorder (BD) is a lifelong, episodic illness characterized by mood recurrences, including manic or hypomanic, depressive, and mixed episodes [1-3]. The burden associated with BD, affecting families, carers, and mental health care systems, is heavy [4]. Community services often struggle in delivering regular monitoring of BD treatment needs, resulting in relapses that seem difficult to predict [4-6].
Language disturbances are among the core symptoms of acute episodes in BD, since speech patterns are modulated by the emotional and neurophysiological status [78]. Therefore, language may play a key role in the assessment of an individual’s current experiences, emotions, thought patterns, and symptoms. While content analysis may reveal grandiosity associated with elevated mood, impulsivity, or changes in goal-directed activities, natural language may provide insights into mood fluctuations, cognitive processes, and behavioral patterns [9]. In particular, changes in the rate of speech are likely to indicate mood oscillations, including pressure of speech and increased verbosity during manic episodes [10] and poverty of speech and increased pause times during depressive episodes [11-13]. Clinicians are trained to recognize variations in language and voice, along with gestures and facial expressions, implicitly assessing both coherence and organization of speech and natural language features. However, this process is inevitably vulnerable to inconsistencies and biases.
Recent research in mental health and computer science has put forward computational approaches for speech analysis across a variety of mental disorders, proposing automated methods to assess and monitor the individual’s mental state through speech patterns [14-18]. Promising techniques in speech acoustic signal processing [10111719-21undefinedundefined], using mobile health (mHealth) technology, can bridge subjective and objective components across various stages, such as prediction of illness onset, diagnostic processes, assessment of severity, and forecast of treatment outcomes [22-25]. Indeed, natural language processing (NLP) techniques, exploring language resources (eg, lexical choices, syntax, and semantics) both qualitatively and quantitatively (eg, topic modeling, clustering, and classification), may produce deeper insights across different clinical conditions [926]. For example, observable linguistic traits (eg, increased use of both first-person pronouns and negative emotion expressions) can be identified among people with BD [23]. However, although linguistic features are informative, they are context-dependent and inferred according to word transcriptions [27]. Thus, speech analyses combining acoustic-dependent features (eg, speech prosody and voice quality) with NLP-based measures appear more promising in terms of model predictions, possibly providing a more accurate mental health assessment [232728].
Indeed, research has shown that acoustic features are markers of emotional states in BD [29], and that quantifiable speech differences can predict the scores of scales such as the Young Mania Rating Scale (YMRS) and the Montgomery-Åsberg Depression Rating Scale (MADRS) [1327]. On the other hand, recent evidence has shown how smartphone-based voice data [30] can enhance BD monitoring in real time, detecting possible mood changes [3132]. Thus, speech-based systems embedded in smartphones might be useful tools for complementary, continuous assessments of BD clinical states. We therefore built an app-based tool, jointly using acoustic and NLP-based features from the speech of people with BD who delivered a narrative, and carried out a pilot study aimed at testing its feasibility and performance to remotely assess the individual clinical status. Continuous, uninterrupted spoken accounts, as supplied by individuals, provided the unique opportunity to combine communication style information from an in-depth set of acoustic features and NLP-based scores as potential digital markers of symptom severity in speech. We rigorously chose to test the tool’s performance against standard psychometric assessments of mania and depression in order to explore its potential for remote, complementary assessments.
Methods
The report of this study adheres to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement (checklist presented in Multimedia Appendix 1) [33].
Study Design and Sampling Strategies
We conducted a pilot, cross-sectional study involving adult participants (aged 18 years or older) from the caseload of the Nord Milano Mental Health Trust (Italy). The Trust includes 2 psychiatric intensive care units, with a total of 27 beds, and also provides community mental health care for the same 280,000 inhabitants of the northern area of the Metropolitan City of Milan through 4 community mental health teams with multidisciplinary staff. The relevant catchment area comprises highly urbanized, both deprived and affluent, districts.
Inclusion criteria comprised a diagnosis of BD and the willingness to participate in the study. People with physical impairments affecting their acoustic capabilities were excluded. Based on inclusion and exclusion criteria, eligible individuals were identified among individuals consecutively admitted to the Trust. Then, they were approached by the research team, explaining the purpose of the study and, if any, potential risks.
Ethical Considerations
Recruitment efforts were carried out in accordance with ethical guidelines to ensure the well-being and safety of all participants. Study participants signed a written informed consent and were not compensated for their involvement. The study received ethical approval (protocol number 172‐17032023) from the local ethical committee. To maintain participant privacy and confidentiality, all study data were pseudonymized prior to analysis. No individual participants are identifiable in any images included in this manuscript or Multimedia Appendices.
Procedures
Acoustic data were retrieved by asking participants to self-administer verbal performance tasks through a mobile app on their smartphones (SPEAKapp; [34]). Clinical testing and app usage took place on the same day in the study setting (inpatient and outpatient services). Then, the System Usability Scale (SUS), a short 10-item questionnaire based on a 5-point Likert scale, was administered to assess the usability [35] of the app.
Verbal performance in terms of prose recall was based on the Babcock test [36], for which participants were asked to listen to a short story characterized by graphic and intense contents (eg, a death in a car crash) and then to repeat what she or he remembered from this narrative. This enabled to capture speech timing patterns based on sustained speech samples.
The app gathered participants’ verbal production by using the smartphone-integrated microphone, recording and processing participants’ speech by leveraging Google Speech-To-Text APIs [37] and Python libraries (eg, Parselmouth for the Praat software [38]). Recordings involved the use of one audio channel based on the participant’s voice in a controlled environment with minimal acoustic conditions. Both the raw audio data and the transcribed text content were processed to extract acoustic and NLP-based features from speech outputs. NLP and acoustic signal models were embedded in the backend part of the mobile app.
Measures
Consistent with recent evidence, we assumed speech as verbal behavior, the spoken output of the mental system underlying the language [39]. Through speech recognition, acoustic and linguistic features were extracted. Then, based on both NLP and acoustic features, we considered a multidimensional framework in order to generate appropriate discriminative information for the potential use of speech patterns as digital markers in BD [2731]. A full description of selected features is provided in Table S1 in Multimedia Appendix 2.
NLP-Based, Semantic, and Conversational Indices
NLP-based scores were computed according to distributional semantic models, encompassing vectorial representations for the meaning of words in a multi-dimensional space.
Standard linguistic scores included both the number of words, indicative of poverty of speech, and the number of words produced that matched the story text. On the other hand, novel NLP-based scores integrated mean intraword time, estimating the average time taken to articulate or pronounce subsequent words, as an indicator of processing speed, as well as word mover’s distance (WMD), capturing both lexical overlap and semantic similarity. In particular, WMD was estimated as the minimum cumulative distance between words required to exactly match the point cloud of the text of the full correct story (ie, the content distance between the full correct story and the story narrative produced by the participant), thus incorporating the semantic similarity between individual word pairs into the word distance metric [40]. In addition, latency time was calculated as a novel NLP-based score, taking into account the delay between the initiation of a spoken utterance or action and the production of its intended outcome or response when starting the task (ie, the first word).
Additional objective information was extracted from speech data. These quantitative measures included (1) speech duration, (2) speaking time (ie, phonation), (3) silence, (4) ratios of speaking time to speech duration as well as of silence to speaking time, and (5) speech rate.
Acoustic Indices From Vocal Signals (Prosodic Cues Indices)
Measures for prosodic cues (acoustic indices quantifying how people talk during conversations) were based on the signal’s frequency and energy or amplitude. These were assumed to contribute to conveying paralinguistic meaning [41]. Based on nontextual data, acoustic components of speech were defined as the key phonetic elements, that is, objectively and reproducibly quantified speech sounds [2742]. Fundamental frequency (F0) was measured by the frequency of phonation [43]. The short-term instability of the vibration of the vocal cords during phonation (ie, jitter-related indices) was also extracted (Table S1 in Multimedia Appendix 2). Higher jitter values indicated speech patterns likely characterized by irregularities or hesitations, thus mirroring potential underlying psychological distress or emotional instability. Furthermore, microperturbations of the ampleness of the signal (ie, how variable acoustic peaks refer to the period-to-period variability of the signal peak-to-peak amplitude) were identified as small fluctuations in the intensity of vocal sound waves by shimmer-related measures, with higher values indicating greater variability or instability, while lower ones suggesting more stable vocal intensity (ie, smoother and more regular speech production).
Since both periodic and nonperiodic sound waves may characterize the voice, the mean harmonics-to-noise ratio was used to measure the relationship between harmonic and nonharmonic voice elements. Noisier, more raucous voices (ie, not smooth or clear) were expected to show lower harmonics-to-noise ratios, indicating vocal cord tension or irritation, possibly suggesting emotional distress.
Psychometric Measures
Diagnosis of BD was confirmed by the Structured Clinical Interview for DSM-5 (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; SCID-5). Based on clinician-rated assessments, depressive symptom severity was measured by the MADRS [44], while YMRS was used to assess manic symptoms [45]. Scores ranged from 0 to 60 for both MADRS [44] and YMRS [45]. In addition, cutoffs for severe mood symptoms were either a YMRS score ≥20 [4647] or a MADRS score ≥19 [48].
Statistical Analyses
First, we summarized participants’ characteristics, providing standard statistics for continuous and categorical variables. For both MADRS and YMRS, continuous scores were used. However, a supplementary analysis was performed based on clinically meaningful thresholds for symptom severity. A bivariate analysis was then carried out to measure the strength of the potential association between speech indices and psychometric measures. Features’ summary statistics were plotted, and correlation coefficients (Pearson and Spearman, according to assumptions on data distribution, eg, normality) were estimated. Color gradient heat plots were also generated for data visualization. Taking into account potential sex differences in speech acoustic indices [49-51], subgroup analyses were performed. Statistical significance was set at P<.05.
Second, based on state-of-the-art algorithms, NLP and acoustic features extracted from natural language and audio streams (Table S1 in Multimedia Appendix 2) were used to train machine-learning models to detect depressive and manic states by means of scores from MADRS and YMRS. Data were randomly split using a 5-fold nested cross-validation approach for training and testing in order to provide an unbiased evaluation of the model’s performance. In particular, random forest (RF) models, with the potential to handle both linear and nonlinear relationships between features and the target variable, were implemented. The supervised learning algorithm, with no assumptions about the distribution of the target variable, was based on the ensemble learning method of different decision trees, whose predictions were aggregated using the scikit-learn library in Python. Exploiting the bagging techniques, building multiple decision trees, RF contributed to minimizing overfitting issues by randomizing the feature selection during each tree split. This was assumed to reduce sensitivity to noise and to make decision trees less correlated through the use of a unique subset of the initial data for every base model. Moreover, we deemed features scaling unnecessary due to both the properties of the RF model and the performance metrics of comparisons. Relevant models were trained to test final performance by metrics (ie, mean squared error [MSE], mean absolute error [MAE], and R-squared [R2]). These tested overall performance, even controlling for sex. Shapley Additive Explanations values, showing features’ impact, were plotted. Data were analyzed using Stata release 18 and Python (version 3.10.9).
Results
Sample Characteristics
We included 32 subjects with BD (mean age 49.6, SD 14.3 years; 50% [16/32] females). The mean (SD) age at onset was 24.4 (10) years. As a whole, participants experienced more manic (median 4, IQR 8) than depressive episodes (median 2, IQR 5). About 40% (12/32) of participants reported a previous mood episode within 1 year before study enrollment. The MADRS median (IQR) score was 13 (21), while the YMRS median (IQR) score was 5 (16). Considering the app usage, participants reported high SUS scores on average (mean 73.5, SD 19.7). Demographic and clinical details are fully provided in Table 1.
Associations Between Symptom Severity and Speech Features
For descriptive purposes, NLP-based, conversational, and acoustic features are summarized in Figures S1A-S1D and S2A-S2D in Multimedia Appendix 3 by depressive and manic symptom severity, respectively.
In particular, grouping data into 2 categories (Multimedia Appendix 3), statistically significant differences by depressive symptoms’ severity were found for many NLP-based and conversational-like measures, including word number, phonation (also as percentage over the speech duration), and mean intraword time. Correlation analyses, based on Spearman nonparametric analysis of symptom severity continuous scores, are displayed in Figures1A-C 2A-C. These showed that both the total number of words and the length of phonation, as well as the related percentage out of segment duration, were negatively correlated (coefficients=−0.35, −0.32, and −0.42) to depressive symptoms (Figure 1A). Consistent results were observed for the ratio between silence and phonation (coefficient=0.42), as well as for mean intraword time, which was positively correlated to depressive (coefficient=0.53) and negatively to manic (coefficient=−0.34) symptoms. Among items for depressive symptoms assessment, this correlation was particularly clear between acoustic features and suicidal thoughts (coefficients ranging from 0.18 to 0.51). In addition, latency time also showed a moderate, though obviously opposite, correlation with manic and depressive symptoms, respectively (coefficients=−0.28 and 0.15).
Subgroup analyses for NLP-based and conversational features revealed more pronounced relationships in females (Figure 1C) as compared with males (Figure 1B), showing a high correlation between depressive symptoms and mean intraword time (coefficient=0.75), phonation percentage (coefficient=−0.56), and, consequently, the silence-phonation ratio (coefficient=0.56). Similarly, latency time was negatively correlated to manic symptoms among females (coefficient=−0.60).
On the other hand, a small positive correlation was uncovered between depressive symptoms and higher values of instability in speech patterns (jitter-related indices, with coefficients ranging from 0.10 to 0.16). In contrast, small-to-moderate negative correlations were observed between manic symptoms and lower values of instability (jitter-related indices, with coefficients ranging from −0.19 to −0.27). Small estimates were found for F0, respectively (coefficient=0.16 and −0.18; Figure 2A). Except for shimmer_apq11 (manic symptoms coefficient=−0.22), we did not find any substantial relationship between shimmer-related indices (describing stable and unstable vocal intensity and speech production) and symptomatology.
Subgroup analyses suggested a role for sex also in influencing acoustic features. In particular, we found deeper connections in males as compared with females, especially in terms of F0 and jitter-related indices (Figure 2B and C).
Predictive Models From Speech Features
Considering depressive symptoms, performance metrics showed a contribution of NLP-based and conversational features higher than what was attributable to acoustic ones (Table 2). In particular, mean intraword time, silence-phonation ratio, ppq5 jitter (ie, perturbations in F0), WMD, and percentage of phonation over duration all ranked high in terms of relative importance.
Including sex into the analysis, a differential contribution of various features (NLP-based and conversational vs acoustics) to the predictive models for depressive (Figure 3A) and manic (Figure 3B) symptoms can be found. However, as for manic symptoms, although a relative contribution of different NLP-based and acoustic (eg, F0 SD) features was recorded, we could not find any reliable estimates for the relevant model, even including sex. Table 2 shows detailed estimated performance metrics for testing for the trained RF regressors, even controlling for sex.
Discussion
Main Findings
This study aimed at piloting the simultaneous use of speech acoustics, as well as natural language features, to glean insights into BD depressive and manic symptoms. Our findings corroborate evidence on the relationships between symptom severity and speech features, supporting the potential predictive role for clinical purposes of digital mental health applications, embedded in a mHealth integrated system.
First, the speech of participants with BD showed that vocal perturbations (eg, higher instability and hesitations considering voice quality), latency time, and increased silences and pauses over time speaking all correlated to depressive symptoms. Consistently, increased depressive symptoms resulted in NLP-based features such as a smaller number of words and longer mean intraword time, with lower pressure of speech. In our exploratory study, this relationship was particularly clear among females. This effect was corroborated by the predictive model, showing a contribution of NLP-based and conversational features higher than for acoustic ones. This finding aligns with prior evidence, advocating that text-based features contribute more to model accuracy than audio parameters [18]. However, also the latter component (ie, fundamental frequency, jitter- and shimmer-related indices) deserves a careful assessment, since our findings show that these indices might have an impact at least among males to predict future episodes. Indeed, recent evidence from healthy populations sheds light on sex differences in speech markers (eg, prosodic features) with different acoustic cues conveying various emotions [50]. A combination of inherent biological dissimilarities, socialization processes, influences of the social environment, and cultural expectations might contribute to these differences in both expression and perception of related emotional prosody [5253]. Moreover, individuals may modulate their speech to align with the dominant pitch range within a specific linguistic community [54], and similar modulation may occur in conversational dialogues versus monologues and in spontaneous versus elicited speech. Thus, this criterion should be taken into account when designing apps with speech recognition and processing tasks for people with BD [31].
Second, voice instability and hesitations, as well as mean intraword time, were negatively correlated to manic symptoms. However, the interpretation of the relationship between manic features and vocal abnormalities is not straightforward. Mixed findings emerged on the relationships between speech features and manic symptoms, preventing us from supporting our original hypothesis. One plausible explanation may stem from the sample characteristics. Indeed, our participants were more likely to report depressive symptoms, and just a few had severe manic features.
However, the overall moderate correlations between speech markers and symptom severity were consistent with previous work that used speech smartphone data to discriminate between different mood states [2021]. It has been argued that speech features may be useful to detect a trait [55] rather than a state [56] in BD. However, alterations in voice perturbations have been observed when assessing vocal markers of suicidal ideation [57], and this makes further research for vocal features reasonable, at least for depressive conditions.
Smartphone-Based Applications
Consistent with previous research on smartphone-based applications designed to record and analyze speech patterns in real time, our findings emphasize the feasibility of a simple, yet clinically useful, application of digital technology [13]. In particular, we developed the frontend of the app as a basic digital environment, freely managed by participants on their own smartphones. Participants reported a high level of engagement with the tool, showing an acceptable system usability level as assessed by SUS [35], without perceiving intrusiveness of the recording of both elicited and spontaneous conversations.
Comparisons of the vocal performance of people with BD with unaffected relatives and healthy controls have shown a clear speech “fingerprint” of the clinical condition [58], suggesting the utility of multilevel inputs [59]. However, there is also the need for a wider understanding of fluctuations in symptom severity and mood states in this population [60]. The major strength of our study consists in the usefulness of different speech data (eg, linguistic, conversational, acoustics) to differentially identify symptoms of BD. Thus, for relapse prevention purposes, future research should possibly explore systems combining smartphone-based generated objective acoustics data with additional information, such as from facial expressions and gestures [61]. This would ultimately improve BD state prediction, even considering classification tasks [2162-64undefinedundefined].
Clinical Implications: Interdisciplinary Perspective
This pilot study represents a step forward in the identification and utilization of digital biomarkers for BD from natural language and audio streams, with implications for personalized mental health care and early intervention strategies. Our approach holds promise for complementary, remote assessments enhancing depressive and partly manic states prediction by exploiting participants’ speech. This would have significant implications, especially considering BD fluctuating symptomatology. Nonetheless, leveraging live speech recordings as a predictive tool, repeated assessments are needed to identify individuals at risk of transitioning to depressive and manic states.
Despite promising findings from automated assessments, mental health care heavily relies on participant interviews, yet with often subjective reports, cognitive limitations, and stigma [18]. Integrated systems, aiming at taking advantage of candidate digital markers from speech recognition, would possibly boost a care approach in which digital technology enhances, but does not replace, existing models from clinical assessment [30]. Indeed, automated assessment does not inherently lead to adherence and engagement of individuals with BD [65].
Finally, clinical, hypothesis-driven research on BD should not be dismissed, since algorithms may not be considered a black-box replacement for traditional data modeling, but they rather integrate with other systems, embedding a substantial clinical validation [6667].
Limitations and Future Directions
We should acknowledge some limitations of this study. Analyzing speech and natural language in individuals with BD implies a challenge due to the nature of the disorder and to ethical considerations.
First, properties of chosen machine-learning models may hamper identification of unknown patterns based on values that fall outside the training set. Effective NLP and supervised learning models may require high-quality, annotated datasets. While exploratory in nature, the study’s limited sample size may have constrained the model’s statistical power and the ability to capture the full complexity of the underlying data distribution, thereby hindering meaningful subgroup comparisons. Our preliminary findings should be replicated and extended in a larger, more diverse sample of people with BD to mitigate the risks associated with overfitting. Furthermore, future research should address classification approaches based on severity thresholds for both MADRS and YMRS. Accordingly, there is potential for alternative modeling approaches for regression tasks (eg, splines) that might be implemented in the future. While still considering the number of predictors, these may possibly enable a better understanding of the nature of the existing relationships and nonlinear patterns.
Consistently, the lack of standardized (linguistic and acoustic) markers represents a barrier when studying relationships with mood states. Indeed, the model may still learn to overfit to irrelevant or noisy features the data may contain, especially if they are informative in the training set by chance.
Furthermore, the speaker’s identity may show a possible confounding role in a between-subject design. Therefore, studies with a longitudinal design (ie, within-subjects) should be recommended, deploying Ecological Momentary Assessment approaches [2468]. In addition, speech patterns may generate misinterpretations if individual cultural and linguistic factors are not accounted for [69]. Similarly, speech during manic episodes may exhibit circumstantiality or tangentiality, where individuals provide excessive details or veer off-topic. Rapid speech, tangential thinking, or unconventional language use pose challenges for automatic speech recognition systems. Analyzing such complex speech patterns requires a deep evaluation of language and context, achieving appropriate understanding of an individual’s usual way of communicating in order to distinguish changes associated with BD episodes.
Furthermore, in our study, speech features were averaged over relevant duration, thus constraining the role of temporal variations across related measures in predicting symptom severity. Future research should endeavor to integrate dynamic aspects of speech on mood states transitioning.
Finally, other clinical variables, not investigated in our sample, are likely to influence the individual’s speech. For instance, it should be noted that anxiety and anxious distress, often co-occurring with bipolar depression [70], may significantly influence speech features [71], as well as medication prescribed [72-74] and drug or alcohol comorbid conditions [75].
Conclusions
Speech patterns, underlying both linguistic and acoustic features, are able to yield quantifiable differences, thus embodying digital markers of symptom severity. Multimodal, smartphone-integrated digital assessments could serve as powerful tools for clinical purposes to remotely complement standard mental health evaluations, potentially contributing to distinguish clinical conditions in people with BD. Feasibility of similar systems seems promising, though issues related to privacy, intrusiveness, and clinical therapeutic relationships should be carefully considered.
Authors’ Contributions: CC, FB, and GC handled conceptualization. RMC, AC, CN, DP, SP, AB, and MR performed investigation. CC, FB, VS, CB, and MB contributed to methodology. VS and MB assisted with software. CC conducted formal analysis. GC performed supervision. CC contributed to writing – original draft. CC, RMC, AC, CN, DP, SP, AB, MR, VS, CB, MB, FB, and GC contributed to writing – review and editing.
Data Availability: The datasets including audio data streams supporting the conclusions of this article are not publicly available as the original source has not granted permission to share that information but may be available from the corresponding author on reasonable request.
Conflicts of Interest: None declared.
Referencer citeret i publikationen
- 1. BartoliF CrocamoC CarràG Clinical correlates of DSM-5 mixed features in bipolar disorder: a meta-analysisJ Affect DisordNov12020276234240doi10.1016/j.jad.2020.07.035Medline32697704
- 2. GrandeI BerkM BirmaherB VietaE Bipolar disorderLancetApr920163871002715611572doi10.1016/S0140-6736(15)00241-XMedline26388529
- 3. McIntyreRS BerkM BrietzkeE et al Bipolar disordersLancetDec520203961026518411856doi10.1016/S0140-6736(20)31544-0Medline33278937
- 4. KarambelasGJ FiliaK ByrneLK AllottKA JayasingheA CottonSM A systematic review comparing caregiver burden and psychological functioning in caregivers of individuals with schizophrenia spectrum disorders and bipolar disordersBMC PsychiatryJun232022221422doi10.1186/s12888-022-04069-wMedline35733174PMC9219207
- 5. FajutraoL LocklearJ PriaulxJ HeyesA A systematic review of the evidence of the burden of bipolar disorder in EuropeClin Pract Epidemiol Ment Health0123200953doi10.1186/1745-0179-5-3Medline19166608PMC2646705
- 6. OgilvieAD MorantN GoodwinGM The burden on informal caregivers of people with bipolar disorderBipolar Disord20057 Suppl 1Suppl 12532doi10.1111/j.1399-5618.2005.00191.xMedline15762866
- 7. GoodwinFK JamisonKR Manic-Depressive Illness: Bipolar Disorders and Recurrent DepressionOxford University Press, USA2007ISBN9780195135794
- 8. KaramZN ProvostEM SinghS et al Ecologically valid long-term mood monitoring of individuals with bipolar disorder using speechProc IEEE Int Conf Acoust Speech Signal Process052014201448584862doi10.1109/ICASSP.2014.6854525Medline27630535PMC5019119
- 9. HarveyD LobbanF RaysonP WarnerA JonesS Natural language processing methods and bipolar disorder: scoping reviewJMIR Ment HealthApr22202294e35928doi10.2196/35928Medline35451984PMC9077496
- 10. BirnbaumML AbramiA HeisigS et al Acoustic and facial features from clinical interviews for machine learning-based psychiatric diagnosis: algorithm developmentJMIR Ment Health0124202291e24699doi10.2196/24699Medline35072648PMC8822433
- 11. GideonJ ProvostEM McInnisM Mood state prediction from speech of varying acoustic quality for individuals with bipolar disorderProc IEEE Int Conf Acoust Speech Signal ProcessMar2016201623592363doi10.1109/ICASSP.2016.7472099Medline27570493PMC4995442
- 12. GuidiA SchoentgenJ BertschyG GentiliC ScilingoEP VanelloN Features of vocal frequency contour and speech rhythm in bipolar disorderBiomed Signal Process ControlAug2017372331doi10.1016/j.bspc.2017.01.017
- 13. MaxhuniA Muñoz-MeléndezA OsmaniV PerezH MayoraO MoralesEF Classification of bipolar disorder episodes based on analysis of voice and motor activity of patientsPervasive Mob ComputSep2016315066doi10.1016/j.pmcj.2016.01.008
- 14. MattonK McInnisMG ProvostEM Into the wild: transitioning from recognizing mood in clinical interactions to personal conversations for individuals with bipolar disorderPresented atInterspeech 2019Sep 15-19, 201914381442Graz, Austria2019URLhttps://www.isca-archive.org/interspeech_2019doi10.21437/Interspeech.2019-2698
- 15. ArevianAC BoneD MalandrakisN et al Clinical state tracking in serious mental illness through computational analysis of speechPLoS ONE2020151e0225695doi10.1371/journal.pone.0225695Medline31940347PMC6961853
- 16. GirardJM VailAK LiebenthalE et al Computational analysis of spoken language in acute psychosis and maniaSchizophr Res07202224597115doi10.1016/j.schres.2021.06.040Medline34456131PMC8881587
- 17. LowDM BentleyKH GhoshSS Automated assessment of psychiatric disorders using speech: a systematic reviewLaryngoscope Investig OtolaryngolFeb20205196116doi10.1002/lio2.354Medline32128436PMC7042657
- 18. MalgaroliM HullTD ZechJM AlthoffT Natural language processing for mental health interventions: a systematic review and research frameworkTransl PsychiatryOct62023131309doi10.1038/s41398-023-02592-2Medline37798296PMC10556019
- 19. CumminsN BairdA SchullerBW Speech analysis for health: current state-of-the-art and the increasing impact of deep learningMethodsDec120181514154doi10.1016/j.ymeth.2018.07.007Medline30099083
- 20. Faurholt-JepsenM BuskJ FrostM et al Voice analysis as an objective state marker in bipolar disorderTransl Psychiatry0719201667e856doi10.1038/tp.2016.123Medline27434490PMC5545710
- 21. Faurholt-JepsenM RohaniDA BuskJ et al Discriminating between patients with unipolar disorder, bipolar disorder, and healthy control individuals based on voice features collected from naturalistic smartphone callsActa Psychiatr ScandMar20221453255267doi10.1111/acps.13391Medline34923626
- 22. DausH BloecherT EgelerR De KlerkR StorkW BackenstrassM Development of an emotion-sensitive mHealth approach for mood-state recognition in bipolar disorderJMIR Ment Health073202077e14267doi10.2196/14267Medline32618577PMC7367525
- 23. DikaiosK RempelS DumpalaSH OoreS KiefteM UherR Applications of speech analysis in psychiatryHarv Rev Psychiatry2023311113doi10.1097/HRP.0000000000000356Medline36608078
- 24. DunsterGP SwendsenJ MerikangasKR Real-time mobile monitoring of bipolar disorder: a review of evidence and future directionsNeuropsychopharmacology012021461197208doi10.1038/s41386-020-00830-5Medline32919408PMC7688933
- 25. MarzanoL BardillA FieldsB et al The application of mHealth to mental health: opportunities and challengesLancet PsychiatryOct2015210942948doi10.1016/S2215-0366(15)00268-0Medline26462228
- 26. Le GlazA HaralambousY Kim-DuforDH et al Machine learning and natural language processing in mental health: systematic reviewJ Med Internet Res0542021235e15708doi10.2196/15708Medline33944788PMC8132982
- 27. FarrúsM Codina-FilbàJ EscuderoJ Acoustic and prosodic information for home monitoring of bipolar disorderHealth Informatics J20212711460458220972755doi10.1177/1460458220972755Medline33438502
- 28. GongY PoellabauerC Topic modeling based multi-modal depression detectionPresented atMM ’17Oct 23-27, 20172017Mountain View California USAOct232017URLhttps://dl.acm.org/doi/proceedings/10.1145/3133944doi10.1145/3133944.3133945
- 29. MuaremiA GravenhorstF GrünerblA ArnrichB TrösterG Assessing bipolar episodes using speech cues derived from phone callsLect Notes Inst Comput Sci Soc Inform Telecommun Eng2014103114doi10.1007/978-3-319-11564-1_11
- 30. BondRR MulvennaMD PottsC O’NeillS EnnisE TorousJ Digital transformation of mental health servicesNpj Ment Health ResAug2220232113doi10.1038/s44184-023-00033-yMedline38609479PMC10955947
- 31. FlanaganO ChanA RoopP SundramF Using acoustic speech patterns from smartphones to investigate mood disorders: scoping reviewJMIR Mhealth UhealthSep17202199e24352doi10.2196/24352Medline34533465PMC8486998
- 32. de OliveiraL PortugalLCL PereiraM et al Predicting bipolar disorder risk factors in distressed young adults from patterns of brain activation to reward: a machine learning approachBiol Psychiatry Cogn Neurosci NeuroimagingAug201948726733doi10.1016/j.bpsc.2019.04.005Medline31201147PMC6682607
- 33. von ElmE AltmanDG EggerM et al Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studiesBMJOct2020073357624806808doi10.1136/bmj.39335.541782.ADMedline17947786PMC2034723
- 34.Ab.acus SrlURLhttps://www.ab-acus.eu/index.php/portfolio-items/speakapp/Accessed19-03-2024
- 35. HyzyM BondR MulvennaM et al System Usability Scale benchmarking for digital health apps: meta-analysisJMIR Mhealth UhealthAug182022108e37290doi10.2196/37290Medline35980732PMC9437782
- 36.Italian standardization and classification of neuropsychological tests. The Italian Group on the Neuropsychological Study of AgingItal J Neurol SciDec1987Suppl 81-1201120Medline3330072
- 37.Google speech-to-text apisGoogle CloudURLhttps://cloud.google.com/speech-to-textAccessed09-04-2025
- 38. JadoulY ThompsonB de BoerB Introducing Parselmouth: a Python interface to PraatJ PhonNov201871115doi10.1016/j.wocn.2018.07.001
- 39. de BoerJN BrederooSG VoppelAE SommerIEC Anomalies in language as a biomarker for schizophreniaCurr Opin Psychiatry052020333212218doi10.1097/YCO.0000000000000595Medline32049766
- 40. KusnerM SunY KolkinN WeinbergerK From word embeddings to document distancesPresented atProceedings of the 32nd International Conference on Machine Learning, PMLRJul 5-7, 2015957966Lille, France2015
- 41. NadeuM PrietoP Pitch range, gestural information, and perceived politeness in CatalanJ PragmatFeb2011433841854doi10.1016/j.pragma.2010.09.015
- 42. SmithSW Digital Signal Processing: A Practical Guide for Engineers and ScientistsCalifornia Technical Publishing2002978doi10.1016/B978-0-7506-7444-7.X5036-5ISBN978-0-7506-7444-7
- 43. VerveridisD KotropoulosC Emotional speech recognition: resources, features, and methodsSpeech CommunSep200648911621181doi10.1016/j.specom.2006.04.003
- 44. MontgomerySA AsbergM A new depression scale designed to be sensitive to changeBr J PsychiatryApr1979134382389doi10.1192/bjp.134.4.382Medline444788
- 45. YoungRC BiggsJT ZieglerVE MeyerDA A rating scale for mania: reliability, validity and sensitivityBr J PsychiatryNov1978133429435doi10.1192/bjp.133.5.429Medline728692
- 46. LukasiewiczM GerardS BesnardA et al Young Mania Rating Scale: how to interpret the numbers? Determination of a severity threshold and of the minimal clinically significant difference in the EMBLEM cohortInt J Methods Psychiatr ResMar20132214658doi10.1002/mpr.1379Medline23526724PMC6878321
- 47. SamaraMT LevineSZ LeuchtS Linkage of Young Mania Rating Scale to Clinical Global Impression Scale to enhance utility in clinical practice and research trialsPharmacopsychiatry0120235611824doi10.1055/a-1841-6672Medline35896419
- 48. ThaseME HarringtonA CalabreseJ MontgomeryS NiuX PatelMD Evaluation of MADRS severity thresholds in patients with bipolar depressionJ Affect Disord05120212865863doi10.1016/j.jad.2021.02.043Medline33677183
- 49. EichhornJT KentRD AustinD VorperianHK Effects of aging on vocal fundamental frequency and vowel formants in men and womenJ VoiceSep2018325644doi10.1016/j.jvoice.2017.08.003Medline28864082PMC5832520
- 50. ErtürkA GürsesE Kulak KayıkcıME Sex related differences in the perception and production of emotional prosody in adultsPsychol ResMar2024882449457doi10.1007/s00426-023-01865-1Medline37542581
- 51. MendozaE ValenciaN MuñozJ TrujilloH Differences in voice quality between men and women: use of the long-term average spectrum (LTAS)J VoiceMar19961015966doi10.1016/s0892-1997(96)80019-1Medline8653179
- 52. ChaplinTM Gender and emotion expression: a developmental contextual perspectiveEmot Rev012015711421doi10.1177/1754073914544408Medline26089983PMC4469291
- 53. LinY DingH ZhangY Gender differences in identifying facial, prosodic, and semantic emotions show category- and channel-specific effects mediated by encoder’s genderJ Speech Lang Hear ResAug9202164829412955doi10.1044/2021_JSLHR-20-00553Medline34310173
- 54. AungT PutsD Voice pitch: a window into the communication of social powerCurr Opin PsycholJun202033154161doi10.1016/j.copsyc.2019.07.028Medline31445439
- 55. ZhangJ PanZ GuiC et al Analysis on speech signal features of manic patientsJ Psychiatr ResMar2018985963doi10.1016/j.jpsychires.2017.12.012Medline29291581
- 56. GuidiA SalviS OttavianoM et al Smartphone application for the analysis of prosodic features in running speech with a focus on bipolar disorders: system performance evaluation and case studySensors (Basel)Nov6201515112807028087doi10.3390/s151128070Medline26561811PMC4701269
- 57. OzdasA ShiaviRG SilvermanSE SilvermanMK WilkesDM Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal riskIEEE Trans Biomed EngSep200451915301540doi10.1109/TBME.2004.827544Medline15376501
- 58. Faurholt-JepsenM RohaniDA BuskJ VinbergM BardramJE KessingLV Voice analyses using smartphone-based data in patients with bipolar disorder, unaffected relatives and healthy control individuals, and during different affective statesInt J Bipolar DisordDec120219138doi10.1186/s40345-021-00243-3Medline34850296PMC8632566
- 59. TorousJ BucciS BellIH et al The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual realityWorld PsychiatryOct2021203318335doi10.1002/wps.20883Medline34505369PMC8429349
- 60. BalcombeL De LeoD Digital mental health challenges and the horizon ahead for solutionsJMIR Ment HealthMar29202183e26811doi10.2196/26811Medline33779570PMC8077937
- 61. SoenksenLR MaY ZengC et al Integrated multimodal artificial intelligence framework for healthcare applicationsNPJ Digit MedSep20202251149doi10.1038/s41746-022-00689-4Medline36127417PMC9489871
- 62. GrünerblA MuaremiA OsmaniV et al Smartphone-based recognition of states and state changes in bipolar disorder patientsIEEE J Biomed Health Inform012015191140148doi10.1109/JBHI.2014.2343154Medline25073181
- 63. OsmaniV Smartphones in mental health: detecting depressive and manic episodesIEEE Pervasive Comput20151431013doi10.1109/MPRV.2015.54
- 64. CohenJ RichterV NeumannM et al A multimodal dialog approach to mental state characterization in clinically depressed, anxious, and suicidal populationsFront Psychol2023141135469doi10.3389/fpsyg.2023.1135469Medline37767217PMC10520716
- 65. OrF TorousJ OnnelaJP High potential but limited evidence: using voice data from smartphones to monitor and diagnose mood disordersPsychiatr Rehabil JSep2017403320324doi10.1037/prj0000279Medline28891659
- 66. Garcia-CejaE RieglerM NordgreenT JakobsenP OedegaardKJ TørresenJ Mental health monitoring with multimodal sensing and machine learning: a surveyPervasive Mob ComputDec201851126doi10.1016/j.pmcj.2018.09.003
- 67. McCoyLG BrennaCTA ChenSS VoldK DasS Believing in black boxes: machine learning for healthcare does not need explainability to be evidence-basedJ Clin EpidemiolFeb2022142252257doi10.1016/j.jclinepi.2021.11.001Medline34748907
- 68. YerushalmiM SixsmithA Pollock StarA KingDB O’RourkeN Ecological momentary assessment of bipolar disorder symptoms and partner affect: longitudinal pilot studyJMIR Form ResSep2202159e30472doi10.2196/30472Medline34473069PMC8446838
- 69. ClarkEL EastonC VerdonS The impact of linguistic bias upon speech-language pathologists’ attitudes towards non-standard dialects of EnglishClin Linguist PhonJun32021356542559doi10.1080/02699206.2020.1803405Medline32781853
- 70. BartoliF BachiB CalloviniT et al Anxious distress in people with major depressive episodes: a cross-sectional analysis of clinical correlatesCNS SpectrFeb20242914953doi10.1017/S1092852923002377Medline37489522
- 71. MalgaroliM HullTD CalderonA SimonNM Linguistic markers of anxiety and depression in somatic symptom and related disorders: observational study of a digital interventionJ Affect Disord0512024352133137doi10.1016/j.jad.2024.02.012Medline38336165PMC10947071
- 72. BartoliF CrocamoC ClericiM CarràG Allopurinol as add-on treatment for mania symptoms in bipolar disorder: systematic review and meta-analysis of randomised controlled trialsBr J Psychiatry01201721011015doi10.1192/bjp.bp.115.180281Medline27856422
- 73. BartoliF CavaleriD BachiB et al Repurposed drugs as adjunctive treatments for mania and bipolar depression: a meta-review and critical appraisal of meta-analyses of randomized placebo-controlled trialsJ Psychiatr ResNov2021143230238doi10.1016/j.jpsychires.2021.09.018Medline34509090
- 74. BartoliF CavaleriD NastiC et al Long-acting injectable antipsychotics for the treatment of bipolar disorder: evidence from mirror-image studiesTher Adv Psychopharmacol20231320451253231163682doi10.1177/20451253231163682Medline36994116PMC10041584
- 75. CarràG ScioliR MontiMC MarinoniA Severity profiles of substance-abusing patients in Italian community addiction facilities: influence of psychiatric concurrent disordersEur Addict Res200612296101doi10.1159/000090429Medline16543745
Til din tryghed. MitLivMed er et fællesskab. Vi yder ikke krisestøtte eller medicinsk rådgivning og erstatter ikke din behandler. Fortællinger og mestringsforslag er medlemmernes egne erfaringer. Er du i krise? Find akut hjælp her.