Abstract

Background

Digital mental health interventions (DMHIs) to monitor and improve the health of people with psychosis or bipolar disorder show promise; however, user engagement is variable, and integrated clinical use is low.

Objective

This prospectively registered systematic review examined barriers and facilitators of clinician and patient engagement with DMHIs, to inform implementation within real-world settings.

Methods

A systematic search of 7 databases identified empirical studies reporting qualitative or quantitative data about factors affecting staff or patient engagement with DMHIs aiming to monitor or improve the mental or physical health of people with psychosis or bipolar disorder. The Consolidated Framework for Implementation Research was used to synthesize data on barriers and facilitators, following a best-fit framework synthesis approach.

Results

The review included 175 papers (150 studies; 11,446 participants) describing randomized controlled trials; surveys; qualitative interviews; and usability, cohort, and case studies. Samples included people with schizophrenia spectrum psychosis (98/150, 65.3% of studies), bipolar disorder (62/150, 41.3% of studies), and clinicians (26/150, 17.3% of studies). Key facilitators were a strong recognition of DMHIs’ relative advantages, a clear link between intervention focus and specific patient needs, a simple, low-effort digital interface, human-supported delivery, and device provision where needed. Although staff thought patients would lose, damage, or sell devices, reviewed studies found only 11% device loss. Barriers included intervention complexity, perceived risks, user motivation, discomfort with self-reflection, digital poverty, symptoms of psychosis, poor compatibility with existing clinical workflows, staff and patient fears that DMHIs would replace traditional face-to-face care, infrastructure limitations, and limited financial support for delivery.

Conclusions

Identified barriers and facilitators highlight key considerations for DMHI development and implementation. As to broader implications, sustainable business models are needed to ensure that evidence-based DMHIs are maintained and deployed.

Trial Registration

PROSPERO CRD42021282871; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=282871

Introduction

Background

The costs and burdens of schizophrenia spectrum psychosis and bipolar disorder are huge and often neglected in research [1,2]. Digital mental health interventions (DMHIs) can help people with psychosis or bipolar disorder monitor, manage, and improve their symptoms and health [3-6]. Although there is promising evidence of feasibility, acceptability, and efficacy [7-9], and widespread expectations that DMHIs will become core to health care [10], user uptake and engagement are highly variable [11], and integrated clinical use is low [12]. The challenges of implementing DMHIs in health care are well established; attempts to implement well-evidenced DMHIs for common mental health problems in the United States and United Kingdom health services have frequently failed due to lack of patient and provider engagement and difficulty integrating DMHIs into clinical care [13]. For example, while randomized controlled trials (RCTs) of computerized cognitive behavior therapy (cCBT) for depression have reported relatively high engagement (47% of participants completing all cCBT sessions [14]), pragmatic trials delivering cCBT in real-world settings report much lower engagement (16%-18% of participants completing all sessions; most participants completing 1 session only) [15]. Implementing DMHIs for people with psychosis or bipolar disorder, within secondary care mental health services (eg, community mental health teams), presents additional challenges (eg, high staff caseloads and reactive care in response to increased risk or crisis), which have been examined in this review.

Implementation science acknowledges that intervention effectiveness cannot guarantee uptake in services; uptake depends largely on contextual barriers and facilitators [16]. Examining such contextual factors is therefore crucial to improving uptake in real-world settings, allowing tailoring of interventions to maximize engagement, and informing development and testing of implementation strategies. Barriers and facilitators of engagement with DMHIs for common mental health problems (eg, depression or anxiety) were examined relatively recently [11], but the last systematic review of factors affecting DMHI implementation in samples with psychosis or bipolar disorder was 5 years ago [17]. This review provides a timely update on this rapidly developing literature by examining barriers and facilitators of patient and staff engagement with DMHIs for this group.

Using a best-fit framework synthesis approach [18,19], the Consolidated Framework for Implementation Research (CFIR) [20] guided this analysis, enabling consideration of user engagement barriers and facilitators in the context of the complex, multilevel systems within and surrounding health care [21]. In supplementary analyses, we also examined safety reporting within reviewed studies to explore whether perceived harms may be additional barriers to DMHI uptake and to further explore recent findings on safety reporting quality [22,23].

Objective

In summary, this review examines evidence from qualitative and quantitative studies regarding the barriers and facilitators of patient and staff engagement with DMHIs for psychosis or bipolar disorder. By synthesizing this information using CFIR, this review presents important findings about how to maximize engagement with such DMHIs, and ideal conditions for DMHI implementation in secondary mental health services.

Methods

Design

This systematic review follows PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses; see Multimedia Appendix 1 for PRISMA checklist) guidance [24], with eligibility criteria developed according to the PICOS (participants, interventions, comparators, outcomes, study design) framework. The review was registered with PROSPERO (main review: CRD42021282871; safety review: CRD42022306123).

Search Strategy and Selection Criteria

We conducted a PRISMA-compliant search of 7 databases using search terms relating to psychosis and bipolar disorder, DMHIs, and barriers, facilitators or implementation (see search terms and database list in Multimedia Appendix 2). Searches were restricted to peer-reviewed articles reporting on studies with human participants, published in English between January 2010 and October 2021. By restricting the search in this way, we were able to capture the period between the first studies of digital mental health tools being conducted (consistent with previous reviews [11,25]) and the COVID-19 pandemic. The search end date (1.5 years after the worldwide COVID-19 restrictions peaked [26]), allows for the lag from data collection to publication. The search retrieved very few papers reporting data gathered during or after the pandemic, suggesting this was a suitable watershed date. A future systematic review will examine studies from October 2021 onward.

Author EE combined, deduplicated, and screened the titles and abstracts of the search results; author AP independently screened a random selection (10%). Full-text of the remaining articles were independently screened against PICOS inclusion and exclusion criteria by authors EE and AP. Studies included and excluded by each researcher were compared, disagreements were resolved by consensus, and final reasons for all exclusions were recorded. Full inclusion and exclusion criteria are provided in Multimedia Appendix 2. In summary, the review included all published studies reporting qualitative or quantitative data on hypothetical or actual barriers or facilitators of patient or staff engagement with a DMHI aiming to monitor or improve the mental or physical health of adults with a bipolar or schizophrenia spectrum diagnosis using a digital method (eg, smartphone app, SMS text messaging, website, or wearable).

Data Extraction

A data extraction form was used to extract relevant data: study metadata, design, DMHI details, and sample characteristics. Quantitative findings relating to barriers and facilitators of user engagement were extracted and imported into NVivo software (Lumivero) [27], as were the full results and discussion sections of qualitative studies. A second researcher checked all the extracted data. Study quality was assessed using the Mixed Methods Appraisal Tool [28]. In addition, from the main paper text and supplements of studies that tested the actual use of a DMHI, we extracted data on how adverse events (AEs) were monitored, classified (relatedness or severity), analyzed, reported, and discussed, and whether reporting followed the CONSORT (Consolidated Standards of Reporting Trials) harms checklist [29]. Given that the reporting of formal AE data is typically poor for DMHIs [22,23], we also extracted information about incidents that might be classifiable as AEs but were not described as such by the authors.

Data Analysis

Study characteristics, quality checks, and AE information were tabulated and summarized descriptively. A best-fit framework synthesis approach [18,19] was used to synthesize qualitative and quantitative data from all included studies to address the following research question: What are the barriers and facilitators of service user or staff engagement with DMHIs for people with psychosis or bipolar disorder? Generally, within a best-fit framework approach, relevant published frameworks, models, or theories are systematically identified, and key elements of these are integrated into an a priori coding framework for use in the review [19]. Primary studies are then deductively coded into the a priori coding framework, with data not fitting within the framework coded inductively. In this review, we were able to use a single existing implementation framework, the CFIR, as the a priori coding framework [20,30], followed by inductive coding. CFIR was developed using a comprehensive systematic review of the implementation science literature [20,30]. It is a meta-theoretical framework incorporating constructs from a wide range of existing implementation theories into a single comprehensive framework and is used by researchers and practitioners to identify barriers and facilitators of successful implementation and tailor their strategies accordingly. Thus, it is an appropriate choice of framework to guide the systematic evaluation of potential barriers and facilitators of user engagement with DMHIs in this review. CFIR has 5 major domains as provided in Textbox 1.

Major domains of Consolidated Framework for Implementation Research.Innovation characteristics: features of the innovation or intervention being implemented, including its complexity, adaptability, design, and evidence base. For example, a smartphone app for monitoring symptoms of psychosis.Outer setting: the external context in which the organization is situated, including economic, political, legal, and social factors that may influence implementation. For example, a particular country or state.Inner setting: features of the organization in which the intervention is being implemented, including its culture, structure, resources, and readiness for change. For example, a secondary care mental health service.Individuals: characteristics of individuals involved in the implementation process, such as their knowledge, beliefs, and motivation. For example, patients with psychosis or bipolar disorder, and mental health staff.Implementation process: the implementation process itself, including planning, engaging, executing, and reflecting.

Results

Search Results

Our search retrieved 7617 records after deduplication (Figure 1); 503 underwent full-text review, of which 175 were included based on inclusion and exclusion criteria. A list of the excluded studies and their reasons for exclusion are provided in Table S1 in Multimedia Appendix 2. Studies that did not contain data on barriers or facilitators of DMHIs were excluded (psychosis samples n=36; bipolar samples n=57). However, to supplement findings regarding the potential harms of DMHIs, we examined AE reporting in the 36 studies of psychosis samples.

Description of Included Studies

We included 175 papers reporting 150 studies (11,446 participants). Tables S2-S5 in Multimedia Appendix 2 outline the study design, sample characteristics, and quality checks. Most DMHIs focused on general symptom management, relapse prevention, or similar. Some focused on a specific symptom (eg, cognitive difficulties and paranoia), and a minority on medication adherence, lifestyle, physical activity, or smoking cessation. Common technology types were mobile apps (77/150, 51.3% studies), websites (42/150, 28% studies), SMS text messaging (13/150, 8.7% studies), and wearables (15/150, 10% studies). Slightly more DMHIs were blended (ie, delivered with human support, 49/150, 32.7% studies) or stand-alone (24/150, 16% studies) than minimally blended (23/150, 15.3% studies) or included setup or technical input only (20/150, 13.3% studies). The median DMHI use was 8 (IQR 4-19) weeks. Study types included single-arm interventional studies (47/150, 31.3%), RCTs (25/150, 16.7%), qualitative studies (20/150, 13.3%), cross-sectional surveys (14/150, 9.3%), observational studies (7/150, 4.7%), intervention development studies (13/150, 8.7%), usability testing studies (11/150, 7.3%), and case studies (7/150, 4.7%). Most studies examined actual DMHI use (106/150, 70.7%), though some were hypothetical (ie, asking participants about potential DMHI use, 44/150, 29.3%), and some tested in-session usability rather than use in daily life (9/150, 6%). Samples included people with schizophrenia spectrum disorders (98/150, 65.3% studies), bipolar disorder (62/150, 41.3% studies), and clinicians (26/150, 17.3% studies). Participants were recruited from a range of settings, including inpatient services (21/150, 14% studies), community services (115/150, 76.7% studies), and online adverts (16/150, 10.7% studies). The Mixed Methods Appraisal Tool subsections (scored out of 5; Table S4 in Multimedia Appendix 2) indicated the moderate quality of qualitative (mean 4.1, SD 1.57) and quantitative (mean 4.0, SD 1.0) study elements, and lower mixed methods quality (mean 3.1, SD 1.9). The CONSORT harms checklist showed AE reporting to be mostly poor or missing (Table S5 in Multimedia Appendix 2); thus, it is difficult to draw informative conclusions regarding the harms of DMHIs.

Findings Not Fitting Within CFIR Framework

Although most relevant findings were coded into CFIR, a few could not be coded: patterns of DMHI use over time, associations between demographic variables and DMHI engagement, and safety reporting findings.

Patterns of Use

Sixteen studies reported reduced DMHI use over time in this study, 7 reported stable use and none reported an increase.

Demographics

Demographic findings are outlined in the Table S6 in Multimedia Appendix 2. Only 28% (11/39) studies examining age found significant effects: 6 found higher engagement in older people, 4 in younger people, and 1 found mixed effects. Conversely, all staff participants in qualitative studies hypothesized higher DMHI engagement among younger people [31-35]. Only 28% (7/25) studies examining gender or sex reported significant effects; most (n=5) found higher DMHI engagement among women. Only 20% (4/20) studies examining education reported significant effects; all 4 found higher education associated with higher engagement. Only 20% (2/11) studies reported significant effects of race, ethnicity, or immigration: Ben-Zeev et al [36] reported significantly higher engagement among White participants than African American participants (and among White participants than Hispanic participants for some, but not all, engagement indices), while Bonet et al [37] reported significantly higher agreement to use a DMHI among non-native participants than native Spanish participants. Of studies examining the effects of IQ, 67% (2/3) showed significant effects, with higher IQ associated with higher DMHI engagement [38,39]. Reading level (0/1), employment (0/5), and marital status (0/2) showed no significant effects.

Safety Reporting

Multimedia Appendix 2 describes safety information extracted from 168 papers (139 studies) reporting actual DMHI use. Only 86 papers (76 studies) reported AE-related content; 23 reported AE frequency or percentage (median number of AEs reported per study 2, IQR 0.8-39.2). AEs included hospitalizations, psychological distress, suicidal ideation, physical health-related events, and death. Relatedness of AEs to the intervention was only reported in 11 papers, 8 of which deemed some events related.

CFIR Coding

Tables 1-5 provide detailed findings regarding the coding of reviewed studies into the 5 domains of the CFIR framework: Innovation (Table 1), Outer setting (Table 2), Inner setting (Table 3), Individual characteristics (Table 4), and Implementation process (Table 5). Findings were most frequently related to Innovation (Table 1; n=149 papers) and Individual (Table 4; n=137) domains of CFIR, with aspects of the Outer setting (Table 2; n=9), Inner setting (Table 3; n=20), and Implementation process (Table 5; n=5) domains rarely described. Figure 2 shows the number of papers coded into each CFIR domain per year of publication. Studies published between 2010 and 2015 were coded almost exclusively against the Innovation or Individual CFIR domains. Occasional studies were then coded against the Inner setting domain from 2016 onward, and against the Inner setting, Outer setting, and Implementation process domains from 2018 onward.

Prominent subcodes within the Innovation domain were Relative advantage (99/175, 56.6% papers), Complexity (56/175, 32% papers), and Design (128/175, 73.1% papers). As many papers commented on DMHI design, details of inductive subcodes are provided separately in Table 6. These included design features to increase DMHI engagement, customization, repetition vs variety, type of device used, user interface design, passive sensing, data security and privacy, and technical difficulties. Need (48/175, 27.4% papers), Capability (88/175, 50.3% papers), Opportunity (48/175, 27.4% papers), and Motivation (73/175, 41.7% papers) from the Individuals domain were all widely used, although almost exclusively in relation to innovation recipients rather than other roles. Details of how prominent findings from the CFIR-coded data related to the 5 crosscutting themes, are provided in the Crosscutting Themes section.

Crosscutting Theme 1: DMHIs to Meet Specific Needs, Not Replace Human Care

Findings emphasized that DMHI functions and content should align with specific patient needs and that patients and staff must perceive a benefit if they are to engage with a DMHI (n=34 studies; Individuals, Need; Table 4). Patients preferred DMHIs with a clear aim, which did not try to do too many things (Innovation: Complexity; Table 1). Several DMHI benefits came through clearly in reviewed studies (Innovation: Relative advantage), including their value in being available anywhere and anytime (n=37 studies), promoting autonomy (n=26), facilitating self-awareness and memory (n=61), connecting people with peers (n=29), and normalizing mental health experiences (n=26; Innovation: Relative advantage). These advantages contributed to perceptions that, if successfully implemented, DMHIs could widen the reach of services. Nevertheless, while patients and staff acknowledged that DMHIs might enhance care in specific ways, they emphasized that they should not replace human care (n=16; Innovation: Relative advantage) and staff worried that the increasing emphasis on DMHIs might detract from health services’ ability to deliver care (Inner setting: Mission alignment; Table 3).

Seeing the benefit of a DMHI in meeting their needs prompted patients’ initial use and helped sustain use (Individuals: Need). Studies (n=27) highlighted that patient needs may change over time, depending on illness stage and symptom levels. Certain DMHI content (eg, psychoeducation) and functions (eg, symptom tracking) may be best suited to the needs of people with recent onset illness or ongoing symptoms. Indeed, 3 studies [54,120,121] testing DMHI use qualitatively reported discontinuations due to improved mental health, as participants no longer felt they needed the DMHI. Tailoring or customization (Design; Table 6) allows DMHIs to be adapted for varied needs; eg, different psychoeducation “levels,” staggered unlocking of content based on users’ evolving needs, or adaptive frequency of symptom monitoring depending on symptom levels.

While staff considered some aspects of DMHIs (Innovation: Relative advantage) aligned with their core “mission”, they saw human contact as a central tenet of their mission that was potentially undermined by DMHIs (Inner setting: Mission alignment). Staff worried that “patients would see using the app system as a replacement for care, something it was not designed to do” [76], and doubted DMHIs could be effective without a therapeutic relationship: “If you are going to talk evidence-based, the biggest thing in every single intervention... is therapeutic relationship, so you are just removing the most effective part” [33]. Such views could be a prominent barrier to DMHI implementation, particularly for stand-alone or minimally blended DMHIs. This underlines the importance of clearly communicating the specific purpose of the DMHI, and its limits, and reassuring staff that it is not intended to replace them.

Crosscutting Theme 2: Human Support for DMHIs

Forty-nine studies highlighted human support as a key facilitator of DMHI engagement (Individuals: Motivation; Table 4), underlining a need to align DMHI implementation with clinical workflows. Reviewed studies found a lack of compatibility with current job roles and workflows (n=17), with DMHIs seen as creating extra work in already stretched teams (Inner setting: Compatibility; Table 3) and as less important than other tasks (Inner setting: Mission alignment). Clinicians did not want to receive a “constant stream of information” about patients [33]. They thought symptom monitoring-based DMHIs would increase their responsibilities, and questioned who would respond out-of-hours. Staff lacked time to facilitate DMHI use, found data from symptom-monitoring apps complex to interpret (Innovation: Complexity; Table 1), and sometimes lacked digital skills (Inner setting: Access to knowledge and information).

Data on types of support needed suggested that not all support must be provided by someone clinically trained (Individuals: Motivation). Support needs varied, depending on the innovation’s or individual’s status on other CFIR constructs (see CFIR codes in parentheses), but included: training in basic digital skills (Individuals: Capability), recommending suitable DMHIs, onboarding, managing technical difficulties (Design; Table 6), support navigating DMHIs (Innovation Complexity), ongoing coaching (Individuals: Motivation), moderation of social media components, and interpreting and triaging real-time symptom data. Arguably, these tasks could be undertaken by a suitably trained nonclinical staff member, with referrals made to clinicians as needed (eg, when symptoms increase above a predefined threshold, or risk is indicated). Similar observations have prompted the idea of restructuring services to include new job roles for DMHI facilitation (eg, digital navigator, peer supporter). Reviewed studies did not test the implementation of such roles within clinical services; although, many DMHIs were successfully delivered by someone in an equivalent role in a research context. Of 4 RCTs quantitatively examining the effects of support, 2 demonstrated significantly higher DMHI engagement in peer-supported than unsupported trial arms [121,134,135], 1 found a significant positive effect of brief interviewer-facilitated symptom reviews on app adherence for younger women (no significant overall effect [136]), and 1 found moderate effect sizes in favor of the supported group [137] but was not powered to detect a significant difference (Individuals: Motivation).

Six studies provided evidence regarding whom staff or patients might prefer to deliver DMHI support (ie, clinicians or nonclinicians). Studies with staff suggested a preference for nonclinician support for initial onboarding [83], technical [83,114,156], or peer-supported [114] aspects of DMHI use, with input from a clinician or team leader for more clinical aspects [114,157]. Patients expressed mixed views, with some preferring support from a known clinician [68,157], but others viewing this as more “intrusive” than nonclinician support [122]. Qualitative studies reported that, as well as overcoming practical barriers (eg, poor digital literacy, technical difficulties, complex DMHI layout), human connection helped address barriers like low motivation and anxiety about DMHI use, and provided accountability: “I’m going to see [coach]... within a few days, and we were going to discuss this, then there was accountability, I had to get some things done” [87]. Peer support was valued, whether as a formal part of the intervention (peer facilitators) or via social media (Innovation: Relative advantage), as peers “...can relate to each other and they know what they’ve been through” [54], and participants did “not feel scared people will judge you because they have been through the same thing and will be supportive” [55].

Crosscutting Theme 3: Practical Helps and Hindrances

Reviewed studies outlined practical factors that can help or hinder DMHI implementation. Arising across CFIR constructs, these factors tended to hinge on the principle that DMHI use should be as low-effort as possible for users, and practical barriers to access should be addressed.

To maximize engagement, DMHIs should ideally be free or inexpensive to patients (Innovation: Cost; Table 1), use similar formats to everyday technology (eg, smartphone app), and allow on-demand, out-of-hours use (Innovation: Relative advantage). DMHIs should use straightforward language, be visually appealing (Design; Table 6), minimize complexity, avoid lengthy login processes, use an easy-to-navigate app or website structure (shallow structure, <200 words/page), and be compatible with users’ own devices (Innovation: Complexity). Patients found smartphone apps more convenient than computer or SMS text messaging systems, though apps must be thoroughly tested to minimize technical difficulties (Design). Other design elements to increase engagement included push-notification reminders; gamification features (eg, badges for completed tasks); automated feedback about symptoms or DMHI use (eg, symptom graph, step count, and minutes DMHI used); and tailored or varied content to minimize repetition.

These factors interact with each other. For example, tailoring DMHIs may increase engagement, but can also increase complexity, an engagement barrier. In turn, other factors, such as training and support, and individual capability (eg, cognitive ability and digital literacy) may moderate the negative effects of complexity on engagement. Nevertheless, even with training and support, and high individual capability, more complex interventions require more time and effort from users, making disengagement more likely. DMHI design teams must weigh the relative advantages and disadvantages of such interacting factors.

Some engagement barriers could be mitigated given sufficient resources. Lack of access to technology (Individuals: Opportunity; Table 4) and lack of digital literacy (Individuals: Capability) could be addressed, in theory, by providing devices and training in how to use them. Although staff thought patients would lose, damage, or sell devices, reviewed studies found only an 11% loss of study-provided devices. Data security and privacy concerns were frequently mentioned barriers; patients were concerned about sensitive symptom data being disclosed, due to the potential stigma associated with it. Measures to protect data or mitigate the effects of data loss included the use of pseudonyms, password protection, and the use of general terms (eg, “illness”) rather than specific diagnoses.

Some broader practical barriers may be difficult for implementation teams to address directly. These included the lack of mobile phone signal, publicly-funded smartphones being too basic, lack of national guidelines and policies on DMHI use in health services (Outer setting; Table 2), aging computers, poor bandwidth, no wireless connection in hospitals, poor organizational support for IT developments, and lack of quiet space (Inner setting; Table 3), patient time, or suitable environment (Individuals: Opportunity).

Crosscutting Theme 4: Interactions Between Symptoms and DMHI Engagement

Approximately one-third of quantitative analyses (18/52, 35%) reported a significant effect of symptoms on DMHI engagement (Individuals: Capability; Table 4; Table S7 in Multimedia Appendix 2). Among these, higher baseline positive, negative, or cognitive symptoms were associated with lower DMHI engagement. Qualitative data suggested that staff and patients anticipated such a pattern (and so may recommend or agree to DMHI use accordingly), and some patients avoided DMHIs due to experiencing difficult interactions between symptoms and technology (eg, technology-related delusions or paranoia). Findings for functioning and depression were more mixed, with higher baseline functioning or depression sometimes predicting higher and sometimes lower DMHI engagement. Qualitative studies suggested that while symptoms of depression (eg, poor concentration and motivation) may interfere with DMHI engagement, low mood may prompt higher engagement for some “because of a desire to find meanings and solutions for their depressive symptoms” [72]. No quantitative studies examined the effects of relapse on engagement; qualitative studies noted decreased DMHI use during relapse; however, participants usually restarted DMHIs on remission. Some were hesitant to (accurately) report symptoms via DMHIs for fear it may lead to more restrictive treatment (hospital admission and increased medication).

Negative interactions between DMHIs and symptoms, or perceived risk of such interactions, emerged as possible motivational barriers. Patients sometimes experienced DMHIs as “deflating, confronting, or triggering,” reminding them how unwell they were, or about their diagnosis, which could be disheartening, “self-stigmatizing,” and often prompted disengagement (Individuals: Motivation). There were numerous quotations about this, across 10 studies; however, it was difficult to tell how many participants were typically affected. While 1 study reported that half the sample felt negatively influenced by logging their moods and symptoms 5 times per day [157], in other studies with less intensive report schedules, very few (1/27, 4%) participants found the DMHI to be confronting [88]. Relatedly, fear of relapse was a barrier to DMHI use. Staff and patients anticipated that, by increasing awareness of symptoms, DMHIs would heighten users’ anxiety about deterioration, which may worsen symptoms. In studies testing actual DMHI use, there was qualitative [45,66,88,102] and quantitative [126] evidence that fear of relapse and increased rumination about symptoms prevented DMHI engagement but the proportion of participants affected was unclear. Reviewed papers did not typically report such negative reactions to DMHI use (ie, finding the DMHI “confronting,” or experiencing fear of relapse) as adverse reactions in the context of formal AE monitoring (Multimedia Appendix 2). Arguably, doing so would allow a quantitative examination of how many participants were affected across studies.

Crosscutting Theme 5: Financial Provision for Implementing DMHIs

There was scant empirical information across CFIR constructs about the effects of financial provision on DMHI implementation, probably because most studies examined DMHI use in research studies rather than services. However, findings implied that financial limitations concerning software development and maintenance may indirectly reduce engagement. One study noted that some participant-suggested technical improvements were not possible due to financial limitations [107]. Another noted that the look and feel of the study app were considered “old school” but insufficient funding precluded improvements [109] (Innovation: Cost; Table 1); study authors highlighted the challenge of competing with commercial apps: “It is difficult to obtain... funding at a level that can compete with the funding that goes into app design for functions that people routinely use... leaving [study app] looking ‘old school’” [109]. Others noted that, while patients prefer free-to-use DMHIs (Outer setting: Financing; Table 2), sustainable business models for maintaining evidence-based DMHIs are often absent: “...apps that have been empirically evaluated and developed in accordance with clinical guidelines are rarely made available via the app marketplace... Thoughtful plans for commercialization and fiscal sustainability are important, given respondents with BD endorsed free/low-cost apps” [47]. This statement was supported by a striking absence of information from other reviewed studies about their intended model for DMHI dissemination and implementation, except a US-based study which mentioned costs were offset by the service’s billable nature [110] and a Canadian study that anticipated care provider networks would purchase DMHI licenses [109].

Some studies (5/150, 3.3%) listed costs associated with delivering DMHIs in the research context, including web hosting, software licensing fees, texting fees, technical support, and staff time for DMHI delivery. Financial provision for these and other DMHI-associated costs will be key to scaling up DMHI use in services but was not directly discussed in reviewed studies. Nevertheless, to maximize engagement with DMHIs for psychosis or bipolar disorder per the cross-cutting themes outlined above, considerable financial support would be necessary, particularly to provide human support for DMHI use (theme 2), to source or develop well-designed DMHIs that are as low-effort as possible for users (theme 3), and to address practical needs, such as digital poverty (theme 3).

Discussion

Principal Results

This systematic review found that patient and staff engagement with DMHIs is likely to be strengthened by using DMHIs to meet specific needs, not replace human care; providing human support for DMHIs and considering compatibility with current workflows; making DMHIs as low effort as possible for patients and staff, inquiring about common practical barriers to use (eg, digital poverty or training needs) and addressing these where possible; acknowledging interactions between symptoms and DMHIs, including motivational barriers and fear of relapse; and ensuring appropriate financial provision to implement DMHIs in line with these recommendations.

Comparison With Prior Work

Similar to the review by Aref-Adib et al [17] on DMHIs for psychosis or bipolar disorder, most data were coded into CFIR Intervention (149/175, 85.1% papers) or Individuals (137/175, 78.3% papers) domains, with very little covering Outer setting (9/175, 5.1% papers) or Implementation process (5/175, 2.9% papers). We found more data related to the Inner setting (20/175, 11.4% papers) than the previous review, with studies highlighting a lack of compatibility with clinical workflows (17/175, 9.7% papers) and lack of alignment with staff perceptions of their clinical “mission” (5/175, 2.9% papers) as important barriers to DMHI implementation. This difference appears to reflect progress in the psychosis and bipolar disorder DMHI literature over the intervening years (Figure 1). Workflow-related barriers are mentioned in reviews covering DMHIs for common mental health disorders [21,158,159], a literature which has a chronological head-start on the equivalent literature for psychosis and bipolar disorder.

Our finding that human support was a key facilitator of DMHI engagement is consistent with previous reviews of factors affecting the implementation of DMHIs for psychosis or bipolar disorder [17,160], common mental health problems [11,21,158,161,162], and a meta-synthesis of staff views [158]. Relatedly, our review and others reported that clinicians’ negative attitudes to DMHIs, resistance to change, and lack of digital skills were sometimes barriers to DMHI implementation [17,158,163]. Nevertheless, such considerations are likely eclipsed by a need for compatibility with clinical workflows (which our findings suggest is currently absent; see Inner setting: Compatibility). Restructuring services to include new “digital navigator” roles, alongside traditional clinical roles, has been suggested as a model for implementing DMHIs in mental health services [10,164], and meta-analysis shows similar effectiveness across DMHIs guided by clinicians and nonclinicians [165]. Whether this model or any other service model is used, staff members (clinicians and nonclinicians) would need allocated time, training, and supervision to support patients’ DMHI use [158], all of which have financial implications [166].

DMHIs are frequently cited as potential methods of extending care in a cost-effective manner [167]. There is some empirical evidence supporting this, albeit from DMHIs tested in a research context [168], but economic evaluations of mental health DMHIs are rare [169]. Nevertheless, up-front costs of restructuring services and ongoing costs of delivering DMHIs will likely be substantial [17]. Few reviewed papers (5/175, 2.9%) directly evaluated financial factors, but our findings highlight the importance of considering staff-related costs and costs of addressing engagement barriers (eg, dated-looking digital interfaces; patients lacking suitable devices or skills). Without financial investment, DMHI engagement will likely be poor, potentially perpetuating existing treatment disparities [170,171]. More broadly, reviewed studies highlighted the difficulties of developing evidence-based DMHIs against a background of rapid technological advances and attractive but poorly evidenced commercial apps. Hill et al [172] outlined similar issues in relation to developing evidence-based DMHIs for common mental health problems; they recommended that researchers collaborate with industry partners to commercialize well-evidence DMHIs, generating funds to cover ongoing maintenance and development, and enhancing long-term sustainability.

We found that mental health symptoms and DMHIs sometimes interacted in ways that prevented DMHI engagement, which echoes and expands on findings from the review by Aref-Adib et al [17]. We found more evidence of motivational barriers relating to increased rumination or fear of relapse (26/175, 14.9% papers) than that found in the previous review (1/26, 4%). We also found that patients disengaged with DMHIs because they found increased self-reflection and attention toward negative emotional states too “confronting,” which has not been reported in previous reviews. Evidence from reviewed studies did not allow an estimate of how many patients are affected by these barriers. Future studies should consider examining such reactions quantitatively, perhaps as part of formal AE monitoring, to help future users make informed choices about DMHI risks and benefits. More generally, our supplementary analysis of safety reporting suggested that AE reporting warrants considerably more attention, in line with recent findings from other reviews [22,23,173] and recommendations regarding AE monitoring and reporting for DMHIs for psychosis [174].

Strengths and Limitations of the Evidence Base

The reviewed literature contained a wealth of qualitative and quantitative data outlining factors that may affect staff and patient engagement with DMHIs for psychosis or bipolar disorder. However, the current evidence base has important limitations. First, most studies were conducted in traditional research settings rather than evaluating the process of implementing DMHIs directly in health services. Consequently, identified barriers and facilitators may not all translate and generalize to care provision settings. Nevertheless, the review provides an early indication of likely barriers and facilitators, across the CFIR domains, enabling these to be considered when designing future DMHIs and DMHI implementation strategies. This information will also be valuable in highlighting key implementation factors to prospectively examine in future implementation studies. Second, most studies seeking health professional views sought this information hypothetically (ie, they asked staff, who had not necessarily already used DMHIs, what they thought about using DMHIs in their clinical practice, in theory), whereas studies seeking information from patients more often asked about actual experiences of DMHI use (ie, they asked patients, who had already used a specific DMHI, about what it was like to use that DMHI). Berry et al [175] suggested that hypothetical views and actual experiences of DMHIs do not always align. Therefore, the difference in methods between patient and health professional studies must be kept in mind when considering the findings of this review.

Strengths and Limitations of This Review

A key strength was the use of CFIR as a framework, allowing examination of interactions between staff and patient barriers and facilitators and wider contextual factors, and sensitizing us to implementation factors that may not otherwise have been identified in reviewed studies. Studies were identified using an inclusive search strategy, with intentionally wide inclusion criteria (eg, DMHI types, settings, and engagement indicators) to allow examination of a broad range of barriers and facilitators. However, the resulting studies were extremely heterogeneous, precluding meta-analysis and potentially conflating results from dissimilar studies. Likewise, the review covered a 21-year period (2010 to 2021), spanning a time of great technological advancement during which societal acceptance of DMHIs may have changed considerably, adding further heterogeneity. Finally, we limited our search to the period before the COVID-19 pandemic. Although this avoided COVID-19 pandemic–related heterogeneity, if the search had been conducted later, findings related to the impacts of the COVID-19 pandemic on digital uptake (Outer setting: Critical incidents) would likely have been prominent. We recommend that a future review should focus on mixed methods studies published since the pandemic, to examine the shifting views of staff and patients regarding potential barriers and facilitators of digital health tools for psychosis and bipolar disorder in the intervening period.

Conclusions

While DMHIs have a role in supporting those with psychosis or bipolar disorder, intervention, individual, and contextual factors all impact uptake. To enhance engagement, DMHIs must be simple, low effort, meet specific needs, and not be heralded as a replacement for face-to-face care. Human support can overcome engagement barriers related to motivation and practicalities of DMHI engagement. Financial provision is likely to be key to sustainable DMHI implementation.

Authors' Contributions: EE analyzed the data and drafted the paper as the first author, with support from SB and SF. All coauthors had the opportunity to contribute to drafts of the paper, to critically review the manuscript, and to approve the final version. EE and SB devised the study aims and methods and supervised the researchers during data collection and analysis. PW and JN provided methodological advice. SA, HB, DDB, JN, XZ and AP contributed to data collection.

Conflicts of Interest: SB is a director and shareholder of CareLoop Health Ltd, which develops and markets digital therapeutics for schizophrenia and a digital screening app for postnatal depression. All other authors declare no conflicts of interest.