Dataset Information

Content-Based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study.

ABSTRACT:

Background

The working environment of a suicide prevention helpline requires high emotional and cognitive awareness from chat counselors. A shared opinion among counselors is that as a chat conversation becomes more difficult, it takes more effort and a longer amount of time to compose a response, which, in turn, can lead to writer's block.

Objective

This study evaluates and then designs supportive technology to determine if a support system that provides inspiration can help counselors resolve writer's block when they encounter difficult situations in chats with help-seekers.

Methods

A content-based recommender system with sentence embedding was used to search a chat corpus for similar chat situations. The system showed a counselor the most similar parts of former chat conversations so that the counselor would be able to use approaches previously taken by their colleagues as inspiration. In a within-subject experiment, counselors' chat replies when confronted with a difficult situation were analyzed to determine if experts could see a noticeable difference in chat replies that were obtained in 3 conditions: (1) with the help of the support system, (2) with written advice from a senior counselor, or (3) when receiving no help. In addition, the system's utility and usability were measured, and the validity of the algorithm was examined.

Results

A total of 24 counselors used a prototype of the support system; the results showed that, by reading chat replies, experts were able to significantly predict if counselors had received help from the support system or from a senior counselor (P=.004). Counselors scored the information they received from a senior counselor (M=1.46, SD 1.91) as significantly more helpful than the information received from the support system or when no help was given at all (M=-0.21, SD 2.26). Finally, compared with randomly selected former chat conversations, counselors rated the ones identified by the content-based recommendation system as significantly more similar to their current chats (?=.30, P<.001).

Conclusions

Support given to counselors influenced how they responded in difficult conversations. However, the higher utility scores given for the advice from senior counselors seem to indicate that specific actionable instructions are preferred. We expect that these findings will be beneficial for developing a system that can use similar chat situations to generate advice in a descriptive style, hence helping counselors through writer's block.

SUBMITTER: Salmi S

PROVIDER: S-EPMC7819775 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Content-Based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study.

Salmi Salim S Mérelle Saskia S Gilissen Renske R Brinkman Willem-Paul WP

Journal of medical Internet research 20210107 1

<h4>Background</h4>The working environment of a suicide prevention helpline requires high emotional and cognitive awareness from chat counselors. A shared opinion among counselors is that as a chat conversation becomes more difficult, it takes more effort and a longer amount of time to compose a response, which, in turn, can lead to writer's block.<h4>Objective</h4>This study evaluates and then designs supportive technology to determine if a support system that provides inspiration can help coun ...[more]

PMID: 33410755

Similar Datasets

Project description:BackgroundWith the rise of computer science and artificial intelligence, analyzing large data sets promises enormous potential in gaining insights for developing and improving evidence-based health interventions. One such intervention is the counseling strategy motivational interviewing (MI), which has been found effective in improving a wide range of health-related behaviors. Despite the simplicity of its principles, MI can be a challenging skill to learn and requires expertise to apply effectively.ObjectiveThis study aims to investigate the performance of artificial intelligence models in classifying MI behavior and explore the feasibility of using these models in online helplines for mental health as an automated support tool for counselors in clinical practice.MethodsWe used a coded data set of 253 MI counseling chat sessions from the 113 Suicide Prevention helpline. With 23,982 messages coded with the MI Sequential Code for Observing Process Exchanges codebook, we trained and evaluated 4 machine learning models and 1 deep learning model to classify client- and counselor MI behavior based on language use.ResultsThe deep learning model BERTje outperformed all machine learning models, accurately predicting counselor behavior (accuracy=0.72, area under the curve [AUC]=0.95, Cohen κ=0.69). It differentiated MI congruent and incongruent counselor behavior (AUC=0.92, κ=0.65) and evocative and nonevocative language (AUC=0.92, κ=0.66). For client behavior, the model achieved an accuracy of 0.70 (AUC=0.89, κ=0.55). The model's interpretable predictions discerned client change talk and sustain talk, counselor affirmations, and reflection types, facilitating valuable counselor feedback.ConclusionsThe results of this study demonstrate that artificial intelligence techniques can accurately classify MI behavior, indicating their potential as a valuable tool for enhancing MI proficiency in online helplines for mental health. Provided that the data set size is sufficiently large with enough training samples for each behavioral code, these methods can be trained and applied to other domains and languages, offering a scalable and cost-effective way to evaluate MI adherence, accelerate behavioral coding, and provide therapists with personalized, quick, and objective feedback.

Project description:A Recommender System (RS) is an intelligent system that assists users in finding the items of their interest (e.g. books, movies, music) by preventing them to go through huge piles of data available online. In an effort to overcome the data sparsity issue in recommender systems, this research incorporates a content based filtering technique with fuzzy inference system and a conformal prediction approach introducing a new framework called Hybrid Content based Fuzzy Conformal Recommender System (HCF-CRS). The proposed framework is implemented to be used in the domain of movies and it provides quality recommendations to users with a confidence level and an improved accuracy. In our proposed framework, first, a Content Based Filtering (CBF) technique is applied to create a user profile by considering the history of each user. CBF is useful in the situations like: lack of demographic information and the data sparsity problems. Second, a Fuzzy based technique is incorporated to find the similarities and differences between the user profile and the movies in the dataset using a set of fuzzy rules to get a predicted rating for each movie. Third, a Conformal prediction algorithm is implemented to calculate the non-conformity measure between the predicted ratings produced by fuzzy system and the actual ratings from the dataset. A p-value (confidence measure) is computed to give a level of confidence to each recommended item and a bound is set on the confidence level called a significance level ?, according to which the movies only above the specified significance level are recommended to user. By building a confidence centric hybrid conformal recommender system using the content based filtering approach with fuzzy logic and conformal prediction algorithm, the reliability and the accuracy of the system is considerably enhanced. The experiments are evaluated on MovieLens and Movie Tweetings datasets for recommending movies to the users and they are compared with other state-of-the-art recommender systems. Finally, the results confirm that the proposed algorithms perform better than the traditional ones.

Project description:IntroductionYouth suicide is a significant public health priority, and is the second leading cause of death among young people between 15 and 29 years of age. An emerging intervention in suicide prevention programming with youth is peer support. Although increasingly used in other settings, the evidence for peer support interventions in youth suicide prevention remains nascent. This article presents a protocol for a scoping review aimed at systematically mapping the current evidence on peer support for youth suicide prevention.Methods and analysisArksey and O'Malley's scoping review framework will guide the review methods. The search strategy will be developed with guidance from a health sciences librarian. Multiple databases (Medline, Embase, PsycINFO, Cumulative Index for Nursing and Allied Health Literature) and grey literature will be identified using terms related to peer support and youth suicide prevention. Publication date restrictions will not be applied. All identified records of published literature will be collated and uploaded to a systematic review management software, Covidence, for review and selection. Screening will be completed in duplicate by two reviewers using predefined inclusion and exclusion criteria. Conflicts during screening will be resolved by a third reviewer. The title and abstract screening and full-text review will be completed in Covidence. Two reviewers will complete data extraction from the selected records, using a tailored extraction form. Screening and data extraction will be completed between January and April 2021. A narrative summary will be completed to synthesise key findings as well as contextual information about the use of peer support interventions in youth suicide prevention programming.Ethics and disseminationThe results will be disseminated through a community research report, presentations of findings at relevant conferences and academic publications of the scoping review protocol and results. The data used for this scoping review will be derived from published resources; therefore, ethics approval is not required for this study.

Project description:BackgroundPeer-led support models have gained increasing popularity in suicide prevention. While previous reviews show positive effects of peer-led support for people with mental health problems and those bereaved by suicide, little is known about the types of lived experience peer support programs in suicide prevention and whether these are effective in improving the health and wellbeing of people at risk of suicide. The aim of this paper is to provide an overview of peer support programs that aim to reduce suicidality and are led by people with lived experience of suicide.MethodWe conducted a systematic scoping review, involving a search of three academic (Medline, PsycINFO, Embase) and selected grey literature databases (Google Scholar, WHO Clinical Trials Registry) for publications between 2000 and 2019. We also contacted suicide prevention experts and relevant internet sites to identify peer support programs that exist but have not been evaluated. The screening of records followed a systematic two-stage process in alignment with PRISMA guidelines.ResultsWe identified 8 records accounting for 7 programs focussed on peer-led support programs in suicide prevention. These programs employed a range of different designs and included a variety of settings (schools, communities, rural and online). Only 3 of the 7 programs contained data on effectiveness. With the small number of eligible programs the findings from this review are limited and must be interpreted with caution.ConclusionsDespite the increased focus of policymakers on the importance of peer support programs in suicide prevention, our scoping review confirms an evidence gap in research knowledge regarding program design, implementation, and effectiveness. More rigour is required in reporting peer-led support initiatives to clarify the underlying definition of peer support and lived experience and to enhance our understanding of the types of current peer support programs available to those experiencing suicidality. Further, we need formal and high-quality evaluations of peer support suicide prevention programs led by people with lived experience to better understand their effectiveness on participant health across different settings and delivery modalities and to allow for comprehensive systematic reviews and meta-analysis in future.

Dataset Information

Content-Based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study.

Background

Objective

Methods

Results

Conclusions

Publications

Content-Based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets