Dataset Information

Validation of a novel online depression symptom severity rating scale: the R8 Depression.

ABSTRACT:

Background

An automated web-based assessment and monitoring system ( www.psynary.com ) has been developed to assist non-specialist clinicians in managing common mood and anxiety disorders. Psynary promotes the use of standardised outcome measures to assess symptom severity and optimise treatments with the aim of improving outcomes and enabling faster recovery. This paper analyses the results from two parallel studies in New Zealand and Japan (OptiMA-1 NZ and Japan) to assess the validity of the R8 Depression scale, one of the system's core outcome measures.

Methods

Clinical samples were recruited from a public secondary care and a private psychiatry clinic. Participants completed the outcome measures for the study via the online Psynary system. The R8 Depression scale is a 30-item questionnaire which includes all symptom domains covered in the ICD-10 classification of depression. The Patient Health Questionnaire (PHQ-9) was completed at the same time points as the R8 Depression, with a smaller sample also completing a paper-based Quick Inventory of Depressive Symptomatology (QIDS-SR16). Internal validity was quantified via Cronbach's alpha and Guttman lower bounds method. External validation against the PHQ-9 and QIDS used the Pearson's and Kendall's correlation coefficients. Severity categories were set using a multivariate regression model.

Results

270 patients participated in the study and completed a maximum of 1 baseline and 5 reviews within a 90-day period, giving a total of 1124 assessments with the PHQ-9 also being completed in 1053 of these assessments. R8 Depression normative data was also collected from 204 non-clinical volunteers with 187 of these also completing the PHQ9. Internal reliability scores were all higher than 0.9 (n = 1328). There was overall good external validity when comparing the R8 Depression to the PHQ-9, with a correlation of 0.91 for the combined normative and clinical samples (n = 1240).

Conclusions

The R8 Depression has been developed as a patient-rated outcome measure for depression for administration on an online system called "Psynary". It has high internal and external validity against current widely used scales. Further work is underway to determine the sensitivity to change of the R8 Depression.

SUBMITTER: Takao Y

PROVIDER: S-EPMC8196428 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe lack of culturally adapted and validated instruments for child mental health and psychosocial support in low and middle-income countries is a barrier to assessing prevalence of mental health problems, evaluating interventions, and determining program cost-effectiveness. Alternative procedures are needed to validate instruments in these settings.MethodsSix criteria are proposed to evaluate cross-cultural validity of child mental health instruments: (i) purpose of instrument, (ii) construct measured, (iii) contents of construct, (iv) local idioms employed, (v) structure of response sets, and (vi) comparison with other measurable phenomena. These criteria are applied to transcultural translation and alternative validation for the Depression Self-Rating Scale (DSRS) and Child PTSD Symptom Scale (CPSS) in Nepal, which recently suffered a decade of war including conscription of child soldiers and widespread displacement of youth. Transcultural translation was conducted with Nepali mental health professionals and six focus groups with children (n=64) aged 11-15 years old. Because of the lack of child mental health professionals in Nepal, a psychosocial counselor performed an alternative validation procedure using psychosocial functioning as a criterion for intervention. The validation sample was 162 children (11-14 years old). The Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS) and Global Assessment of Psychosocial Disability (GAPD) were used to derive indication for treatment as the external criterion.ResultsThe instruments displayed moderate to good psychometric properties: DSRS (area under the curve (AUC)=0.82, sensitivity=0.71, specificity=0.81, cutoff score ? 14); CPSS (AUC=0.77, sensitivity=0.68, specificity=0.73, cutoff score ? 20). The DSRS items with significant discriminant validity were "having energy to complete daily activities" (DSRS.7), "feeling that life is not worth living" (DSRS.10), and "feeling lonely" (DSRS.15). The CPSS items with significant discriminant validity were nightmares (CPSS.2), flashbacks (CPSS.3), traumatic amnesia (CPSS.8), feelings of a foreshortened future (CPSS.12), and easily irritated at small matters (CPSS.14).ConclusionsTranscultural translation and alternative validation feasibly can be performed in low clinical resource settings through task-shifting the validation process to trained mental health paraprofessionals using structured interviews. This process is helpful to evaluate cost-effectiveness of psychosocial interventions.

Project description:Post stroke dysphagia (PSD) is common and associated with poor outcome. The Dysphagia Severity Rating Scale (DSRS), which grades how severe dysphagia is based on fluid and diet modification and supervision requirements for feeding, is used for clinical research but has limited published validation information. Multiple approaches were taken to validate the DSRS, including concurrent- and predictive criterion validity, internal consistency, inter- and intra-rater reliability and sensitivity to change. This was done using data from four studies involving pharyngeal electrical stimulation in acute stroke patients with dysphagia, an individual patient data meta-analysis and unpublished studies (NCT03499574, NCT03700853). In addition, consensual- and content validity and the Minimal Clinically Important Difference (MCID) were assessed using anonymous surveys sent to UK-based Speech and Language Therapists (SLTs). Scores for consensual validity were mostly moderate (62.5-78%) to high or excellent (89-100%) for most scenarios. All but two assessments of content validity were excellent. In concurrent criterion validity assessments, DSRS was most closely associated with measures of radiological aspiration (penetration aspiration scale, Spearman rank rs?=?0.49, p?<?0.001) and swallowing (functional oral intake scale, FOIS, rs?=?-0.96, p?<?0.001); weaker but statistically significant associations were seen with impairment, disability and dependency. A similar pattern of relationships was seen for predictive criterion validity. Internal consistency (Cronbach's alpha) was either "good" or "excellent". Intra and inter-rater reliability were largely "excellent" (intraclass correlation >0.90). DSRS was sensitive to positive change during recovery (medians: 7, 4 and 1 at baseline and 2 and 13 weeks respectively) and in response to an intervention, pharyngeal electrical stimulation, in a published meta-analysis. The MCID was 1.0 and DSRS and FOIS scores may be estimated from each other. The DSRS appears to be a valid tool for grading the severity of swallowing impairment in patients with post stroke dysphagia and is appropriate for use in clinical research and clinical service delivery.

Project description:BackgroundIn developing the PatientsLikeMe online platform for patients with Multiple Sclerosis (MS), we required a patient-reported assessment of functional status that was easy to complete and identified disability in domains other than walking. Existing measures of functional status were inadequate, clinician-reported, focused on walking, and burdensome to complete. In response, we developed the Multiple Sclerosis Rating Scale (MSRS).MethodsWe adapted a clinician-rated measure, the Guy's Neurological Disability Scale, to a self-report scale and deployed it to an online community. As part of our validation process we reviewed discussions between patients, conducted patient cognitive debriefing, and made minor improvements to form a revised scale (MSRS-R) before deploying a cross-sectional survey to patients with relapsing-remitting MS (RRMS) on the PatientsLikeMe platform. The survey included MSRS-R and comparator measures: MSIS-29, PDDS, NARCOMS Performance Scales, PRIMUS, and MSWS-12.ResultsIn total, 816 RRMS patients responded (19% response rate). The MSRS-R exhibited high internal consistency (Cronbach's alpha = .86). The MSRS-R walking item was highly correlated with alternative walking measures (PDDS, ρ = .84; MSWS-12, ρ = .83; NARCOMS mobility question, ρ = .86). MSRS-R correlated well with comparison instruments and differentiated between known groups by PDDS disease stage and relapse burden in the past two years. Factor analysis suggested a single factor accounting for 51.5% of variance.ConclusionsThe MSRS-R is a concise measure of MS-related functional disability, and may have advantages for disease measurement over longer and more burdensome instruments that are restricted to a smaller number of domains or measure quality of life. Studies are underway describing the use of the instrument in contexts outside our online platform such as clinical practice or trials. The MSRS-R is released for use under creative commons license.

Project description:BackgroundMobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity.ObjectiveThis study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS.MethodsData was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Non-centrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation.ResultsIn total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05).ConclusionThe metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.