Dataset Information

A unified multi-level model approach to assessing patient responsiveness including; return to normal, minimally important differences and minimal clinically important improvement for patient reported outcome measures.

ABSTRACT: OBJECTIVE:This article reviews and compares four commonly used approaches to assess patient responsiveness with a treatment or therapy (return to normal (RTN), minimal important difference (MID), minimal clinically important improvement (MCII), OMERACT-OARSI [Outcome Measures in Rheumatology-Osteoarthris Reseach Society International] (OO)) and demonstrates how each of the methods can be formulated in a multilevel modelling (MLM) framework. DESIGN:Cohort study. SETTING:A cohort of patients undergoing total hip and knee replacement were recruited from a single UK National Health Service hospital. POPULATION:400 patients from the Arthroplasty Pain Experience cohort study undergoing total hip (n=210) and knee (n=190) replacement who completed the Intermittent and Constant Osteoarthritis Pain questionnaire prior to surgery and then at 3, 6 and 12 months after surgery. PRIMARY OUTCOMES:The primary outcome was defined as a response to treatment following total hip or knee replacement. We compared baseline scores, change scores and proportion of individuals defined as 'responders' using traditional and MLM approaches with patient responsiveness. RESULTS:Using existing approaches, baseline and change scores are underestimated, and the variance of baseline and change scores overestimated in comparison with MLM approaches. MLM increases the proportion of individuals defined as responding in RTN, MID and OO criteria compared with existing approaches. Using MLM with the MCII criteria reduces the number of individuals identified as responders. CONCLUSION:MLM improves the estimation of the SD of baseline and change scores by explicitly incorporating measurement error into the model and avoiding regression to the mean when making individual predictions. Using refined definitions of responsiveness may lead to a reduction in misclassification when attempting to predict who does and does not respond to an intervention and clarifies the similarities between existing methods.

SUBMITTER: Sayers A

PROVIDER: S-EPMC5577889 | biostudies-other | 2017 Jul

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

A unified multi-level model approach to assessing patient responsiveness including; return to normal, minimally important differences and minimal clinically important improvement for patient reported outcome measures.

Sayers Adrian A Wylde Vikki V Lenguerrand Erik E Gooberman-Hill Rachael R Dawson Jill J Beard David D Price Andrew A Blom Ashley W AW

BMJ open 20170721 7

<h4>Objective</h4>This article reviews and compares four commonly used approaches to assess patient responsiveness with a treatment or therapy (return to normal (RTN), minimal important difference (MID), minimal clinically important improvement (MCII), OMERACT-OARSI [Outcome Measures in Rheumatology-Osteoarthris Reseach Society International] (OO)) and demonstrates how each of the methods can be formulated in a multilevel modelling (MLM) framework.<h4>Design</h4>Cohort study.<h4>Setting</h4>A co ...[more]

PMID: 28733296

Similar Datasets

Project description:BACKGROUND:The Roland Morris Disability Questionnaire (RMDQ), visual analog scale (VAS) of pain intensity, and numerical rating scale (NRS) are among the most commonly used outcome measures in trials of interventions for low back pain. Their use in paper form is well established. Few data are available on the metric properties of electronic counterparts. OBJECTIVE:The goal of our research was to establish responsiveness, minimally important change (MIC) thresholds, reliability, and minimal detectable change at a 95% level (MDC95) for electronic versions of the RMDQ, VAS, and NRS as delivered via iOS and Android apps and Web browser. METHODS:We recruited adults with low back pain who visited osteopaths. We invited participants to complete the eRMDQ, eVAS, and eNRS at baseline, 1 week, and 6 weeks along with a health transition question at 1 and 6 weeks. Data from participants reporting recovery were used in MIC and responsiveness analyses using receiver operator characteristic (ROC) curves and areas under the ROC curves (AUCs). Data from participants reporting stability were used for analyses of reliability (intraclass correlation coefficient [ICC] agreement) and MDC95. RESULTS:We included 442 participants. At 1 and 6 weeks, ROC AUCs were 0.69 (95% CI 0.59 to 0.80) and 0.67 (95% CI 0.46 to 0.87) for the eRMDQ, 0.69 (95% CI 0.58 to 0.80) and 0.74 (95% CI 0.53 to 0.95) for the eVAS, and 0.73 (95% CI 0.66 to 0.80) and 0.81 (95% CI 0.69 to 0.92) for the eNRS, respectively. Associated MIC thresholds were estimated as 1 (0 to 2) and 2 (-1 to 5), 13 (9 to 17) and 7 (-12 to 26), and 2 (1 to 3) and 1 (0 to 2) points, respectively. Over a 1-week period in participants categorized as "stable" and "about the same" using the transition question, ICCs were 0.87 (95% CI 0.66 to 0.95) and 0.84 (95% CI 0.73 to 0.91) for the eRMDQ with MDC95 of 4 and 5, 0.31 (95% CI -0.25 to 0.71) and 0.61 (95% CI 0.36 to 0.77) for the eVAS with MDC95 of 39 and 34, and 0.52 (95% CI 0.14 to 0.77) to 0.67 (95% CI 0.51 to 0.78) with MDC95 of 4 and 3 for the eNRS. CONCLUSIONS:The eRMDQ was reliable with borderline adequate responsiveness. The eNRS was responsive with borderline reliability. While the eVAS had adequate responsiveness, it did not have an attractive reliability profile. Thus, the eNRS might be preferred over the eVAS for measuring pain intensity. The observed electronic outcome measures' metric properties are within the ranges of values reported in the literature for their paper counterparts and are adequate for measuring changes in a low back pain population.

Project description:BACKGROUND:Idiopathic pulmonary fibrosis (IPF) specific version of St. George's Respiratory Questionnaire (SGRQ-I) and King's Brief Interstitial Lung Disease questionnaire (K-BILD) are validated health-related quality of life (HRQL) instruments, but no or limited data exist on their responsiveness and minimal clinically important difference (MCID). The objectives of this study were to assess responsiveness of SGRQ-I and K-BILD and determine MCID separately for deterioration and improvement in a large, prospective cohort of patients with IPF in a real-world setting. METHODS:Consecutive patients with IPF were recruited. SGRQ-I, K-BILD, SGRQ, Shortness of Breath Questionnaire, pulmonary function tests and 6-min walk test measurements were obtained at baseline and at six and 12?months; at six and 12?months, patients also completed Global Rating of Change Scales. Responsiveness was assessed using correlation coefficients and linear regression. Cox regression was used for mortality analyses. MCID was estimated using receiver operating characteristic curves with separate analyses for improvement and deterioration. RESULTS:A total of 150 IPF patients were included and 124 completed the 12-month follow-up. Based on all HRQL anchors and most physiological anchors, responsiveness analyses supported the evidence pointing towards SGRQ-I and K-BILD as responsive instruments. Multivariate analyses showed an association between SGRQ-I and mortality (HR: 1.18, 95% CI: 1.02 to 1.36, p?=?0.03) and a trend was found for K-BILD (HR: 0.82, 95% CI: 0.64 to 1.05, p?=?0.12). MCID was estimated for all domains of SGRQ-I and K-BILD. MCID for improvement differed from deterioration for both SGRQ-I Total (3.9 and 4.9) and K-BILD Total (4.7 and 2.7). CONCLUSIONS:SGRQ-I and K-BILD were responsive to change concerning both HRQL and most physiological anchors. MCID was determined separately for improvement and deterioration, resulting in different estimates; especially a smaller estimate for deterioration compared to improvement in K-BILD. TRIAL REGISTRATION:Clinicaltrials.gov, no. NCT02818712. Registered 30 June 2016.

Project description:BACKGROUND:Using a real dataset, we highlighted several major methodological issues raised by the estimation of the Minimal Clinically Important Difference (MCID) of a Patient-Reported Outcomes instrument. We especially considered the management of missing data and the use of more than two times of measurement. While inappropriate missing data management and inappropriate use of multiple time points can lead to loss of precision and/or bias in MCID estimation, these issues are almost never dealt with and require cautious considerations in the context of MCID estimation. METHODS:We used the LIGALONGO study (French Randomized Controlled Trial). We estimated MCID on the SF-36 General Health score by comparing many methods (distribution or anchor-based). Different techniques for imputation of missing data were performed (simple and multiple imputations). We also consider all measurement occasions by longitudinal modeling, and the dependence of the score difference on baseline. RESULTS:Three hundred ninety-three patients were studied. With distribution-based methods, a great variability in MCID was observed (from 3 to 26 points for improvement). Only 0.2 SD and 1/3 SD distribution methods gave MCID values consistent with anchor-based methods (from 4 to 7 points for improvement). The choice of missing data imputation technique clearly had an impact on MCID estimates. Simple imputation by mean score seemed to lead to out-of-range estimate, but as missing not at random mechanism can be hypothesized, even multiple imputations techniques can have led to an slight underestimation of MCID. Using 3 measurement occasions for improvement led to an increase in precision but lowered estimates. CONCLUSION:This practical example illustrates the substantial impact of some methodological issues that are usually never dealt with for MCID estimation. Simulation studies are needed to investigate those issues. TRIAL REGISTRATION:NCT01240772 (ClinicalTrials.gov) registered on November 15, 2010.

Project description:ObjectivesTo systematically review studies using machine learning (ML) algorithms to predict whether patients undergoing total knee or total hip arthroplasty achieve an improvement as high or higher than the minimal clinically important differences (MCID) in patient reported outcome measures (PROMs) (classification problem).MethodsStudies were eligible to be included in the review if they collected PROMs both pre- and postintervention, reported the method of MCID calculation and applied ML. ML was defined as a family of models which automatically learn from data when selecting features, identifying nonlinear relations or interactions. Predictive performance must have been assessed using common metrics. Studies were searched on MEDLINE, PubMed Central, Web of Science Core Collection, Google Scholar and Cochrane Library. Study selection and risk of bias assessment (ROB) was conducted by two independent researchers.Results517 studies were eligible for title and abstract screening. After screening title and abstract, 18 studies qualified for full-text screening. Finally, six studies were included. The most commonly applied ML algorithms were random forest and gradient boosting. Overall, eleven different ML algorithms have been applied in all papers. All studies reported at least fair predictive performance, with two reporting excellent performance. Sample size varied widely across studies, with 587 to 34,110 individuals observed. PROMs also varied widely across studies, with sixteen applied to TKA and six applied to THA. There was no single PROM utilized commonly in all studies. All studies calculated MCIDs for PROMs based on anchor-based or distribution-based methods or referred to literature which did so. Five studies reported variable importance for their models. Two studies were at high risk of bias.DiscussionNo ML model was identified to perform best at the problem stated, nor can any PROM said to be best predictable. Reporting standards must be improved to reduce risk of bias and improve comparability to other studies.

Project description:OBJECTIVE:Valid, reliable, and clinically relevant outcome measures are necessary in clinical studies of snake envenomation. The aim of this study was to evaluate the psychometric (validity and reliability) and clinimetric (minimal clinically important difference [MCID]) properties of the Patient-Specific Functional Scale (PSFS) in snakebite envenomation. METHODS:We performed a secondary analysis of two existing snakebite trials that measured clinical outcomes using the PSFS as well as other quality of life and functional assessments. Data were collected at 3, 7, 10, and 17 days. Reliability was determined using Cronbach's alpha for internal consistency and the intraclass correlation coefficient (ICC) for temporal stability at 10 and 17 days. Validity was assessed using concurrent validity correlating with the other assessments. The MCID was evaluated using the following criteria: (1) the distribution of stable patients according to both standard error of measurement (SEM) and responsiveness techniques, and (2) anchor-based methods to compare between individuals and to detect discriminant ability of a positive change with a receiver operator characteristic (ROC) curve and optimal cutoff point. RESULTS:A total of 86 patients were evaluated in this study. The average PSFS scores were 5.37 (SD 3.23), 7.95 (SD 2.22), and 9.12 (SD 1.37) at 3, 7, and 10 days, respectively. Negligible floor effect was observed (maximum of 8% at 3 days); however, a ceiling effect was observed at 17 days (25%). The PSFS showed good reliability with an internal consistency of 0.91 (Cronbach's alpha) (95% CI 0.88, 0.95) and a temporal stability of 0.83 (ICC) (95% CI 0.72, 0.89). The PSFS showed a strong positive correlation with quality of life and functional assessments. The MCID was approximately 1.0 for all methods. CONCLUSIONS:With an MCID of approximately 1 point, the PSFS is a valid and reliable tool to assess quality of life and functionality in patients with snake envenomation.

Dataset Information

A unified multi-level model approach to assessing patient responsiveness including; return to normal, minimally important differences and minimal clinically important improvement for patient reported outcome measures.

Publications

A unified multi-level model approach to assessing patient responsiveness including; return to normal, minimally important differences and minimal clinically important improvement for patient reported outcome measures.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets