Dataset Information

Validation of a Musculoskeletal Digital Assessment Routing Tool: Protocol for a Pilot Randomized Crossover Noninferiority Trial.

ABSTRACT:

Background

Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing demand for health care use. Digital technologies to improve health care outcomes and efficiency are considered a priority; however, innovations are rarely tested with sufficient rigor in clinical trials, which is the gold standard for clinical proof of safety and efficacy. We have developed a new musculoskeletal digital assessment routing tool (DART) that allows users to self-assess and be directed to the right care. DART requires validation in a real-world setting before implementation.

Objective

This pilot study aims to assess the feasibility of a future trial by exploring the key aspects of trial methodology, assessing the procedures, and collecting exploratory data to inform the design of a definitive randomized crossover noninferiority trial to assess DART safety and effectiveness.

Methods

We will collect data from 76 adults with a musculoskeletal condition presenting to general practitioners within a National Health Service (NHS) in England. Participants will complete both a DART assessment and a physiotherapist-led triage, with the order determined by randomization. The primary analysis will involve an absolute agreement intraclass correlation (A,1) estimate with 95% CI between DART and the clinician for assessment outcomes signposting to condition management pathways. Data will be collected to allow the analysis of participant recruitment and retention, randomization, allocation concealment, blinding, data collection process, and bias. In addition, the impact of trial burden and potential barriers to intervention delivery will be considered. The DART user satisfaction will be measured using the system usability scale.

Results

A UK NHS ethics submission was done during June 2021 and is pending approval; recruitment will commence in early 2022, with data collection anticipated to last for 3 months. The results will be reported in a follow-up paper in 2022.

Conclusions

This study will inform the design of a randomized controlled crossover noninferiority study that will provide evidence concerning mobile health DART system clinical signposting in an NHS setting before real-world implementation. Success should produce evidence of a safe, effective system with good usability, potentially facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This rigorous approach to mobile health system testing could be used as a guide for other developers of similar applications.

Trial registration

ClinicalTrials.gov NCT04904029; http://clinicaltrials.gov/ct2/show/NCT04904029.

International registered report identifier (irrid)

PRR1-10.2196/31541.

SUBMITTER: Lowe C

PROVIDER: S-EPMC8713101 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMusculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.ObjectiveThis study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.MethodsWe will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.ResultsThis study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.ConclusionsThis study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.International registered report identifier (irrid)DERR1-10.2196/27205.

Project description:BACKGROUND:Defining the transition from relapsing-remitting multiple sclerosis (RRMS) to secondary progressive multiple sclerosis (SPMS) can be challenging and delayed. A digital tool (MSProDiscuss) was developed to facilitate physician-patient discussion in evaluating early, subtle signs of multiple sclerosis (MS) disease progression representing this transition. OBJECTIVE:This study aimed to determine cut-off values and corresponding sensitivity and specificity for predefined scoring algorithms, with or without including Expanded Disability Status Scale (EDSS) scores, to differentiate between RRMS and SPMS patients and to evaluate psychometric properties. METHODS:Experienced neurologists completed the tool for patients with confirmed RRMS or SPMS and those suspected to be transitioning to SPMS. In addition to age and EDSS score, each patient's current disease status (disease activity, symptoms, and its impacts on daily life) was collected while completing the draft tool. Receiver operating characteristic (ROC) curves determined optimal cut-off values (sensitivity and specificity) for the classification of RRMS and SPMS. RESULTS:Twenty neurologists completed the draft tool for 198 patients. Mean scores for patients with RRMS (n=89), transitioning to SPMS (n=47), and SPMS (n=62) were 38.1 (SD 12.5), 55.2 (SD 11.1), and 69.6 (SD 12.0), respectively (P<.001, each between-groups comparison). Area under the ROC curve (AUC) including and excluding EDSS were for RRMS (including) AUC 0.91, 95% CI 0.87-0.95, RRMS (excluding) AUC 0.88, 95% CI 0.84-0.93, SPMS (including) AUC 0.91, 95% CI 0.86-0.95, and SPMS (excluding) AUC 0.86, 95% CI 0.81-0.91. In the algorithm with EDSS, the optimal cut-off values were ?51.6 for RRMS patients (sensitivity=0.83; specificity=0.82) and ?58.9 for SPMS patients (sensitivity=0.82; specificity=0.84). The optimal cut-offs without EDSS were ?46.3 and ?57.8 and resulted in similar high sensitivity and specificity (0.76-0.86). The draft tool showed excellent interrater reliability (intraclass correlation coefficient=.95). CONCLUSIONS:The MSProDiscuss tool differentiated RRMS patients from SPMS patients with high sensitivity and specificity. In clinical practice, it may be a useful tool to evaluate early, subtle signs of MS disease progression indicating the evolution of RRMS to SPMS. MSProDiscuss will help assess the current level of progression in an individual patient and facilitate a more informed physician-patient discussion.

Project description:BACKGROUND:Patient-reported outcomes (PROs) translate subjective outcomes into objective data that can be quantified and analyzed. Nevertheless, the use of PROs in their traditional paper format is not practical for clinical practice due to limitations associated with the analysis and management of the data. To address the need for a viable way to group and utilize the main functioning assessment tools in the field of musculoskeletal disorders, the Physiotherapy Questionnaires app was developed. OBJECTIVE:This study aims to explain the development of the app, to validate it using two questionnaires, and to analyze whether participants prefer to use the app or the paper version of the questionnaires. METHODS:In the first stage, the app for an Android operational system was developed. In the second stage, the aim was to select questionnaires that were most often used in musculoskeletal clinical practice and research. The Foot and Ankle Outcome Score (FAOS) and American Orthopaedic Foot and Ankle Society (AOFAS) questionnaire were selected to validate the app. In total, 50 participants completed the paper and app versions of the AOFAS and 50 completed the FAOS. The study's outcomes were the correlation of the data between the paper and app versions as well as the preference of the participants between the two versions. RESULTS:The app was approved by experts after the adaptations of the layout for mobile phones and a total of 18 questionnaires were included in the app. Moreover, the app allows the generation of PDF and Excel files with the patients' data. In regards to validity, the mean of the total scores of the FAOS were 91.54% (SD 8.86%) for the paper version and 91.74% (SD 9.20%) for the app. There was no statistically significant differences in the means of the total scores or the subscales (P=.11-.94). The mean total scores for the AOFAS were 93.94 (SD 8.47) for the paper version and 93.96 (SD 8.48) for the app. No statistically significant differences were found for the total scores for the AOFAS or the subscales (P>.99). The app showed excellent agreement with the paper version of the FAOS, with an ICC value of 0.98 for the total score (95% CI 0.98-0.99), which was also found for the AOFAS with the ICC for the total score of 0.99 (95% CI 0.98-0.99). For compliance, 72% (36/50) of the participants in the FAOS group and 94% (47/50) in the AOFAS group preferred the app version. CONCLUSIONS:The Physiotherapy Questionnaires app showed validity and high levels of compliance for the FAOS and AOFAS, which indicates it is not inferior to the paper version of these two questionnaires and confirms its viability and feasibility for use in clinical practice.