Dataset Information

Validation of a Musculoskeletal Digital Assessment Routing Tool: Protocol for a Pilot Randomized Crossover Noninferiority Trial.

ABSTRACT:

Background

Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing demand for health care use. Digital technologies to improve health care outcomes and efficiency are considered a priority; however, innovations are rarely tested with sufficient rigor in clinical trials, which is the gold standard for clinical proof of safety and efficacy. We have developed a new musculoskeletal digital assessment routing tool (DART) that allows users to self-assess and be directed to the right care. DART requires validation in a real-world setting before implementation.

Objective

This pilot study aims to assess the feasibility of a future trial by exploring the key aspects of trial methodology, assessing the procedures, and collecting exploratory data to inform the design of a definitive randomized crossover noninferiority trial to assess DART safety and effectiveness.

Methods

We will collect data from 76 adults with a musculoskeletal condition presenting to general practitioners within a National Health Service (NHS) in England. Participants will complete both a DART assessment and a physiotherapist-led triage, with the order determined by randomization. The primary analysis will involve an absolute agreement intraclass correlation (A,1) estimate with 95% CI between DART and the clinician for assessment outcomes signposting to condition management pathways. Data will be collected to allow the analysis of participant recruitment and retention, randomization, allocation concealment, blinding, data collection process, and bias. In addition, the impact of trial burden and potential barriers to intervention delivery will be considered. The DART user satisfaction will be measured using the system usability scale.

Results

A UK NHS ethics submission was done during June 2021 and is pending approval; recruitment will commence in early 2022, with data collection anticipated to last for 3 months. The results will be reported in a follow-up paper in 2022.

Conclusions

This study will inform the design of a randomized controlled crossover noninferiority study that will provide evidence concerning mobile health DART system clinical signposting in an NHS setting before real-world implementation. Success should produce evidence of a safe, effective system with good usability, potentially facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This rigorous approach to mobile health system testing could be used as a guide for other developers of similar applications.

Trial registration

ClinicalTrials.gov NCT04904029; http://clinicaltrials.gov/ct2/show/NCT04904029.

International registered report identifier (irrid)

PRR1-10.2196/31541.

SUBMITTER: Lowe C

PROVIDER: S-EPMC8713101 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMusculoskeletal conditions account for 16% of global disability, resulting in a negative effect on patients and increasing demand for health care use. Triage directing patients to appropriate level intervention improving health outcomes and efficiency has been prioritized. We developed a musculoskeletal digital assessment routing tool (DART) mobile health (mHealth) system, which requires evaluation prior to implementation. Such innovations are rarely rigorously tested in clinical trials-considered the gold standard for evaluating safety and efficacy. This pilot study is a precursor to a trial assessing DART performance with a physiotherapist-led triage assessment.ObjectiveThe study aims to evaluate trial design, assess procedures, and collect exploratory data to establish the feasibility of delivering an adequately powered, definitive randomized trial, assessing DART safety and efficacy in an NHS primary care setting.MethodsA crossover, noninferiority pilot trial using an integrated knowledge translation approach within a National Health Service England primary care setting. Participants were patients seeking assessment for a musculoskeletal condition, completing a DART assessment and the history-taking element of a face-to-face physiotherapist-led triage in a randomized order. The primary outcome was agreement between DART and physiotherapist triage recommendation. Data allowed analysis of participant recruitment and retention, randomization, blinding, study burden, and potential barriers to intervention delivery. Participant satisfaction was measured using the System Usability Scale.ResultsOver 8 weeks, 129 patients were invited to participate. Of these, 92% (119/129) proceeded to eligibility assessment, with 60% (78/129) meeting the inclusion criteria and being randomized into each intervention arm (39/39). There were no dropouts and data were analyzed for all 78 participants. Agreement between physiotherapist and DART across all participants and all primary triage outcomes was 41% (32/78; 95% CI 22-45), intraclass correlation coefficient 0.37 (95% CI 0.16-0.55), indicating that the reliability of DART was poor to moderate. Feedback from the clinical service team led to an adjusted analysis yielding of 78% (61/78; 95% CI 47-78) and an intraclass correlation coefficient of 0.57 (95% CI 0.40-0.70). Participant satisfaction was measured quantitively using amalgamated System Usability Scale scores (n=78; mean score 84.0; 90% CI +2.94 to -2.94), equating to an "excellent" system. There were no study incidents, and the trial burden was acceptable.ConclusionsPhysiotherapist-DART agreement of 78%, with no adverse triage decisions and high patient satisfaction, was sufficient to conclude DART had the potential to improve the musculoskeletal pathway. Study validity was enhanced by the recruitment of real-world patients and using an integrated knowledge translation approach. Completion of a context-specific consensus process is recommended to provide definitive definitions of safety criteria, range of appropriateness, noninferiority margin, and sample size. This pilot demonstrated an adequately powered definitive trial is feasible, which would provide evidence of DART safety and efficacy, ultimately informing potential for DART implementation.Trial registrationClinicalTrials.gov NCT04904029; http://clinicaltrials.gov/ct2/show/NCT04904029.International registered report identifier (irrid)RR2-10.2196/31541.

Project description:BackgroundMusculoskeletal disorders negatively affect millions of patients worldwide, placing significant demand on health care systems. Digital technologies that improve clinical outcomes and efficiency across the care pathway are development priorities. We developed the musculoskeletal Digital Assessment Routing Tool (DART) to enable self-assessment and immediate direction to the right care.ObjectiveWe aimed to assess and resolve all serious DART usability issues to create a positive user experience and enhance system adoption before conducting randomized controlled trials for the integration of DART into musculoskeletal management pathways.MethodsAn iterative, convergent mixed methods design was used, with 22 adult participants assessing 50 different clinical presentations over 5 testing rounds across 4 DART iterations. Participants were recruited using purposive sampling, with quotas for age, habitual internet use, and English-language ability. Quantitative data collection was defined by the constructs within the International Organization for Standardization 9241-210-2019 standard, with user satisfaction measured by the System Usability Scale. Study end points were resolution of all grade 1 and 2 usability problems and a mean System Usability Scale score of ≥80 across a minimum of 3 user group sessions.ResultsAll participants (mean age 48.6, SD 15.2; range 20-77 years) completed the study. Every assessment resulted in a recommendation with no DART system errors and a mean completion time of 5.2 (SD 4.44, range 1-18) minutes. Usability problems were reduced from 12 to 0, with trust and intention to act improving during the study. The relationship between eHealth literacy and age, as explored with a scatter plot and calculation of the Pearson correlation coefficient, was performed for all participants (r=-0.2; 20/22, 91%) and repeated with a potential outlier removed (r=-0.23), with no meaningful relationships observed or found for either. The mean satisfaction for daily internet users was highest (19/22, 86%; mean 86.5, SD 4.48; 90% confidence level [CL] 1.78 or -1.78), with nonnative English speakers (6/22, 27%; mean 78.1, SD 4.60; 90% CL 3.79 or -3.79) and infrequent internet users scoring the lowest (3/22, 14%; mean 70.8, SD 5.44; 90% CL 9.17 or -9.17), although the CIs overlap. The mean score across all groups was 84.3 (SD 4.67), corresponding to an excellent system, with qualitative data from all participants confirming that DART was simple to use.ConclusionsAll serious DART usability issues were resolved, and a good level of satisfaction, trust, and willingness to act on the DART recommendation was demonstrated, thus allowing progression to randomized controlled trials that assess safety and effectiveness against usual care comparators. The iterative, convergent mixed methods design proved highly effective in fully evaluating DART from a user perspective and could provide a blueprint for other researchers of mobile health systems.International registered report identifier (irrid)RR2-10.2196/27205.

Project description:BackgroundMusculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.ObjectiveThis study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.MethodsWe will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.ResultsThis study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.ConclusionsThis study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.International registered report identifier (irrid)DERR1-10.2196/27205.

Project description:BackgroundResource-rich countries are facing the challenge of aging societies, a high risk of dependence, and a high cost of care. Researchers attempted to address these issues by using cost-efficient, innovative technology to promote healthy aging and regain functionality. After an injury, efficient rehabilitation is crucial to promote returning home and prevent institutionalization. However, there is often a lack of motivation to carry out physical therapies. Consequently, there is a growing interest in testing new approaches like gamified physical rehabilitation to achieve functional targets and prevent rehospitalization.ObjectiveThe purpose of this study is to assess the effectiveness of a personal mobility device compared with standard care in the rehabilitation treatment of patients with musculoskeletal issues.MethodsA total of 57 patients aged 67-95 years were randomly assigned to the intervention group (n=35) using the gamified rehabilitation equipment 3 times a week or to the control group (n=22) receiving usual standard care. Due to dropout, only 41 patients were included in the postintervention analysis. Outcome measures included the short physical performance battery (SPPB), isometric hand grip strength (IHGS), functional independence measure (FIM), and the number of steps.ResultsA noninferiority related to the primary outcome (SPPB) was identified during the hospital stay, and no significant differences were found between the control and intervention groups for any of the secondary outcomes (IHGS, FIM, or steps), which demonstrates the potential of the serious game-based intervention to be as effective as the standard physical rehabilitation at the hospital. The analysis by mixed-effects regression on SPPB showed a group×time interaction (SPPB_I_t1=-0.77, 95% CI -2.03 to 0.50, P=.23; SPPB_I_t2=0.21, 95% CI -1.07 to 0.48, P=.75). Although not significant, a positive IHGS improvement of more than 2 kg (Right: 2.52 kg, 95% CI -0.72 to 5.37, P=.13; Left: 2.43 kg, 95% CI -0.18 to 4.23, P=.07) for the patient from the intervention group was observed.ConclusionsSerious game-based rehabilitation could potentially be an effective alternative for older patients to regain their functional capacities.Trial registrationClinicalTrials.gov NCT03847454; https://clinicaltrials.gov/ct2/show/NCT03847454.

Project description:BackgroundMany people with harmful addictive behaviors may not meet formal diagnostic thresholds for a disorder. A dimensional approach, by contrast, including clinical and community samples, is potentially key to early detection, prevention, and intervention. Importantly, while neurocognitive dysfunction underpins addictive behaviors, established assessment tools for neurocognitive assessment are lengthy and unengaging, difficult to administer at scale, and not suited to clinical or community needs. The BrainPark Assessment of Cognition (BrainPAC) Project sought to develop and validate an engaging and user-friendly digital assessment tool purpose-built to comprehensively assess the main consensus-driven constructs underpinning addictive behaviors.ObjectiveThe purpose of this study was to psychometrically validate a gamified battery of consensus-based neurocognitive tasks against standard laboratory paradigms, ascertain test-retest reliability, and determine their sensitivity to addictive behaviors (eg, alcohol use) and other risk factors (eg, trait impulsivity).MethodsGold standard laboratory paradigms were selected to measure key neurocognitive constructs (Balloon Analogue Risk Task [BART], Stop Signal Task [SST], Delay Discounting Task [DDT], Value-Modulated Attentional Capture [VMAC] Task, and Sequential Decision-Making Task [SDT]), as endorsed by an international panel of addiction experts; namely, response selection and inhibition, reward valuation, action selection, reward learning, expectancy and reward prediction error, habit, and compulsivity. Working with game developers, BrainPAC tasks were developed and validated in 3 successive cohorts (total N=600) and a separate test-retest cohort (N=50) via Mechanical Turk using a cross-sectional design.ResultsBrainPAC tasks were significantly correlated with the original laboratory paradigms on most metrics (r=0.18-0.63, P<.05). With the exception of the DDT k function and VMAC total points, all other task metrics across the 5 tasks did not differ between the gamified and nongamified versions (P>.05). Out of 5 tasks, 4 demonstrated adequate to excellent test-retest reliability (intraclass correlation coefficient 0.72-0.91, P<.001; except SDT). Gamified metrics were significantly associated with addictive behaviors on behavioral inventories, though largely independent of trait-based scales known to predict addiction risk.ConclusionsA purpose-built battery of digitally gamified tasks is sufficiently valid for the scalable assessment of key neurocognitive processes underpinning addictive behaviors. This validation provides evidence that a novel approach, purported to enhance task engagement, in the assessment of addiction-related neurocognition is feasible and empirically defensible. These findings have significant implications for risk detection and the successful deployment of next-generation assessment tools for substance use or misuse and other mental disorders characterized by neurocognitive anomalies related to motivation and self-regulation. Future development and validation of the BrainPAC tool should consider further enhancing convergence with established measures as well as collecting population-representative data to use clinically as normative comparisons.

Project description:IntroductionDigital diagnostic decision support tools promise to accelerate diagnosis and increase health care efficiency in rheumatology. Rheumatic? is an online tool developed by specialists in rheumatology and general medicine together with patients and patient organizations. It calculates a risk score for several rheumatic diseases. We ran a pilot study retrospectively testing Rheumatic? for its ability to differentiate symptoms from existing or emerging immune-mediated rheumatic diseases from other rheumatic and musculoskeletal complaints and disorders in patients visiting rheumatology clinics.Materials and methodsThe performance of Rheumatic? was tested using in three university rheumatology centers: (A) patients at Risk for RA (Karolinska Institutet, n = 50 individuals with musculoskeletal complaints and anti-citrullinated protein antibody positivity) (B) patients with early joint swelling [dataset B (Erlangen) n = 52]. (C) Patients with early arthritis where the clinician considered it likely to be of auto-immune origin [dataset C (Leiden) n = 73]. In dataset A we tested whether Rheumatic? could predict the development of arthritis. In dataset B and C we tested whether Rheumatic? could predict the development of an immune-mediated rheumatic diseases. We examined the discriminative power of the total score with the Wilcoxon rank test and the area-under-the-receiver-operating-characteristic curve (AUC-ROC). Next, we calculated the test characteristics for these patients passing the first or second expert-based Rheumatic? scoring threshold.ResultsThe total test scores differentiated between: (A) Individuals developing arthritis or not, median 245 vs. 163, P < 0.0001, AUC-ROC = 75.3; (B) patients with an immune-mediated arthritic disease or not median 191 vs. 107, P < 0.0001, AUC-ROC = 79.0; but less patients with an immune-mediated arthritic disease or not amongst those where the clinician already considered an immune mediated disease most likely (median 262 vs. 212, P < 0.0001, AUC-ROC = 53.6). Threshold-1 (advising to visit primary care doctor) was highly specific in dataset A and B (0.72, 0.87, and 0.23, respectively) and sensitive (0.67, 0.61, and 0.67). Threshold-2 (advising to visit rheumatologic care) was very specific in all three centers but not very sensitive: specificity of 1.0, 0.96, and 0.91, sensitivity 0.05, 0.07, 0.14 in dataset A, B, and C, respectively.ConclusionRheumatic? is a web-based patient-centered multilingual diagnostic tool capable of differentiating immune-mediated rheumatic conditions from other musculoskeletal problems. The current scoring system needs to be further optimized.