Dataset Information

Construct validity of the Physiotherapy Evidence Database (PEDro) quality scale for randomized trials: Item response theory and factor analyses.

ABSTRACT:

Background

There is an agreement that the methodological quality of randomized trials should be assessed in systematic reviews, but there is a debate on how this should be done. We conducted a construct validation study of the Physiotherapy Evidence Database (PEDro) scale, which is widely used to assess the quality of trials in physical therapy and rehabilitation.

Methods

We analyzed 345 trials that were included in Cochrane reviews and for which a PEDro summary score was available. We used one- and two-parameter logistic item response theory (IRT) models to study the psychometric properties of the PEDro scale and assessed the items' difficulty and discrimination parameters. We ran goodness of fit post estimations and examined the IRT unidimensionality assumption with a multidimensional IRT (MIRT) model.

Results

Out of a maximum of 10, the mean PEDro summary score was 5.46 (SD = 1.51). The allocation concealment and intention-to-treat scale items contributed most of the information on the underlying construct (with discriminations of 1.79 and 2.05, respectively) at similar difficulties (0.63 and 0.65, respectively). The other items provided little additional information and did not distinguish trials of different quality. There was substantial evidence of departure from the unidimensionality assumption, suggesting that the PEDro items relate to more than one latent trait.

Conclusions

Our findings question the construct validity of the PEDro scale to assess the methodological quality of clinical trials. PEDro summary scores should not be used; rather, the physiotherapy community should consider working with the individual items of the scale.

SUBMITTER: Albanese E

PROVIDER: S-EPMC7079093 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe Cochrane risk of bias (CROB) tool and Physiotherapy Evidence Database (PEDro) scale are used to evaluate risk of bias of randomized controlled trials. We assessed the level of agreement between the instruments.MethodsWe searched the Cochrane Library to identify trials included in systematic reviews evaluating physical therapy interventions. For trials that met our inclusion criteria (primary reference in Cochrane review, review used CROB (2008 version), indexed in PEDro), CROB items were extracted from the reviews and PEDro items and total score were downloaded from PEDro. Kappa statistics were used to determine the agreement between CROB and PEDro scale items that evaluate similar constructs (e.g., randomization). The total PEDro score was compared to the CROB summary score (% of items met) using an Intraclass Correlation Coefficient. Sensitivity analyses explored the impact of the CROB "unclear" category and variants of CROB blinding items. Kappa statistics were used to determine agreement between different thresholds for "acceptable" risk of bias between CROB and PEDro scale summary scores.ResultsWe included 1442 trials from 108 Cochrane reviews. Agreement was "moderate" for three of the six CROB and PEDro scale items that evaluate similar constructs (allocation concealment, participant blinding, assessor blinding; Kappa = 0.479-0.582). Agreement between the summary scores was "poor" (Intraclass Correlation Coefficient = 0.285). Agreement was highest when the CROB "unclear" category was collapsed with "high" and when participant, personnel and assessor blinding were evaluated separately in CROB. Agreement for different thresholds for "acceptable" risk of bias between CROB and PEDro summary scores was, at best, "fair".ConclusionThere was moderate agreement for half of the PEDro and CROB items that evaluate similar constructs. Interpretation of the CROB "unclear" category and variants of the CROB blinding items substantially influenced agreement. Either instrument can be used to quantify risk of bias, but they can't be used interchangeably.

Project description:BackgroundAlthough nontremor and tremor Part 3 Movement Disorder Society-Unified Parkinson's Disease Rating Scale items measure different impairment domains, their distinct progression and drug responsivity remain unstudied longitudinally. The total score may obscure important time-based and treatment-based changes occurring in the individual domains.ObjectiveUsing the unique advantages of item response theory (IRT), we developed novel longitudinal unidimensional and multidimensional models to investigate nontremor and tremor changes occurring in an interventional Parkinson's disease (PD) study.MethodWith unidimensional longitudinal IRT, we assessed the 33 Part 3 item data (22 nontremor and 10 tremor items) of 336 patients with early PD from the STEADY-PD III (Safety, Tolerability, and Efficacy Assessment of Isradipine for PD, placebo vs. isradipine) study. With multidimensional longitudinal IRT, we assessed the progression rates over time and treatment (in overall motor severity, nontremor, and tremor domains) using Markov Chain Monte Carlo implemented in Stan.ResultsRegardless of treatment, patients showed significant but different time-based deterioration rates for total motor, nontremor, and tremor scores. Isradipine was associated with additional significant deterioration over placebo in total score and nontremor scores, but not in tremor score. Further highlighting the 2 separate latent domains, nontremor and tremor severity changes were positively but weakly correlated (correlation coefficient, 0.108).ConclusionsLongitudinal IRT analysis is a novel statistical method highly applicable to PD clinical trials. It addresses limitations of traditional linear regression approaches and previous IRT investigations that either applied cross-sectional IRT models to longitudinal data or failed to estimate all parameters simultaneously. It is particularly useful because it can separate nontremor and tremor changes both over time and in response to treatment interventions.

Project description:ObjectivesThe aim of the present study was to investigate the construct validity, responsiveness and reliability of the Feline Grimace Scale (FGS) in kittens.MethodsA total of 36 healthy female kittens (aged 10 weeks to 6 months) were included in a prospective, randomized, blinded study. Video recordings of all kittens were made before and 1 and 2 h after ovariohysterectomy using an opioid-free injectable anesthetic protocol with or without multimodal analgesia. Additional recordings were taken before and 1 h after administration of rescue analgesia (buprenorphine 0.02 mg/kg IM) to painful kittens. Screenshots of facial images were collected from the video recordings for FGS scoring. Four observers blinded to treatment groups and time points scored 111 randomized images twice with a 5-week interval using the FGS. Five action units (AUs) were scored (ear position, orbital tightening, muzzle tension, whiskers position and head position; 0-2 each). Construct validity, responsiveness, and inter- and intra-rater reliability were evaluated using linear models with Benjamini-Hochberg correction, Wilcoxon signed-rank test and single intraclass correlation coefficients (ICCsingle), respectively (P <0.05).ResultsFGS total ratio scores were higher at 1 and 2 h after ovariohysterectomy (median [interquartile range, IQR]: 0.30 [0.20-0.40] and 0.30 [0.20-0.40], respectively) than at baseline (median [IQR]: 0.10 [0.00-0.30]) (P <0.001). FGS total ratio scores were lower after the administration of rescue analgesia (median [IQR] before and after rescue analgesia) 0.40 [0.20-0.50] and 0.20 [0.10-0.38], respectively (P <0.001). Inter-rater ICCsingle was 0.68 for the FGS total ratio scores and 0.35-0.70 for all AUs considered individually. Intra-rater ICCsingle was 0.77-0.91 for the FGS total ratio scores and 0.55-1.00 for all AUs considered individually.Conclusions and relevanceThe FGS is a valid and responsive acute pain-scoring instrument with moderate inter-rater reliability and good to excellent intra-rater reliability in kittens.