Dataset Information

Interrater Reliability of Four Sensory Measures in People with Multiple Sclerosis.

ABSTRACT:

Background

Sensory disturbances are a major problem for people with multiple sclerosis (MS), and up to 80% of people with MS present with various sensory deficits. To date, only one study has investigated the reliability of sensory measures in people with MS. We sought to determine the interrater reliability of the verbal analogue scale (VAS), the Erasmus MC modifications to the revised Nottingham Sensory Assessment (EmNSA), Semmes-Weinstein monofilaments (SWMs), and the neurothesiometer (NT) in people with MS.

Methods

A random sample of 34 people with MS who could walk independently with or without a device was tested by two raters on the same day. For categorical data, percentage agreement, Cohen's kappa, and prevalence-adjusted bias-adjusted kappa were used. For continuous data, interclass correlation coefficient (ICC[2,1]) with 95% confidence intervals (95% CIs), Bland and Altman analysis, and standard error of measurement (SEM) were calculated.

Results

For NT, ICC(2,1) values were good, with the highest for first metatarsophalangeal joint (ICC[2,1] = 0.84, 95% CI = 0.69-0.92, SEM = 4.98). The highest ICC(2,1) for VAS was for the question relating to feeling numbness in the hand (ICC[2,1] = 0.93, 95% CI = 0.86-0.96, SEM = 0.64). Findings for EmNSA and SWMs need further verification owing to possible ceiling effects.

Conclusions

The NT and VAS had good interrater reliability and should be considered for measuring sensation in ambulatory people with MS. Findings for EmNSA and SWMs revealed either questionable or poor reliability, suggesting the need for further investigation.

SUBMITTER: Uszynski M

PROVIDER: S-EPMC4849401 | biostudies-literature | 2016 Mar-Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interrater Reliability of Four Sensory Measures in People with Multiple Sclerosis.

Uszynski Marcin M Purtill Helen H Coote Susan S

International journal of MS care 20160301 2

<h4>Background</h4>Sensory disturbances are a major problem for people with multiple sclerosis (MS), and up to 80% of people with MS present with various sensory deficits. To date, only one study has investigated the reliability of sensory measures in people with MS. We sought to determine the interrater reliability of the verbal analogue scale (VAS), the Erasmus MC modifications to the revised Nottingham Sensory Assessment (EmNSA), Semmes-Weinstein monofilaments (SWMs), and the neurothesiometer ...[more]

PMID: 27134582

Similar Datasets

Project description:BackgroundInterrater reliability, aka intercoder reliability, is defined as true agreement between raters, aka coders, without chance agreement. It is used across many disciplines including medical and health research to measure the quality of ratings, coding, diagnoses, or other observations and judgements. While numerous indices of interrater reliability are available, experts disagree on which ones are legitimate or more appropriate. Almost all agree that percent agreement (ao), the oldest and the simplest index, is also the most flawed because it fails to estimate and remove chance agreement, which is produced by raters' random rating. The experts, however, disagree on which chance estimators are legitimate or better. The experts also disagree on which of the three factors, rating category, distribution skew, or task difficulty, an index should rely on to estimate chance agreement, or which factors the known indices in fact rely on. The most popular chance-adjusted indices, according to a functionalist view of mathematical statistics, assume that all raters conduct intentional and maximum random rating while typical raters conduct involuntary and reluctant random rating. The mismatches between the assumed and the actual rater behaviors cause the indices to rely on mistaken factors to estimate chance agreement, leading to the numerous paradoxes, abnormalities, and other misbehaviors of the indices identified by prior studies.MethodsWe conducted a 4 × 8 × 3 between-subject controlled experiment with 4 subjects per cell. Each subject was a rating session with 100 pairs of rating by two raters, totaling 384 rating sessions as the experimental subjects. The experiment tested seven best-known indices of interrater reliability against the observed reliabilities and chance agreements. Impacts of the three factors, i.e., rating category, distribution skew, and task difficulty, on the indices were tested.ResultsThe most criticized index, percent agreement (ao), showed as the most accurate predictor of reliability, reporting directional r2 = .84. It was also the third best approximator, overestimating observed reliability by 13 percentage points on average. The three most acclaimed and most popular indices, Scott's π, Cohen's κ and Krippendorff's α, underperformed all other indices, reporting directional r2 = .312 and underestimated reliability by 31.4 ~ 31.8 points. The newest index, Gwet's AC1, emerged as the second-best predictor and the most accurate approximator. Bennett et al's S ranked behind AC1, and Perreault and Leigh's Ir ranked the fourth both for prediction and approximation. The reliance on category and skew and failure to rely on difficulty explain why the six chance-adjusted indices often underperformed ao, which they were created to outperform. The evidence corroborated the notion that the chance-adjusted indices assume intentional and maximum random rating while the raters instead exhibited involuntary and reluctant random rating.ConclusionThe authors call for more empirical studies and especially more controlled experiments to falsify or qualify this study. If the main findings are replicated and the underlying theories supported, new thinking and new indices may be needed. Index designers may need to refrain from assuming intentional and maximum random rating, and instead assume involuntary and reluctant random rating. Accordingly, the new indices may need to rely on task difficulty, rather than distribution skew or rating category, to estimate chance agreement.

Project description:Balance impairment is common in people with multiple sclerosis (PwMS) and frequently impacts quality of life by decreasing mobility and increasing the risk of falling. However, there are only scarce data examining the contribution of specific neurological functional systems on balance measures in MS. Therefore, the primary aim of our study was to examine the differences in posturography parameters and fall incidence according to the pyramidal, cerebellar and sensory systems functional systems in PwMS. The study included 342 PwMS, 211 women and mean disease duration of 8.2 (S.D = 8.3) years. The study sample was divided into six groups according to the pyramidal, cerebellar and sensory functional system scores, derived from the Expanded Disability Status Scale (EDSS) data. Static postural control parameters were obtained from the Zebris FDM-T Treadmill (zebris® Medical GmbH, Germany). Participants were defined as "fallers" and "non-fallers" based on their fall history. Our findings revealed a trend that PwMS affected solely in the pyramidal system, have reduced stability compared to patients with cerebellar and sensory dysfunctions. Moreover, the addition of sensory impairments to pyramidal dysfunction does not exacerbate postural control. The patients in the pure sensory group demonstrated increased stability compared to each of the three combined groups; pyramidal-cerebellar, pyramidal-sensory and pyramidal-cerebellar-sensory groups. As for fall status, the percentage of fallers in the pure pyramidal, cerebellar and sensory groups were 44.3%, 33.3% and 19.5%, respectively. As for the combined functional system groups, the percentage of fallers in the pyramidal-cerebellar, pyramidal-sensory and pyramidal-cerebellar-sensory groups were 59.7%, 40.7% and 65%, respectively. This study confirms that disorders in neurological functional systems generate different effects on postural control and incidence of falls in the MS population. From a clinical standpoint, the present information can benefit all those engaged in physical rehabilitation of PwMS.

Project description:Manual muscle testing (MMT) and hand-held dynamometry (HHD) are commonly used in people with inflammatory myopathy (IM), but their clinimetric properties have not yet been sufficiently studied. To evaluate the reliability and validity of MMT and HHD, maximum isometric strength was measured in eight muscle groups across three measurement events. To evaluate reliability of HHD, intra-class correlation coefficients (ICC), the standard error of measurements (SEM) and smallest detectable changes (SDC) were calculated. To measure reliability of MMT linear Cohen`s Kappa was computed for single muscle groups and ICC for total score. Additionally, correlations between MMT8 and HHD were evaluated with Spearman Correlation Coefficients. Fifty people with myositis (56±14 years, 76% female) were included in the study. Intra-and interrater reliability of HHD yielded excellent ICCs (0.75-0.97) for all muscle groups, except for interrater reliability of ankle extension (0.61). The corresponding SEMs% ranged from 8 to 28% and the SDCs% from 23 to 65%. MMT8 total score revealed excellent intra-and interrater reliability (ICC>0.9). Intrarater reliability of single muscle groups was substantial for shoulder and hip abduction, elbow and neck flexion, and hip extension (0.64-0.69); moderate for wrist (0.53) and knee extension (0.49) and fair for ankle extension (0.35). Interrater reliability was moderate for neck flexion (0.54) and hip abduction (0.44); fair for shoulder abduction, elbow flexion, wrist and ankle extension (0.20-0.33); and slight for knee extension (0.08). Correlations between the two tests were low for wrist, knee, ankle, and hip extension; moderate for elbow flexion, neck flexion and hip abduction; and good for shoulder abduction. In conclusion, the MMT8 total score is a reliable assessment to consider general muscle weakness in people with myositis but not for single muscle groups. In contrast, our results confirm that HHD can be recommended to evaluate strength of single muscle groups.

Dataset Information

Interrater Reliability of Four Sensory Measures in People with Multiple Sclerosis.

Background

Methods

Results

Conclusions

Publications

Interrater Reliability of Four Sensory Measures in People with Multiple Sclerosis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets