Dataset Information

Interrater reliability of physical examination tests in the acute phase of shoulder injuries.

ABSTRACT:

Background

The physical examination is one of the cornerstones of the diagnostic process in patients with acute shoulder injuries. The discriminative properties of a given examination test depend both on its validity and reliability. The aim of the present study was to assess the interrater reliability of 13 physical examination manoeuvres for acute rotator cuff tears in patients with acute soft tissue shoulder injuries.

Methods

In a large walk-in orthopaedic emergency department, 120 consecutive patients ≥40 years of age were included in a diagnostic study. Patients who had follow-up within three weeks of an acute shoulder injury without fracture on radiographs were eligible. Four emergency department physicians participated as examiners. In a subset of 48 patients, the physical examination tests were performed by two physicians, randomly chosen by their work rotation. The physicians were blinded to the findings of each other and the results of the ultrasound screening. The interrater reliability was assessed by Cohen's kappa, intraclass correlation coefficient (ICC), standard error of measurement (SEM) and Bland-Altman plots depending on whether the examination test result was registered as a binary, ordered categorical or continuous numerical variable.

Results

The median age was 55.5 years, 46% were female. Twenty-seven percent had a rotator cuff full-thickness tear on ultrasound screening; all but one involved the supraspinatus tendon. Cohen's kappa for binary tests ranged from excellent to fair. Excellent agreement (kappa > 0.8) was found for the inability to abduct above 90° and abduction strength. External rotation strength expressed substantial agreement (kappa 0.7). The lowest scores were registered for Hawkins` test and the external rotation lag sign (kappa 0.25 and 0.40, respectively). The ICCs for active range of abduction and external rotation were 0.93 (0.88-0.96) and 0.84 (0.72-0.91), whereas the SEM was 15 and 9, respectively.

Conclusions

The results indicate that examination manoeuvres assessing abduction and external rotation range of motion and strength are more reliable than manoeuvres assessing pain in patients in the acute phase of traumatic shoulder injury. The poor agreement observed is likely to limit the validity in the present setting of two commonly used tests.

Trial registration

The Norwegian Regional Ethics Committee South East ( 2015/195 ).

SUBMITTER: Schmidt M

PROVIDER: S-EPMC8427844 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionClinicians rely on certain physical examination tests to diagnose and potentially grade ankle sprains and ankle instability. Diagnostic error and inaccurate prognosis may have important repercussions for clinical decision-making and patient outcomes. Therefore, it is important to recognize the diagnostic value of orthopaedic tests through understanding the reliability and validity of these tests.ObjectiveTo systematically review and report evidence on the reliability and validity of orthopaedic tests for the diagnosis of ankle sprains and instability.MethodsPubMed, CINAHL, Scopus, and Cochrane databases were searched from inception to December 2021. In addition, the reference list of included studies, located systematic reviews, and orthopaedic textbooks were searched. All articles reporting reliability or validity of physical examination or orthopaedic tests to diagnose ankle instability or sprains were included. Methodological quality of the reliability and the validity studies was assessed with The Quality Appraisal for Reliability studies checklist and the Quality Assessment of Diagnostic Accuracy Studies-2 respectively. We identified the number of times the orthopaedic test was investigated and the validity and/or reliability of each test.ResultsOverall, sixteen studies were included. Three studies assessed reliability, eight assessed validity, and five evaluated both. Overall, fifteen tests were evaluated, none demonstrated robust reliability and validity scores. The anterolateral talar palpation test reported the highest diagnostic accuracy. Further, the anterior drawer test, the anterolateral talar palpation, the reverse anterior lateral drawer test, and palpation of the anterior talofibular ligament reported the highest sensitivity. The highest specificity was attributed to the anterior drawer test, the anterolateral drawer test, the reverse anterior lateral drawer test, tenderness on palpation of the proximal fibular, and the squeeze test.ConclusionOverall, the diagnostic accuracy, reliability, and validity of physical examination tests for the assessment of ankle instability were limited. Physical examination tests should not be used in isolation, but rather in combination with the clinical history to diagnose an ankle sprain. Preliminary evidence suggests that the overall validity of physical examination for the ankle may be better if conducted five days after the injury rather than within 48 h of injury.

Project description:BackgroundInterrater reliability, aka intercoder reliability, is defined as true agreement between raters, aka coders, without chance agreement. It is used across many disciplines including medical and health research to measure the quality of ratings, coding, diagnoses, or other observations and judgements. While numerous indices of interrater reliability are available, experts disagree on which ones are legitimate or more appropriate. Almost all agree that percent agreement (ao), the oldest and the simplest index, is also the most flawed because it fails to estimate and remove chance agreement, which is produced by raters' random rating. The experts, however, disagree on which chance estimators are legitimate or better. The experts also disagree on which of the three factors, rating category, distribution skew, or task difficulty, an index should rely on to estimate chance agreement, or which factors the known indices in fact rely on. The most popular chance-adjusted indices, according to a functionalist view of mathematical statistics, assume that all raters conduct intentional and maximum random rating while typical raters conduct involuntary and reluctant random rating. The mismatches between the assumed and the actual rater behaviors cause the indices to rely on mistaken factors to estimate chance agreement, leading to the numerous paradoxes, abnormalities, and other misbehaviors of the indices identified by prior studies.MethodsWe conducted a 4 × 8 × 3 between-subject controlled experiment with 4 subjects per cell. Each subject was a rating session with 100 pairs of rating by two raters, totaling 384 rating sessions as the experimental subjects. The experiment tested seven best-known indices of interrater reliability against the observed reliabilities and chance agreements. Impacts of the three factors, i.e., rating category, distribution skew, and task difficulty, on the indices were tested.ResultsThe most criticized index, percent agreement (ao), showed as the most accurate predictor of reliability, reporting directional r2 = .84. It was also the third best approximator, overestimating observed reliability by 13 percentage points on average. The three most acclaimed and most popular indices, Scott's π, Cohen's κ and Krippendorff's α, underperformed all other indices, reporting directional r2 = .312 and underestimated reliability by 31.4 ~ 31.8 points. The newest index, Gwet's AC1, emerged as the second-best predictor and the most accurate approximator. Bennett et al's S ranked behind AC1, and Perreault and Leigh's Ir ranked the fourth both for prediction and approximation. The reliance on category and skew and failure to rely on difficulty explain why the six chance-adjusted indices often underperformed ao, which they were created to outperform. The evidence corroborated the notion that the chance-adjusted indices assume intentional and maximum random rating while the raters instead exhibited involuntary and reluctant random rating.ConclusionThe authors call for more empirical studies and especially more controlled experiments to falsify or qualify this study. If the main findings are replicated and the underlying theories supported, new thinking and new indices may be needed. Index designers may need to refrain from assuming intentional and maximum random rating, and instead assume involuntary and reluctant random rating. Accordingly, the new indices may need to rely on task difficulty, rather than distribution skew or rating category, to estimate chance agreement.

Project description:BackgroundAlthough posterior shoulder tightness (PST) has been associated with shoulder pathology and altered glenohumeral joint kinematics, uncertainty remains regarding its cause and definition. To understand the efficacy of treatments for PST, it must be possible to identify people with PST for the purposes of research and clinical decision-making. Clinical tests for PST must demonstrate acceptable levels of measurement reliability in order to identify the condition and to evaluate the response to intervention. There is currently a lack of research describing intersession reliability for measures of PST. The aim of this study was to quantify the inter-session reliability for three clinical tests used to identify PST over a 6-10 week interval.MethodsA convenience sample of 26 asymptomatic adult participants (52 shoulders) were recruited from a university setting over a five-month duration. Participants attended the human movement laboratory for measurement of glenohumeral joint internal rotation, horizontal adduction and low flexion on two occasions separated by an interval of 6-10 weeks.Intra-class correlation coefficients were calculated from the mean square values derived from the within-subject, single factor (repeated measures) ANOVA. Test-retest measurement stability was evaluated by calculating the standard error of measurement and the minimum detectable change for each measurement.ResultsAll 3 tests demonstrated good intersession intra-rater reliability (0.86-0.88), and the standard error of measurement (95%) were 7.3° for glenohumeral horizontal adduction, 9.4° for internal rotation, and 6.9° for low flexion. The minimum detectable change for glenohumeral horizontal adduction was 10.2°, internal rotation was 13.3°, and low flexion was 9.7°.ConclusionIn this population of people without symptoms, the 3 measures of PST all demonstrated acceptable inter-session reliability. The standard error of measurement and minimum detectable change results can be used to determine if a change in measures of PST are due to measurement error or an actual change over time.