Dataset Information

Rater agreement for assessment of equine back mobility at walk and trot compared to quantitative gait analysis.

ABSTRACT:

Background

Lameness assessment in horses is still predominantly performed using subjective methods. Visual assessment is known to have moderate to good intra-rater agreement but relatively poor inter-rater agreement. Little is known about inter- and intra-rater agreement on the evaluation of back motion, for which no objective measurement technique in a clinical setting is available thus far.

Objectives

To describe inter- and intra-rater agreement of visual evaluation of equine back mobility.

Study design

Rater reliability study using a fully crossed design in which all horses are rated by all observers. This data is compared with objective gait analysis.

Methods

Seventy equine professionals (veterinarians and physiotherapists) and veterinary students evaluated videos of 12 healthy horses at walk and trot on a hard, straight line. Nine parameters related to back mobility were scored: general mobility, thoracic, lumbar, lumbosacral flexion and extension and left and right thoracolumbar latero-flexion. All parameters were compared with simultaneously measured quantitative motion parameters. After 1 month, six randomly chosen horses were re-evaluated by 57 observers.

Results

For each parameter inter- and intra-rater agreements were calculated using intra-class correlation coefficients. For all parameters, inter-rater agreement was very poor (<0.2). The mean intra-rater agreement of all observers and for all parameters was poor (~0.4) but varied between 0.0 and 0.96 for individual observers. There was no correlation between the visual subjective scoring and objective gait analysis measurements.

Main limitations

Horses were scored from videos and by lack of any existing (semi-) quantitative system, a custom-made system had to be used.

Conclusions

The poor inter- and intra-rater agreements of visual scoring of mobility of the equine back and the disagreement between subjective and objective gait analysis data, demonstrate the need for the development and introduction of objective, quantitative and repeatable techniques to assess equine back motion.

SUBMITTER: Spoormakers TJP

PROVIDER: S-EPMC8177646 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:There is a shortage of agreement studies relevant for measuring changes over time in lumbar intervertebral disc structures. The objectives of this study were: 1) to develop a method for measurement of intervertebral disc height, anterior and posterior disc material and dural sac diameter using MRI, 2) to evaluate intra- and inter-rater agreement and reliability for the measurements included, and 3) to identify factors compromising agreement.Measurements were performed on MRIs from 16 people with and 16 without lumbar disc herniation, purposefully chosen to represent all possible disc contours among participants in a general population study cohort. Using the new method, MRIs were measured twice by one rater and once by a second rater. Agreement on the sagittal start- and end-slice was evaluated using weighted Kappa. Length and volume measurements were conducted on available slices between intervertebral foramens, and cross-sectional areas (CSA) were calculated from length measurements and slice thickness. Results were reported as Bland and Altman's limits of agreement (LOA) and intraclass correlation coefficients (ICC).Weighted Kappa (Kw (95% CI)) for start- and end-slice were: intra-: 0.82(0.60;0.97) & 0.71(0.43;0.93); inter-rater: 0.56(0.29;0.78) & 0.60(0.35;0.81). For length measurements, LOA ranged from [-1.0;1.0] mm to [-2.0;2.3] mm for intra-; and from [-1.1; 1.4] mm to [-2.6;2.0] mm for inter-rater. For volume measurements, LOA ranged from [-293;199] mm3 to [-582;382] mm3 for intra-, and from [-17;801] mm3 to [-450;713] mm3 for inter-rater. For CSAs, LOA ranged between [-21.3; 18.8] mm2 and [-31.2; 43.7] mm2 for intra-, and between [-10.8; 16.4] mm2 and [-64.6; 27.1] mm2 for inter-rater. In general, LOA as a proportion of mean values gradually decreased with increasing size of the measured structures. Agreement was compromised by difficulties in identifying the vertebral corners, the anterior and posterior boundaries of the intervertebral disc and the dural sac posterior boundary. With two exceptions, ICCs were above 0.81.Length measurements and calculated CSAs of disc morphology and dural sac diameter from MRIs showed acceptable intra- and inter-rater agreement and reliability. However, caution should be taken when measuring very small structures and defining anatomical landmarks.

Project description:Reasons for performing studyLungeing is an important part of lameness examinations as the circular path may accentuate low-grade lameness. Movement asymmetries related to the circular path, to compensatory movements and to pain make the lameness evaluation complex. Scientific studies have shown high inter-rater variation when assessing lameness during straight line movement.ObjectivesThe aim was to estimate inter- and intra-rater agreement of equine veterinarians evaluating lameness from videos of sound and lame horses during lungeing and to investigate the influence of veterinarians' experience and the objective degree of movement asymmetry on rater agreement.Study designCross-sectional observational study.MethodsVideo recordings and quantitative gait analysis with inertial sensors were performed in 23 riding horses of various breeds. The horses were examined at trot on a straight line and during lungeing on soft or hard surfaces in both directions. One video sequence was recorded per condition and the horses were classified as forelimb lame, hindlimb lame or sound from objective straight line symmetry measurements. Equine veterinarians (n = 86), including 43 with >5 years of orthopaedic experience, participated in a web-based survey and were asked to identify the lamest limb on 60 videos, including 10 repeats. The agreements between (inter-rater) and within (intra-rater) veterinarians were analysed with κ statistics (Fleiss, Cohen).ResultsInter-rater agreement κ was 0.31 (0.38/0.25 for experienced/less experienced) and higher for forelimb (0.33) than for hindlimb lameness (0.11) or soundness (0.08) evaluation. Median intra-rater agreement κ was 0.57.ConclusionsInter-rater agreement was poor for less experienced raters, and for all raters when evaluating hindlimb lameness. Since identification of the lame limb/limbs is a prerequisite for successful diagnosis, treatment and recovery, the high inter-rater variation when evaluating lameness on the lunge is likely to influence the accuracy and repeatability of lameness examinations and, indirectly, the efficacy of treatment.