Dataset Information

Rater agreement of visual lameness assessment in horses during lungeing.

ABSTRACT:

Reasons for performing study

Lungeing is an important part of lameness examinations as the circular path may accentuate low-grade lameness. Movement asymmetries related to the circular path, to compensatory movements and to pain make the lameness evaluation complex. Scientific studies have shown high inter-rater variation when assessing lameness during straight line movement.

Objectives

The aim was to estimate inter- and intra-rater agreement of equine veterinarians evaluating lameness from videos of sound and lame horses during lungeing and to investigate the influence of veterinarians' experience and the objective degree of movement asymmetry on rater agreement.

Study design

Cross-sectional observational study.

Methods

Video recordings and quantitative gait analysis with inertial sensors were performed in 23 riding horses of various breeds. The horses were examined at trot on a straight line and during lungeing on soft or hard surfaces in both directions. One video sequence was recorded per condition and the horses were classified as forelimb lame, hindlimb lame or sound from objective straight line symmetry measurements. Equine veterinarians (n = 86), including 43 with >5 years of orthopaedic experience, participated in a web-based survey and were asked to identify the lamest limb on 60 videos, including 10 repeats. The agreements between (inter-rater) and within (intra-rater) veterinarians were analysed with κ statistics (Fleiss, Cohen).

Results

Inter-rater agreement κ was 0.31 (0.38/0.25 for experienced/less experienced) and higher for forelimb (0.33) than for hindlimb lameness (0.11) or soundness (0.08) evaluation. Median intra-rater agreement κ was 0.57.

Conclusions

Inter-rater agreement was poor for less experienced raters, and for all raters when evaluating hindlimb lameness. Since identification of the lame limb/limbs is a prerequisite for successful diagnosis, treatment and recovery, the high inter-rater variation when evaluating lameness on the lunge is likely to influence the accuracy and repeatability of lameness examinations and, indirectly, the efficacy of treatment.

SUBMITTER: Hammarberg M

PROVIDER: S-EPMC4964936 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:OBJECTIVE:Surgical site infection (SSI) is the most common nosocomial infection, in vascular surgery patients, who experience a high rate of readmission. Facilitating transition from hospital to outpatient care with digital image-based wound monitoring has the potential to detect and to enable treatment of SSI at an early stage. In this study, we evaluated whether smartphone digital images can supplant in-person evaluation of postoperative vascular surgery wounds. METHODS:We developed a wound assessment checklist using previously validated criteria. We recruited adults who underwent a vascular surgical procedure between 2014 and 2015, involving an incision of at least 3 cm, from a high-volume academic vascular surgery service. Vascular surgery care providers evaluated wounds in person using the assessment checklist; a different group of providers evaluated wounds by a smartphone digital image. Inter-rater agreement coefficients for wound characteristics and treatment plan were calculated within and between the in-person group and the digital image group; the sensitivity and specificity of digital images relative to in-person evaluation were determined. RESULTS:We assessed a total of 80 wounds. Regardless of modality, inter-rater agreement was poor when wounds were evaluated for the presence of ecchymosis and redness; moderate for cellulitis; and high for the presence of a drain, necrosis, or dehiscence. As expected, the presence of drainage was more readily observed in person. Inter-rater agreement was high for both in-person and image-based assessment with respect to course of treatment, with near-perfect agreement for treatments ranging from antibiotics to surgical débridement to hospital readmission. No difference in agreement emerged when raters evaluated poor-quality compared with high-quality images. For most parameters, specificity was higher than sensitivity for image-based compared with "gold standard" in-person assessment. CONCLUSIONS:Using smartphone digital images is a valid method for evaluating postoperative vascular surgery wounds and is comparable to in-person evaluation with regard to most wound characteristics. The inter-rater reliability for determining treatment recommendations was universally high. Remote wound monitoring and assessment may play an integral role in future transitional care models to decrease readmission for SSI in vascular or other surgical patients. These findings will inform smartphone implementation in the clinical care setting as wound images transition from informal clinical communication to becoming part of the care standard.

Project description:BackgroundThe increasing popularity of objective gait analysis makes application in prepurchase examinations (PPE) a logical next step. Therefore, there is a need to have more understanding of asymmetry during a PPE in horses described on clinical evaluation as subtly lame.ObjectivesThe objective of this study is to objectively compare asymmetry in horses raising minor vet concerns in a PPE and in horses raising major vet concerns with that found in horses presented with subtle single-limb lameness, and to investigate the effect of age/discipline on the clinicians' interpretation of asymmetry on the classification of minor vet concerns in a PPE.Study designClinical case-series.MethodsHorses presented for PPE (n = 98) or subjectively evaluated as single limb low-grade (1-2/5) lame (n = 24, 13 forelimb lame, 11 hindlimb lame), from the patient population of a single clinic, were enrolled in the study provided that owners were willing to participate. Horses undergoing PPE were assigned a classification of having minor vet concerns (n = 84) or major vet concerns (n = 14) based on findings during the dynamic-orthopaedic part of the PPE. Lame horses were only included if pain-related lameness was confirmed by an objective improvement after diagnostic analgesia exceeding daily variation determined for equine symmetry parameters using optical motion capture. Clinical evaluation was performed by six different clinicians, each with ≥8 years of equine orthopaedic experience. Vertical movement symmetry was measured using optical motion capture, simultaneously with the orthopaedic examination. Data were analysed using previously described parameters and mixed model analysis and least squares means were used to calculate differences between groups.ResultsThere was no effect of age or discipline on the levels of asymmetry within PPE horses raising minor vet concerns. MinDiff and RUD of the head discriminated between forelimb lame and PPE horses raising minor vet concerns; MinDiff, MaxDiff, RUD of the Pelvis, HHDswing and HHDstance did so for hindlimb lameness. Two lameness patterns differentiated both forelimb and hindlimb lame from PPE horses with minor vet concerns: RUD Poll + MinDiff Withers - RUD Pelvis and RUD Pelvis + RUD Poll - MinDiff Withers. Correcting for vertical range of motion enabled differentiation of PPE horses with minor vet concerns from PPE horses with major vet concerns.Main limitationsObjective data only based on trot on soft surface, limited number of PPE horses with major vet concerns.ConclusionsCombinations of kinematic parameters discriminate between PPE horses with minor vet concerns and subtly lame horses, though overlap exists.

Project description:BackgroundLameness assessment in horses is still predominantly performed using subjective methods. Visual assessment is known to have moderate to good intra-rater agreement but relatively poor inter-rater agreement. Little is known about inter- and intra-rater agreement on the evaluation of back motion, for which no objective measurement technique in a clinical setting is available thus far.ObjectivesTo describe inter- and intra-rater agreement of visual evaluation of equine back mobility.Study designRater reliability study using a fully crossed design in which all horses are rated by all observers. This data is compared with objective gait analysis.MethodsSeventy equine professionals (veterinarians and physiotherapists) and veterinary students evaluated videos of 12 healthy horses at walk and trot on a hard, straight line. Nine parameters related to back mobility were scored: general mobility, thoracic, lumbar, lumbosacral flexion and extension and left and right thoracolumbar latero-flexion. All parameters were compared with simultaneously measured quantitative motion parameters. After 1 month, six randomly chosen horses were re-evaluated by 57 observers.ResultsFor each parameter inter- and intra-rater agreements were calculated using intra-class correlation coefficients. For all parameters, inter-rater agreement was very poor (<0.2). The mean intra-rater agreement of all observers and for all parameters was poor (~0.4) but varied between 0.0 and 0.96 for individual observers. There was no correlation between the visual subjective scoring and objective gait analysis measurements.Main limitationsHorses were scored from videos and by lack of any existing (semi-) quantitative system, a custom-made system had to be used.ConclusionsThe poor inter- and intra-rater agreements of visual scoring of mobility of the equine back and the disagreement between subjective and objective gait analysis data, demonstrate the need for the development and introduction of objective, quantitative and repeatable techniques to assess equine back motion.