Dataset Information

Rater agreement on gait assessment during neurologic examination of horses.

ABSTRACT: Reproducible and accurate recognition of presence and severity of ataxia in horses with neurologic disease is important when establishing a diagnosis, assessing response to treatment, and making recommendations that might influence rider safety or a decision for euthanasia.To determine the reproducibility and validity of the gait assessment component in the neurologic examination of horses.Twenty-five horses referred to the Royal Veterinary College Equine Referral Hospital for neurological assessment (n = 15), purchased (without a history of gait abnormalities) for an unrelated study (n = 5), or donated because of perceived ataxia (n = 5).Utilizing a prospective study design; a group of board-certified medicine (n = 2) and surgery (n = 2) clinicians and residents (n = 2) assessed components of the equine neurologic examination (live and video recorded) and assigned individual and overall neurologic gait deficit grades (0-4). Inter-rater agreement and assessment-reassessment reliability were quantified using intraclass correlation coefficients (ICC).The ICCs of the selected components of the neurologic examination ranged from 0 to 0.69. "Backing up" and "recognition of mistakes over obstacle" were the only components with an ICC > 0.6. Assessment-reassessment agreement was poor to fair. The agreement on gait grading was good overall (ICC = 0.74), but poor for grades ? 1 (ICC = 0.08) and fair for ataxia grades ? 2 (ICC = 0.43). Clinicians with prior knowledge of a possible gait abnormality were more likely to assign a grade higher than the median grade.Clinicians should be aware of poor agreement even between skilled observers of equine gait abnormalities, especially when the clinical signs are subtle.

SUBMITTER: Olsen E

PROVIDER: S-EPMC4857973 | biostudies-literature | 2014 Mar-Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Rater agreement on gait assessment during neurologic examination of horses.

Olsen E E Dunkel B B Barker W H J WH Finding E J T EJ Perkins J D JD Witte T H TH Yates L J LJ Andersen P H PH Baiker K K Piercy R J RJ

Journal of veterinary internal medicine 20140224 2

<h4>Background</h4>Reproducible and accurate recognition of presence and severity of ataxia in horses with neurologic disease is important when establishing a diagnosis, assessing response to treatment, and making recommendations that might influence rider safety or a decision for euthanasia.<h4>Objectives</h4>To determine the reproducibility and validity of the gait assessment component in the neurologic examination of horses.<h4>Animals</h4>Twenty-five horses referred to the Royal Veterinary C ...[more]

PMID: 24612411

Similar Datasets

Project description:Reasons for performing studyLungeing is an important part of lameness examinations as the circular path may accentuate low-grade lameness. Movement asymmetries related to the circular path, to compensatory movements and to pain make the lameness evaluation complex. Scientific studies have shown high inter-rater variation when assessing lameness during straight line movement.ObjectivesThe aim was to estimate inter- and intra-rater agreement of equine veterinarians evaluating lameness from videos of sound and lame horses during lungeing and to investigate the influence of veterinarians' experience and the objective degree of movement asymmetry on rater agreement.Study designCross-sectional observational study.MethodsVideo recordings and quantitative gait analysis with inertial sensors were performed in 23 riding horses of various breeds. The horses were examined at trot on a straight line and during lungeing on soft or hard surfaces in both directions. One video sequence was recorded per condition and the horses were classified as forelimb lame, hindlimb lame or sound from objective straight line symmetry measurements. Equine veterinarians (n = 86), including 43 with >5 years of orthopaedic experience, participated in a web-based survey and were asked to identify the lamest limb on 60 videos, including 10 repeats. The agreements between (inter-rater) and within (intra-rater) veterinarians were analysed with κ statistics (Fleiss, Cohen).ResultsInter-rater agreement κ was 0.31 (0.38/0.25 for experienced/less experienced) and higher for forelimb (0.33) than for hindlimb lameness (0.11) or soundness (0.08) evaluation. Median intra-rater agreement κ was 0.57.ConclusionsInter-rater agreement was poor for less experienced raters, and for all raters when evaluating hindlimb lameness. Since identification of the lame limb/limbs is a prerequisite for successful diagnosis, treatment and recovery, the high inter-rater variation when evaluating lameness on the lunge is likely to influence the accuracy and repeatability of lameness examinations and, indirectly, the efficacy of treatment.

Project description:BackgroundLameness assessment in horses is still predominantly performed using subjective methods. Visual assessment is known to have moderate to good intra-rater agreement but relatively poor inter-rater agreement. Little is known about inter- and intra-rater agreement on the evaluation of back motion, for which no objective measurement technique in a clinical setting is available thus far.ObjectivesTo describe inter- and intra-rater agreement of visual evaluation of equine back mobility.Study designRater reliability study using a fully crossed design in which all horses are rated by all observers. This data is compared with objective gait analysis.MethodsSeventy equine professionals (veterinarians and physiotherapists) and veterinary students evaluated videos of 12 healthy horses at walk and trot on a hard, straight line. Nine parameters related to back mobility were scored: general mobility, thoracic, lumbar, lumbosacral flexion and extension and left and right thoracolumbar latero-flexion. All parameters were compared with simultaneously measured quantitative motion parameters. After 1 month, six randomly chosen horses were re-evaluated by 57 observers.ResultsFor each parameter inter- and intra-rater agreements were calculated using intra-class correlation coefficients. For all parameters, inter-rater agreement was very poor (<0.2). The mean intra-rater agreement of all observers and for all parameters was poor (~0.4) but varied between 0.0 and 0.96 for individual observers. There was no correlation between the visual subjective scoring and objective gait analysis measurements.Main limitationsHorses were scored from videos and by lack of any existing (semi-) quantitative system, a custom-made system had to be used.ConclusionsThe poor inter- and intra-rater agreements of visual scoring of mobility of the equine back and the disagreement between subjective and objective gait analysis data, demonstrate the need for the development and introduction of objective, quantitative and repeatable techniques to assess equine back motion.

Project description:OBJECTIVE:Surgical site infection (SSI) is the most common nosocomial infection, in vascular surgery patients, who experience a high rate of readmission. Facilitating transition from hospital to outpatient care with digital image-based wound monitoring has the potential to detect and to enable treatment of SSI at an early stage. In this study, we evaluated whether smartphone digital images can supplant in-person evaluation of postoperative vascular surgery wounds. METHODS:We developed a wound assessment checklist using previously validated criteria. We recruited adults who underwent a vascular surgical procedure between 2014 and 2015, involving an incision of at least 3 cm, from a high-volume academic vascular surgery service. Vascular surgery care providers evaluated wounds in person using the assessment checklist; a different group of providers evaluated wounds by a smartphone digital image. Inter-rater agreement coefficients for wound characteristics and treatment plan were calculated within and between the in-person group and the digital image group; the sensitivity and specificity of digital images relative to in-person evaluation were determined. RESULTS:We assessed a total of 80 wounds. Regardless of modality, inter-rater agreement was poor when wounds were evaluated for the presence of ecchymosis and redness; moderate for cellulitis; and high for the presence of a drain, necrosis, or dehiscence. As expected, the presence of drainage was more readily observed in person. Inter-rater agreement was high for both in-person and image-based assessment with respect to course of treatment, with near-perfect agreement for treatments ranging from antibiotics to surgical débridement to hospital readmission. No difference in agreement emerged when raters evaluated poor-quality compared with high-quality images. For most parameters, specificity was higher than sensitivity for image-based compared with "gold standard" in-person assessment. CONCLUSIONS:Using smartphone digital images is a valid method for evaluating postoperative vascular surgery wounds and is comparable to in-person evaluation with regard to most wound characteristics. The inter-rater reliability for determining treatment recommendations was universally high. Remote wound monitoring and assessment may play an integral role in future transitional care models to decrease readmission for SSI in vascular or other surgical patients. These findings will inform smartphone implementation in the clinical care setting as wound images transition from informal clinical communication to becoming part of the care standard.

Project description:Low concordance between drug-drug interaction (DDI) knowledge bases is a well-documented concern. One potential cause of inconsistency is variability between drug experts in approach to assessing evidence about potential DDIs. In this study, we examined the face validity and inter-rater reliability of a novel DDI evidence evaluation instrument designed to be simple and easy to use.MethodsA convenience sample of participants with professional experience evaluating DDI evidence was recruited. Participants independently evaluated pre-selected evidence items for 5 drug pairs using the new instrument. For each drug pair, participants labeled each evidence item as sufficient or insufficient to establish the existence of a DDI based on the evidence categories provided by the instrument. Participants also decided if the overall body of evidence supported a DDI involving the drug pair. Agreement was computed both at the evidence item and drug pair levels. A cut-off of ≥ 70% was chosen as the agreement threshold for percent agreement, while a coefficient > 0.6 was used as the cut-off for chance-corrected agreement. Open ended comments were collected and coded to identify themes related to the participants' experience using the novel approach.ResultsThe face validity of the new instrument was established by two rounds of evaluation involving a total of 6 experts. Fifteen experts agreed to participate in the reliability assessment, and 14 completed the study. Participant agreement on the sufficiency of 22 of the 34 evidence items (65%) did not exceed the a priori agreement threshold. Similarly, agreement on the sufficiency of evidence for 3 of the 5 drug pairs (60%) was poor. Chance-corrected agreement at the drug pair level further confirmed the poor interrater reliability of the instrument (Gwet's AC1 = 0.24, Conger's Kappa = 0.24). Participant comments suggested several possible reasons for the disagreements including unaddressed subjectivity in assessing an evidence item's type and study design, an infeasible separation of evidence evaluation from the consideration of clinical relevance, and potential issues related to the evaluation of DDI case reports.ConclusionsEven though the key findings were negative, the study's results shed light on how experts approach DDI evidence assessment, including the importance situating evidence assessment within the context of consideration of clinical relevance. Analysis of participant comments within the context of the negative findings identified several promising future research directions including: novel computer-based support for evidence assessment; formal evaluation of a more comprehensive evidence assessment approach that requires consideration of specific, explicitly stated, clinical consequences; and more formal investigation of DDI case report assessment instruments.

Project description:Objectives To explore agreement among healthcare professionals assessing eligibility for work disability benefits.Design Systematic review and narrative synthesis of reproducibility studies.Data sources Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies.Eligibility criteria Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations. : Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed.Results From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to ?-0.10). Inter-rater reliability was poor in six studies (37%) and excellent in only two (13%). This contrasts with studies conducted in the research setting, where the median inter-rater reliability was 0.76 (range 0.91-0.53), and 71% (5/7) studies achieved excellent inter-rater reliability. Reliability between assessing professionals was higher when the evaluation was guided by a standardised instrument (23 studies, P=0.006). No such association was detected for subjective or chronic health conditions or the studies' generalisability to real world evaluation of disability (P=0.46, 0.45, and 0.65, respectively).Conclusions Despite their common use and far reaching consequences for workers claiming disabling injury or illness, research on the reliability of medical evaluations of disability for work is limited and indicates high variation in judgments among assessing professionals. Standardising the evaluation process could improve reliability. Development and testing of instruments and structured approaches to improve reliability in evaluation of disability are urgently needed.

Dataset Information

Rater agreement on gait assessment during neurologic examination of horses.

Publications

Rater agreement on gait assessment during neurologic examination of horses.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure