Dataset Information

Development and inter-rater reliability of the Liverpool adverse drug reaction causality assessment tool.

ABSTRACT:

Aim

To develop and test a new adverse drug reaction (ADR) causality assessment tool (CAT).

Methods

A comparison between seven assessors of a new CAT, formulated by an expert focus group, compared with the Naranjo CAT in 80 cases from a prospective observational study and 37 published ADR case reports (819 causality assessments in total).

Main outcome measures

Utilisation of causality categories, measure of disagreements, inter-rater reliability (IRR).

Results

The Liverpool ADR CAT, using 40 cases from an observational study, showed causality categories of 1 unlikely, 62 possible, 92 probable and 125 definite (1, 62, 92, 125) and 'moderate' IRR (kappa 0.48), compared to Naranjo (0, 100, 172, 8) with 'moderate' IRR (kappa 0.45). In a further 40 cases, the Liverpool tool (0, 66, 81, 133) showed 'good' IRR (kappa 0.6) while Naranjo (1, 90, 185, 4) remained 'moderate'.

Conclusion

The Liverpool tool assigns the full range of causality categories and shows good IRR. Further assessment by different investigators in different settings is needed to fully assess the utility of this tool.

SUBMITTER: Gallagher RM

PROVIDER: S-EPMC3237416 | biostudies-literature | 2011

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and inter-rater reliability of the Liverpool adverse drug reaction causality assessment tool.

Gallagher Ruairi M RM Kirkham Jamie J JJ Mason Jennifer R JR Bird Kim A KA Williamson Paula R PR Nunn Anthony J AJ Turner Mark A MA Smyth Rosalind L RL Pirmohamed Munir M

PloS one 20111214 12

<h4>Aim</h4>To develop and test a new adverse drug reaction (ADR) causality assessment tool (CAT).<h4>Methods</h4>A comparison between seven assessors of a new CAT, formulated by an expert focus group, compared with the Naranjo CAT in 80 cases from a prospective observational study and 37 published ADR case reports (819 causality assessments in total).<h4>Main outcome measures</h4>Utilisation of causality categories, measure of disagreements, inter-rater reliability (IRR).<h4>Results</h4>The Liv ...[more]

PMID: 22194808

Similar Datasets

Project description:AimTo develop and test a new tool to assess the avoidability of adverse drug reactions that is suitable for use in paediatrics but which is also applicable to a variety of other settings.MethodsThe study involved multiple phases. Preliminary work involved using the Hallas scale and a modification of the existing Hallas scale, to assess two different sets of adverse drug reaction (ADR) case reports. Phase 1 defined, modified and refined a new tool using multidisciplinary teams. Phase 2 involved the assessment of 50 ADR case reports from a prospective study of paediatric inpatients by individual assessors. Phase 3 compared assessments with the new tool for individuals and groups in comparison to the 'gold standard' (the avoidability outcome set by a panel of senior investigators: an experienced clinical pharmacologist, paediatrician and pharmacist).Main outcome measuresInter-rater reliability (IRR), measure of disagreement and utilization of avoidability categories.ResultsPreliminary work-Pilot phase: results for the original Hallas cases were fair and pairwise kappa scores ranged from 0.21 to 0.36. Results for the modified Hallas cases were poor, pairwise kappa scores ranged from 0.06 to 0.16. Phase 1: on initial use of the new tool, agreement between the two multidisciplinary groups was found on 13/20 cases with a kappa score of 0.29 (95% CI -0.04 to 0.62). Phase 2: the assessment of 50 ADR case reports by six individual reviewers yielded pairwise kappa scores ranging from poor to good 0.12 to 0.75 and percentage exact agreement (%EA) ranged from 52-90%. Phase 3: Percentage exact agreement ranged from 35-70%. Overall, individuals had better agreement with the 'gold standard'.ConclusionAvoidability assessment is feasible but needs careful attention to methods. The Liverpool ADR avoidability assessment tool showed mixed IRR. We have developed and validated a method for assessing the avoidability of ADRs that is transparent, more objective than previous methods and that can be used by individuals or groups.

Project description:Studies published in general and specialty medical journals have the potential to improve emergency medicine (EM) practice, but there can be delayed awareness of this evidence because emergency physicians (EPs) are unlikely to read most of these journals. Also, not all published studies are intended for or ready for clinical practice application. The authors developed "Best Evidence in Emergency Medicine" (BEEM) to ameliorate these problems by searching for, identifying, appraising, and translating potentially practice-changing studies for EPs. An initial step in the BEEM process is the BEEM rater scale, a novel tool for EPs to collectively evaluate the relative clinical relevance of EM-related studies found in more than 120 journals. The BEEM rater process was designed to serve as a clinical relevance filter to identify those studies with the greatest potential to affect EM practice. Therefore, only those studies identified by BEEM raters as having the highest clinical relevance are selected for the subsequent critical appraisal process and, if found methodologically sound, are promoted as the best evidence in EM.The primary objective was to measure inter-rater reliability (IRR) of the BEEM rater scale. Secondary objectives were to determine the minimum number of EP raters needed for the BEEM rater scale to achieve acceptable reliability and to compare performance of the scale against a previously published evidence rating system, the McMaster Online Rating of Evidence (MORE), in an EP population.The authors electronically distributed the title, conclusion, and a PubMed link for 23 recently published studies related to EM to a volunteer group of 134 EPs. The volunteers answered two demographic questions and rated the articles using one of two randomly assigned seven-point Likert scales, the BEEM rater scale (n = 68) or the MORE scale (n = 66), over two separate administrations. The IRR of each scale was measured using generalizability theory.The IRR of the BEEM rater scale ranged between 0.90 (95% confidence interval [CI] = 0.86 to 0.93) to 0.92 (95% CI = 0.89 to 0.94) across administrations. Decision studies showed a minimum of 12 raters is required for acceptable reliability of the BEEM rater scale. The IRR of the MORE scale was 0.82 to 0.84.The BEEM rater scale is a highly reliable, single-question tool for a small number of EPs to collectively rate the relative clinical relevance within the specialty of EM of recently published studies from a variety of medical journals. It compares favorably with the MORE system because it achieves a high IRR despite simply requiring raters to read each article's title and conclusion.

Project description:AimDespite the impact of spasticity, there is a lack of objective, clinically reliable and valid tools for its assessment. This study aims to evaluate the reliability of various performance- and spasticity-related parameters collected with a manually controlled instrumented spasticity assessment in four lower limb muscles in children with cerebral palsy (CP).MethodThe lateral gastrocnemius, medial hamstrings, rectus femoris and hip adductors of 12 children with spastic CP (12.8 years, ±4.13 years, bilateral/unilateral involvement n=7/5) were passively stretched in the sagittal plane at incremental velocities. Muscle activity, joint motion, and torque were synchronously recorded using electromyography, inertial sensors, and a force/torque load-cell. Reliability was assessed on three levels: (1) intra- and (2) inter-rater within session, and (3) intra-rater between session.ResultsParameters were found to be reliable in all three analyses, with 90% containing intra-class correlation coefficients >0.6, and 70% of standard error of measurement values <20% of the mean values. The most reliable analysis was intra-rater within session, followed by intra-rater between session, and then inter-rater within session. The Adds evaluation had a slightly lower level of reliability than that of the other muscles.ConclusionsLimited intrinsic/extrinsic errors were introduced by repeated stretch repetitions. The parameters were more reliable when the same rater, rather than different raters performed the evaluation. Standardisation and training should be further improved to reduce extrinsic error when different raters perform the measurement. Errors were also muscle specific, or related to the measurement set-up. They need to be accounted for, in particular when assessing pre-post interventions or longitudinal follow-up. The parameters of the instrumented spasticity assessment demonstrate a wide range of applications for both research and clinical environments in the quantification of spasticity.

Project description:ObjectivesEvaluation of the inter-rater reliability of clinical assessment methods for pelvic floor muscles and diastasis recti abdominis post partum.DesignA multicentre inter-rater reliability study.SettingThree primary care rehabilitation centres in Sweden.ParticipantsA total of 222 participants were recruited via advertising at Swedish maternity care units and social media. Eligibility for participation included female gender, ≥18 years, at maximum 3 months after childbirth. Exclusion criteria were chronic pelvic girdle pain and/or low back pain and/or pelvic floor tear grade III/IV. At each centre, 2 physiotherapists, with training and experience in pelvic floor assessment, assessed the 222 women according to a standardised protocol in random order.Outcome measuresInter-rater reliability of the assessment of pelvic floor muscle function (involuntary and voluntary contraction and voluntary relaxation) and diastasis recti abdominis (width, depth and bulging).ResultsVaginal palpation of maximal voluntary contraction revealed a kappa value of 0.69 (95% CI 0.62 to 0.76). Assessments of involuntary contraction and voluntary relaxation yielded inconsistent results, with slight-to-moderate weighted kappa values ranging from 0.10 to 0.51. After 2 months of training in applying this method, diastasis recti abdominis width measured at the umbilicus by calliper yielded an intraclass correlation coefficient value of 0.83 (95% CI 0.76 to 0.87). Assessments of diastasis recti abdominis depth and bulging showed moderate kappa values, with reservation for some inconsistency between the centres.ConclusionsVaginal palpation of pelvic floor muscle strength is a reliable method for the postpartum muscle assessment. Additional research is needed to identify reliable assessment method for other pelvic floor muscle functions like involuntary contraction and voluntary relaxation. With some training, a calliper is a reliable instrument for measuring the postpartum diastasis recti abdominis width. This study provides novel thoughts about how to measure diastasis recti abdominis depth and bulging.Trial registration numberNCT03703804.

Dataset Information

Development and inter-rater reliability of the Liverpool adverse drug reaction causality assessment tool.

Aim

Methods

Main outcome measures

Results

Conclusion

Publications

Development and inter-rater reliability of the Liverpool adverse drug reaction causality assessment tool.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets