Dataset Information

Interrater agreement of two adverse drug reaction causality assessment methods: A randomised comparison of the Liverpool Adverse Drug Reaction Causality Assessment Tool and the World Health Organization-Uppsala Monitoring Centre system.

ABSTRACT:

Introduction

A new method to assess causality of suspected adverse drug reactions, the Liverpool Adverse Drug Reaction Causality Assessment Tool (LCAT), showed high interrater agreement when used by its developers. Our aim was to compare the interrater agreement achieved by LCAT to that achieved by another causality assessment method, the World Health Organization-Uppsala Monitoring Centre system for standardised case causality assessment (WHO-UMC system), in our setting.

Methods

Four raters independently assessed adverse drug reaction causality of 48 drug-event pairs, identified during a hospital-based survey. A randomised design ensured that no washout period was required between assessments with the two methods. We compared the methods' interrater agreement by calculating agreement proportions, kappa statistics, and the intraclass correlation coefficient. We identified potentially problematic questions in the LCAT by comparing raters' responses to individual questions.

Results

Overall unweighted kappa was 0.61 (95% CI 0.43 to 0.80) on the WHO-UMC system and 0.27 (95% CI 0.074 to 0.46) on the LCAT. Pairwise unweighted Cohen kappa ranged from 0.33 to 1.0 on the WHO-UMC system and from 0.094 to 0.71 on the LCAT. The intraclass correlation coefficient was 0.86 (95% CI 0.74 to 0.92) on the WHO-UMC system and 0.61 (95% CI 0.39 to 0.77) on the LCAT. Two LCAT questions were identified as significant points of disagreement.

Discussion

We were unable to replicate the high interrater agreement achieved by the LCAT developers and instead found its interrater agreement to be lower than that achieved when using the WHO-UMC system. We identified potential reasons for this and recommend priority areas for improving the LCAT.

SUBMITTER: Mouton JP

PROVIDER: S-EPMC5325562 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interrater agreement of two adverse drug reaction causality assessment methods: A randomised comparison of the Liverpool Adverse Drug Reaction Causality Assessment Tool and the World Health Organization-Uppsala Monitoring Centre system.

Mouton Johannes P JP Mehta Ushma U Rossiter Dawn P DP Maartens Gary G Cohen Karen K

PloS one 20170224 2

<h4>Introduction</h4>A new method to assess causality of suspected adverse drug reactions, the Liverpool Adverse Drug Reaction Causality Assessment Tool (LCAT), showed high interrater agreement when used by its developers. Our aim was to compare the interrater agreement achieved by LCAT to that achieved by another causality assessment method, the World Health Organization-Uppsala Monitoring Centre system for standardised case causality assessment (WHO-UMC system), in our setting.<h4>Methods</h4> ...[more]

PMID: 28235001

Similar Datasets

Project description:AimTo develop and test a new tool to assess the avoidability of adverse drug reactions that is suitable for use in paediatrics but which is also applicable to a variety of other settings.MethodsThe study involved multiple phases. Preliminary work involved using the Hallas scale and a modification of the existing Hallas scale, to assess two different sets of adverse drug reaction (ADR) case reports. Phase 1 defined, modified and refined a new tool using multidisciplinary teams. Phase 2 involved the assessment of 50 ADR case reports from a prospective study of paediatric inpatients by individual assessors. Phase 3 compared assessments with the new tool for individuals and groups in comparison to the 'gold standard' (the avoidability outcome set by a panel of senior investigators: an experienced clinical pharmacologist, paediatrician and pharmacist).Main outcome measuresInter-rater reliability (IRR), measure of disagreement and utilization of avoidability categories.ResultsPreliminary work-Pilot phase: results for the original Hallas cases were fair and pairwise kappa scores ranged from 0.21 to 0.36. Results for the modified Hallas cases were poor, pairwise kappa scores ranged from 0.06 to 0.16. Phase 1: on initial use of the new tool, agreement between the two multidisciplinary groups was found on 13/20 cases with a kappa score of 0.29 (95% CI -0.04 to 0.62). Phase 2: the assessment of 50 ADR case reports by six individual reviewers yielded pairwise kappa scores ranging from poor to good 0.12 to 0.75 and percentage exact agreement (%EA) ranged from 52-90%. Phase 3: Percentage exact agreement ranged from 35-70%. Overall, individuals had better agreement with the 'gold standard'.ConclusionAvoidability assessment is feasible but needs careful attention to methods. The Liverpool ADR avoidability assessment tool showed mixed IRR. We have developed and validated a method for assessing the avoidability of ADRs that is transparent, more objective than previous methods and that can be used by individuals or groups.

Project description:Adverse drug events (ADEs) are the fifth leading cause of death and thus responsible for a large number of hospital admissions in all over the globe. This study was aimed to assess the antibiotics associated preventability of ADEs and causality of adverse drug reactions (ADRs) among hospitalized patients.A prospective, cross-sectional, observational study was conducted in four tertiary care public sector hospitals of Lahore, Pakistan. Study population consisted of hospitalized patients who were prescribed with one or more antibiotics. Data were collected between 1st January, 2017 and 31st June, 2017 from 1,249 patients (384 patients aged ? 18 years and 865 patients aged >18 years). Schumock and Thornton scale was used to assess the preventability of the ADEs. Medication errors (MEs) that caused preventable ADEs were assessed by MEs tracking form while Naranjo score was used to evaluate the causal relation of ADRs with the antibiotics. Statistical Package for the Social Sciences (IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.) and Microsoft Excel (MS Office, 2010) were used for data analysis.2,686 antibiotics were prescribed to 1,249 patients. Among them, fluoroquinolones (11.8%), macrolides (11.6%) and cephalosporins (10.9%) were the most frequently prescribed antibiotics. The most affected organ system by antibiotics associated ADEs was gastrointestinal tract. A total of 486 ADEs were found. The preventability assessment revealed that most of the ADEs (58.4%) were preventable (43.6% of the ADEs were definitely preventable while 14.8% were probably preventable) and caused by MEs including wrong drug (40.1%) and monitoring errors (25.0%), during the stage of physician ordering (22.2%) and patient monitoring (21.1%). The errors were caused due to non-adherence of policies (38.4%) and lack of information about antibiotics (32%). Most of the non-preventable ADEs or ADRs among adults and children were "probable" (35.5%) and "possible" (35.8%), respectively. Logistic regression analysis revealed that ADEs were significantly less among females (OR = 0.047, 95%CI = 0.018-0.121, p-value = <0.001), patients aged 18-52 years (OR = 0.041, 95%CI = 0.013-0.130, p-value = <0.001), tuberculosis patients (OR = 0.304, 95%CI = 0.186-0.497, p-value = <0.001), patients with acute respiratory tract infections (OR = 0.004, 95%CI = 0.01-0.019, p-value = <0.001) and among the patients prescribed with 2 antibiotics per prescription (OR = 0.455, 95%CI = 0.319-0.650, p-value = <0.001).According to preventability assessment most of the ADEs were definitely preventable and caused by MEs due to non-adherence of policies and lack of information about antibiotics. The causality assessment of non-preventable ADEs showed that most of the ADRs were probable and possible.

Project description:OBJECTIVE:The presence of interictal epileptiform discharges (IED) in the electroencephalogram (EEG) is a key finding in the medical workup of a patient with suspected epilepsy. However, inter-rater agreement (IRA) regarding the presence of IED is imperfect, leading to incorrect and delayed diagnoses. An improved understanding of which IED attributes mediate expert IRA might help in developing automatic methods for IED detection able to emulate the abilities of experts. Therefore, using a set of IED scored by a large number of experts, we set out to determine which attributes of IED predict expert agreement regarding the presence of IED. METHODS:IED were annotated on a 5-point scale by 18 clinical neurophysiologists within 200 30-s EEG segments from recordings of 200 patients. 5538 signal analysis features were extracted from the waveforms, including wavelet coefficients, morphological features, signal energy, nonlinear energy operator response, electrode location, and spectrogram features. Feature selection was performed by applying elastic net regression and support vector regression (SVR) was applied to predict expert opinion, with and without the feature selection procedure and with and without several types of signal normalization. RESULTS:Multiple types of features were useful for predicting expert annotations, but particular types of wavelet features performed best. Local EEG normalization also enhanced best model performance. As the size of the group of EEGers used to train the models was increased, the performance of the models leveled off at a group size of around 11. CONCLUSIONS:The features that best predict inter-rater agreement among experts regarding the presence of IED are wavelet features, using locally standardized EEG. Our models for predicting expert opinion based on EEGer's scores perform best with a large group of EEGers (more than 10). SIGNIFICANCE:By examining a large group of EEG signal analysis features we found that wavelet features with certain wavelet basis functions performed best to identify IEDs. Local normalization also improves predictability, suggesting the importance of IED morphology over amplitude-based features. Although most IED detection studies in the past have used opinion from three or fewer experts, our study suggests a "wisdom of the crowd" effect, such that pooling over a larger number of expert opinions produces a better correlation between expert opinion and objectively quantifiable features of the EEG.

Project description:PURPOSE:Many neonates undergo electroencephalogram (EEG) monitoring to identify and manage acute symptomatic seizures. Information about brain function contained in the EEG background data may also help predict neurobehavioral outcomes. For EEG background features to be useful as prognostic indicators, the interpretation of these features must be standardized across electroencephalographers. We aimed at determining the interrater and intrarater agreement among electroencephalographers interpreting neonatal EEG background patterns. METHODS:Five neonatal electroencephalographers reviewed 5-to-7.5-minute epochs of EEG from full-term neonates who underwent continuous conventional EEG monitoring. The EEG assessment tool used to classify background patterns was based on the American Clinical Neurophysiology Society's guideline for neonatal EEG terminology. Interrater and intrarater agreement were measured using Kappa coefficients. RESULTS:Interrater agreement was consistently highest for voltage (binary: substantial, kappa = 0.783; categorical: moderate, kappa = 0.562), seizure presence (fair-substantial; kappa = 0.375-0.697), continuity (moderate; kappa = 0.481), burst voltage (moderate; kappa = 0.574), suppressed background presence (moderate-substantial; kappa = 0.493-0.643), delta activity presence (fair-moderate; kappa = 0.369-0.432), theta activity presence (fair-moderate; kappa = 0.347-0.600), presence of graphoelements (fair; kappa = 0.381), and overall impression (binary: moderate, kappa = 0.495; categorical: fair-moderate, kappa = 0.347, 0.465). Agreement was poor or inconsistent for all other patterns. Intrarater agreement was variable, with highest average agreement for voltage (binary: substantial, kappa = 0.75; categorical: substantial, kappa = 0.714) and highest consistent agreement for continuity (moderate-substantial; kappa = 0.43-0.67) and overall impression (moderate-substantial; kappa = 0.42-0.68). CONCLUSIONS:This study demonstrates substantial variability in neonatal EEG background interpretation across electroencephalographers, indicating a need for educational and technological strategies aimed at improving performance.

Dataset Information

Interrater agreement of two adverse drug reaction causality assessment methods: A randomised comparison of the Liverpool Adverse Drug Reaction Causality Assessment Tool and the World Health Organization-Uppsala Monitoring Centre system.

Introduction

Methods

Results

Discussion

Publications

Interrater agreement of two adverse drug reaction causality assessment methods: A randomised comparison of the Liverpool Adverse Drug Reaction Causality Assessment Tool and the World Health Organization-Uppsala Monitoring Centre system.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure