Dataset Information

VigiGrade: a tool to identify well-documented individual case reports and highlight systematic data quality issues.

ABSTRACT:

Background

Individual case safety reports of suspected harm from medicines are fundamental to post-marketing surveillance. Their value is directly proportional to the amount of clinically relevant information they include. To improve the quality of the data, communication between stakeholders is essential and can be facilitated by a simple score and visualisation of the results.

Objective

The objective of this study was to propose a measure of completeness and identify predictors of well-documented reports, globally.

Methods

The Uppsala Monitoring Centre has developed the vigiGrade completeness score to measure the amount of clinically relevant information in structured format, without reflecting whether the information establishes causality between the drug and adverse event. The vigiGrade completeness score (C) starts at 1 for reports with information on time-to-onset, age, sex, indication, outcome, report type, dose, country, primary reporter and comments. For each missing dimension, a penalty is detracted which varies with clinical relevance. We classified reports with C > 0.8 as well-documented and identified all such reports in the WHO global individual case safety report database, VigiBase, from 2007 to January 2012. We utilised odds ratios with statistical shrinkage to identify subgroups with unexpectedly high proportions of well-documented reports.

Results

Altogether, 430,000 (13 %) of the studied reports achieved C > 0.8 in VigiBase. For VigiBase as a whole, the median completeness was 0.41 with an interquartile range of 0.26-0.63. Two out of three well-documented reports come from Europe, and two out of three from physicians. Among the countries with more than 1,000 reports in total, the highest rate of well-documented reports is 65 % in Italy. Tunisia, Spain, Portugal, Croatia and Denmark each have rates above 50 %, and another 20 countries have rates above 30 %. On the whole, 24 % of the reports from physicians are well-documented compared with only 4 % for consumers/non-health professionals. Notably, Denmark and Norway have more than 50 % well-documented reports from consumers/non-health professionals and higher rates than for physicians. The rate of well-documented reports for the E2B format is 11 % compared with 22 % for the older INTDIS (International Drug Information System) format. However, for E2B reports entered via the WHO programme's e-reporting system VigiFlow, the rate is 29 %.

Conclusion

Overall, only one report in eight provides the desired level of information, but much higher proportions are observed for individual countries. Physicians and e-reporting tools also generate greater proportions of well-documented reports overall. Reports from consumers/non-health professionals in specific regions have excellent quality, which illustrates their potential for the future. vigiGrade has already provided valuable information by highlighting data quality issues both in Italy and the USA.

SUBMITTER: Bergvall T

PROVIDER: S-EPMC6447519 | biostudies-literature | 2014 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

vigiGrade: a tool to identify well-documented individual case reports and highlight systematic data quality issues.

Bergvall Tomas T Norén G Niklas GN Lindquist Marie M

Drug safety 20140101 1

<h4>Background</h4>Individual case safety reports of suspected harm from medicines are fundamental to post-marketing surveillance. Their value is directly proportional to the amount of clinically relevant information they include. To improve the quality of the data, communication between stakeholders is essential and can be facilitated by a simple score and visualisation of the results.<h4>Objective</h4>The objective of this study was to propose a measure of completeness and identify predictors ...[more]

PMID: 24343765

Similar Datasets

Project description:IntroductionThere is increasing interest in social digital media (SDM) as a data source for pharmacovigilance activities; however, SDM is considered a low information content data source for safety data. Given that pharmacovigilance itself operates in a high-noise, lower-validity environment without objective 'gold standards' beyond process definitions, the introduction of large volumes of SDM into the pharmacovigilance workflow has the potential to exacerbate issues with limited manual resources to perform adverse event identification and processing. Recent advances in medical informatics have resulted in methods for developing programs which can assist human experts in the detection of valid individual case safety reports (ICSRs) within SDM.ObjectiveIn this study, we developed rule-based and machine learning (ML) models for classifying ICSRs from SDM and compared their performance with that of human pharmacovigilance experts.MethodsWe used a random sampling from a collection of 311,189 SDM posts that mentioned Roche products and brands in combination with common medical and scientific terms sourced from Twitter, Tumblr, Facebook, and a spectrum of news media blogs to develop and evaluate three iterations of an automated ICSR classifier. The ICSR classifier models consisted of sub-components to annotate the relevant ICSR elements and a component to make the final decision on the validity of the ICSR. Agreement with human pharmacovigilance experts was chosen as the preferred performance metric and was evaluated by calculating the Gwet AC1 statistic (gKappa). The best performing model was tested against the Roche global pharmacovigilance expert using a blind dataset and put through a time test of the full 311,189-post dataset.ResultsDuring this effort, the initial strict rule-based approach to ICSR classification resulted in a model with an accuracy of 65% and a gKappa of 46%. Adding an ML-based adverse event annotator improved the accuracy to 74% and gKappa to 60%. This was further improved by the addition of an additional ML ICSR detector. On a blind test set of 2500 posts, the final model demonstrated a gKappa of 78% and an accuracy of 83%. In the time test, it took the final model 48 h to complete a task that would have taken an estimated 44,000 h for human experts to perform.ConclusionThe results of this study indicate that an effective and scalable solution to the challenge of ICSR detection in SDM includes a workflow using an automated ML classifier to identify likely ICSRs for further human SME review.

Project description:BackgroundCentral nervous system (CNS) melioidosis is rare. However, delayed diagnosis and treatment could lead to fatality. To identify knowledge of CNS melioidosis, we systematically review case reports and case series.Methodology/principal findingsWe searched through PubMed, Web of Science and Thai-Journal Citation Index databases as well as Google Scholar with the last date on July 10, 2018. The diagnosis of CNS melioidosis had to be confirmed with culture, serology or polymerase chain reaction. We excluded the animal cases and the studies that the clinical data were not available. We identified 1170 relevant studies, while 70 studies with a total of 120 patients were analyzed. Ninety-three percent of patients were reported from the endemic area of melioidosis. Median age was 40 years (IQR 18-53), and 70% were men. A total of 60% had one or more risk factors for melioidosis. The median duration from clinical onset to diagnosis was ten days (IQR 5-25). Fever (82%), headache (54%), unilateral weakness (57%) and cranial nerve deficits (52%) are among the prominent presentation. Most patient (67%) had at least one extraneurological organ involvement. The CSF profile mostly showed mononuclear pleocytosis (64%), high protein (93%) and normal glucose (66%). The rim-enhancing pattern (78%) is the most frequent neuroimaging finding in encephalomyelitis and brain abscess patients. Both brainstem (34%) and frontal lobe (34%) are the most affected locations. Mortality rate was 20%.Conclusions/significanceThis study is the most extensive systematic review of case reports and case series of CNS melioidosis in all age groups. However, the results should be cautiously interpreted due to the missing data issue. The propensity of brainstem involvement which correlates with prominent cranial nerve deficits is the characteristic of CNS melioidosis especially encephalomyelitis type. The presenting features of fever and neurological deficits (especially cranial nerve palsies) along with the mononuclear CSF pleocytosis in a patient who lives in the endemic area and also has the risk factor for melioidosis should raise the CNS melioidosis as the differential diagnosis.