Dataset Information

Artificial Intelligence for Unstructured Healthcare Data: Application to Coding of Patient Reporting of Adverse Drug Reactions.

ABSTRACT: Adverse drug reaction (ADR) reporting is a major component of drug safety monitoring; its input will, however, only be optimized if systems can manage to deal with its tremendous flow of information, based primarily on unstructured text fields. The aim of this study was to develop an automated system allowing to code ADRs from patient reports. Our system was based on a knowledge base about drugs, enriched by supervised machine learning (ML) models trained on patients reporting data. To train our models, we selected all cases of ADRs reported by patients to a French Pharmacovigilance Centre through a national web-portal between March 2017 and March 2019 (n = 2,058 reports). We tested both conventional ML models and deep-learning models. We performed an external validation using a dataset constituted of a random sample of ADRs reported to the Marseille Pharmacovigilance Centre over the same period (n = 187). Here, we show that regarding area under the curve (AUC) and F-measure, the best model to identify ADRs was gradient boosting trees (LGBM), with an AUC of 0.93 (0.92-0.94) and F-measure of 0.72 (0.68-0.75). This model was run for external validation showing an AUC of 0.91 and a F-measure of 0.58. We evaluated an artificial intelligence pipeline that was found able to learn how to identify correctly ADRs from unstructured data. This result allowed us to start a new study using more data to further improve our performance and offer a tool that is useful in practice to efficiently manage drug safety information.

SUBMITTER: Letinier L

PROVIDER: S-EPMC8359992 | biostudies-literature | 2021 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Artificial Intelligence for Unstructured Healthcare Data: Application to Coding of Patient Reporting of Adverse Drug Reactions.

Létinier Louis L Jouganous Julien J Benkebil Mehdi M Bel-Létoile Alicia A Goehrs Clément C Singier Allison A Rouby Franck F Lacroix Clémence C Miremont Ghada G Micallef Joëlle J Salvo Francesco F Pariente Antoine A

Clinical pharmacology and therapeutics 20210508 2

Adverse drug reaction (ADR) reporting is a major component of drug safety monitoring; its input will, however, only be optimized if systems can manage to deal with its tremendous flow of information, based primarily on unstructured text fields. The aim of this study was to develop an automated system allowing to code ADRs from patient reports. Our system was based on a knowledge base about drugs, enriched by supervised machine learning (ML) models trained on patients reporting data. To train our ...[more]

PMID: 33866552

Similar Datasets

Project description:AIMS:To develop a systematic generic method of enabling patients to report symptoms which they believe to be due to a particular prescribed drug. METHODS:A piloted body system-based questionnaire was distributed to patients registered with 79 medical practices in Grampian prescribed one of nine recently marketed 'black triangle' drugs. These comprised four antidepressants, three antiepileptics and two analgesics. This requested respondents to identify any symptoms experienced over the previous year which they thought could be due to the 'black triangle' drug they had used. A sample of medical records was examined to compare symptoms recorded with those reported by patients. A classification system was developed for the study to enable the assessment of symptoms reported for their potential relationship to patients' drug therapy. All symptoms reported were classified, taking into account information provided by patients on their concomitant drugs and diseases. A specialist pharmacist independently re-classified a sample of the symptoms to validate the process. RESULTS:A 36.3% response rate was obtained (837/2307) with 742 respondents (88.6%) reporting at least one symptom. The median per patient was 6.0 (range 0--71), with almost half (406, 48.5%) reporting fewer than five symptoms. Most symptoms (71.0%) were classified as being probably or possibly related to the drugs studied. Agreement between researcher and specialist on the classification of 75.3% of 716 symptoms was obtained (Kappa=0.563). Responses from patients prescribed antidepressant drugs were more likely to include symptoms potentially caused by these drugs (74.5% of all symptoms reported) than those from patients prescribed analgesics (67.4%) or antiepileptics (65.1%, chi2 = 23.858, d.f. = 2, P < 0.001). Patients reporting large numbers of symptoms were more likely to report some which were classed as unlikely to be an ADR or unattributable (chi2 = 80.587, d.f. = 3, P < 0.001). Of the 742 reporting symptoms in questionnaires, 402 (54.2%) claimed to have reported some or all of these to their doctor. Only 162 (22.6%) of 716 patient-reported symptoms were documented in the primary care medical records of 103 patients prescribed tramadol or venlafaxine. CONCLUSIONS:Respondents were clearly willing to report symptoms, the majority of which were classed as possibly/probably related to the drugs studied. The results suggest that patients do not report all symptoms they suspect to be ADRs to their GP and that GPs do not record all symptoms which may be reported to them. The method could help to identify problems which patients perceive as being related to their drug therapy and contribute to increased ADR reporting.

Project description:IntroductionAdverse drug reaction reports are usually manually assessed by pharmacovigilance experts to detect safety signals associated with drugs. With the recent extension of reporting to patients and the emergence of mass media-related sanitary crises, adverse drug reaction reports currently frequently overwhelm pharmacovigilance networks. Artificial intelligence could help support the work of pharmacovigilance experts during such crises, by automatically coding reports, allowing them to prioritise or accelerate their manual assessment. After a previous study showing first results, we developed and compared state-of-the-art machine learning models using a larger nationwide dataset, aiming to automatically pre-code patients' adverse drug reaction reports.ObjectivesWe aimed to determine the best artificial intelligence model identifying adverse drug reactions and assessing seriousness in patients reports from the French national pharmacovigilance web portal.MethodsReports coded by 27 Pharmacovigilance Centres between March 2017 and December 2020 were selected (n = 11,633). For each report, the Portable Document Format form containing free-text information filled by the patient, and the corresponding encodings of adverse event symptoms (in Medical Dictionary for Regulatory Activities Preferred Terms) and seriousness were obtained. This encoding by experts was used as the reference to train and evaluate models, which contained input data processing and machine-learning natural language processing to learn and predict encodings. We developed and compared different approaches for data processing and classifiers. Performance was evaluated using receiver operating characteristic area under the curve (AUC), F-measure, sensitivity, specificity and positive predictive value. We used data from 26 Pharmacovigilance Centres for training and internal validation. External validation was performed using data from the remaining Pharmacovigilance Centres during the same period.ResultsInternal validation: for adverse drug reaction identification, Term Frequency-Inverse Document Frequency (TF-IDF) + Light Gradient Boosted Machine (LGBM) achieved an AUC of 0.97 and an F-measure of 0.80. The Cross-lingual Language Model (XLM) [transformer] obtained an AUC of 0.97 and an F-measure of 0.78. For seriousness assessment, FastText + LGBM achieved an AUC of 0.85 and an F-measure of 0.63. CamemBERT (transformer) + Light Gradient Boosted Machine obtained an AUC of 0.84 and an F-measure of 0.63. External validation for both adverse drug reaction identification and seriousness assessment tasks yielded consistent and robust results.ConclusionsOur artificial intelligence models showed promising performance to automatically code patient adverse drug reaction reports, with very similar results across approaches. Our system has been deployed by national health authorities in France since January 2021 to facilitate pharmacovigilance of COVID-19 vaccines. Further studies will be needed to validate the performance of the tool in real-life settings.

Project description:BackgroundHealthcare professional's knowledge and attitudes to adverse drug reaction (ADR) and ADR reporting play vital role to report any cases of ADR. Positive attitudes may favour ADR reporting by healthcare professionals. This study was aimed to investigate the attitudes towards and ways to improve adverse drug reaction (ADR) reporting among healthcare professionals working at four Regional Pharmacovigilance Centres (RPCs) of Nepal.MethodsA cross sectional study was done by survey using a self-administered structured questionnaire. The questionnaire was distributed to 450 healthcare professionals working at four RPCs.ResultsThe overall response rate was 74.0%. There were 74.8% of healthcare professionals who had seen patient experiencing an ADR; however, only 20.1% had reported. Reporting form not available (48.1%) and other colleagues not reporting ADR cases (46.9%) would significantly discourage the ADR reporting among healthcare professionals working at four RPCs. Healthcare professionals perceived that seriousness of the reaction (75.6%); unusual reaction (64.6%); reaction to new product (71.2%); new reaction to existing product (70.2%); and confidence in diagnosis of ADR (60.8%) were important factors on the decision to report ADR. Awareness among healthcare professionals (85.9%), training (76.0%), collaboration (67.0%), and involve pharmacist for ADR reporting (63.1%) were mostly recognized ways to improve reporting. Regular newsletter on current awareness in drug safety (71.2%), information on new ADR (65.8%), and international drug safety information (64.0%) were the identified feedbacks they would like to receive from the Nepal pharmacovigilance programme.ConclusionHealthcare professionals working at four RPCs of Nepal have positive attitudes towards ADR reporting. Awareness among healthcare professionals, training and collaboration would likely improve reporting provided they would receive appropriate feedback from the national pharamcovigilance programme.

Project description:BackgroundAdverse drug reaction (ADR) reporting has been studied relatively extensively in all the Nordic countries besides Finland, but no definitive solution to decrease under-reporting has been found. Despite many similarities in reporting, the most notable difference compared to other Nordic countries is that ADR reporting is completely voluntary in Finland.PurposeThe purpose was to examine if voluntary reporting influences healthcare professional (HCP) ADR reporting, why HCPs do not report all suspected ADRs, how could reporting be enhanced, and do we need to develop the process for collecting ADR follow-up (F/U) information from HCPs.MethodsAn open and anonymous questionnaire was developed and made available online at the e-form portal of the University of Helsinki. Trade and area unions distributed the questionnaire to their respective member physicians, nurses, and pharmacists. Two independent coders performed the content analysis of answers to open-ended questions.ResultsA total of 149 responses was received. Two fifths (38%) of the HCPs confirmed that they had not always reported suspected ADRs. The main reason for not reporting was that the ADR was already known. HCPs who had no previous ADR reporting experience did not report ADRs mainly because it was not clear how to report them. Seriousness (chosen by 76%) and unexpectedness of the reaction (chosen by 64%) were the most actuating factors in reporting an ADR. Only 52% of the HCPs had received ADR reporting training and only 16% of the HCPs felt that they had enough information about reporting. Most HCPs felt that ADR F/U requests are justified, and these requests did not affect their ADR reporting willingness.ConclusionsAs in other Nordic countries, ADR under-reporting occurs also in Finland despite differences in reporting guidance. ADR reporting rate could be enhanced by organizing recurring training, information campaigns, and including reporting reminders to the patient information systems that HCPs use. Training should primarily aid in recognizing ADRs, educate in how to report, and promote a reporting culture among HCPs.

Project description:Purpose:Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging. Methods:We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting. Results:For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models' Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation. Conclusion:Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.

Dataset Information

Artificial Intelligence for Unstructured Healthcare Data: Application to Coding of Patient Reporting of Adverse Drug Reactions.

Publications

Artificial Intelligence for Unstructured Healthcare Data: Application to Coding of Patient Reporting of Adverse Drug Reactions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets