Dataset Information

Using semantics to scale up evidence-based chemical risk-assessments.

ABSTRACT:

Background

The manual processes used for risk assessments are not scaling to the amount of data available. Although automated approaches appear promising, they must be transparent in a public policy setting.

Objective

Our goal is to create an automated approach that moves beyond retrieval to the extraction step of the information synthesis process, where evidence is characterized as supporting, refuting, or neutral with respect to a given outcome.

Methods

We combine knowledge resources and natural language processing to resolve coordinated ellipses and thus avoid surface level differences between concepts in an ontology and outcomes in an abstract. As with a systematic review, the search criterion, and inclusion and exclusion criterion are explicit.

Results

The system scales to 482K abstracts on 27 chemicals. Results for three endpoints that are critical for cancer risk assessments show that refuting evidence (where the outcome decreased) was higher for cell proliferation (45.9%), and general cell changes (37.7%) than for cell death (25.0%). Moreover, cell death was the only end point where supporting claims were the majority (61.3%). If the number of abstracts that measure an outcome was used as a proxy for association there would be a stronger association with cell proliferation than cell death (20/27 chemicals). However, if the amount of supporting evidence was used (where the outcome increased) the conclusion would change for 21/27 chemicals (20 from proliferation to death and 1 from death to proliferation).

Conclusions

We provide decision makers with a visual representation of supporting, neutral, and refuting evidence whilst maintaining the reproducibility and transparency needed for public policy. Our findings show that results from the retrieval step where the number of abstracts that measure an outcome are reported can be misleading if not accompanied with results from the extraction step where the directionality of the outcome is established.

SUBMITTER: Blake C

PROVIDER: S-EPMC8673667 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe objectives of South Africa's electronic health (eHealth) strategy recognize the value proposition that telemedicine practices hold for rural and urban referrals, but a lack of accepted and formalized scale-up has impeded realization of benefits. While both synchronous and asynchronous teledermatology exist, these remain localized and not scaled-up. Skin pathology is often the first sign of an HIV/AIDS infection, which remains a major cause of morbidity and mortality in South Africa. It is essential to replace the current inefficient dermatology referral process with a swift, organized, and efficacious one.ObjectiveThe objective of this study is to present an evidenced-based teledermatology scale-up framework (TDSF) and implementation roadmap (TDSF-IR).MethodsA qualitative method with a design science research process model was used which consisted of 5 phases: (1) Awareness, which confirmed the need for an evidence-based TDSF and supporting TDSF-IR; (2) Suggestion, where a proposal was delivered on how to develop a TDSF and TDSF-IR; (3) Development, where we identified recommended design requirements and used these to identify and critique existing teledermatology or related scale-up frameworks; (4) Evaluation and validation, where we assessed outputs of the development phase against the design requirements and validated by confirming the veracity of the TDSF and TDSF-IR (validation involved 4 key senior teledermatology stakeholders using a questionnaire with a 5-point Likert scale); and (5) Conclusion, where validation results were used to finalize and communicate the TDSF and TDSF-IR to users.ResultsThe study identified 5 TDSF components: eHealth building blocks, eHealth strategic objectives and budget, scale-up continuum periods, scale-up drivers, and scale-up phases. In addition, 36 subcomponents were identified. Each was further characterized and described to enable design of the final evidence-based TDSF. An implementation roadmap (TDSF-IR) was also prepared as a guide for an implementer with step-by-step instructions for application of the TDSF. For the validation study of the TDSF and supporting TDSF-IR, 4 purposively selected key senior teledermatology management stakeholders were asked if they found it useful as a guide to assist the South African public health system with teledermatology scale-up. The mean (SD) of Likert-scale rating was 4.0 (0.53) where 4=Agree and 33 of 36 responses were either agree or strongly agree.ConclusionsThis study developed a TDSF and supporting roadmap (TDSF-IR) that are evidence-based. The proposed approach and described tools could be adapted to assist with ensuring scale-up and sustainability for other eHealth practices in other locations.

Project description:BackgroundIn patients with autoimmune encephalitis (AE), the prediction of progression to a critically ill status is challenging but essential. However, there is currently no standard prediction model that comprehensively integrates the disease severity and other clinical features. The clinical assessment scale in autoimmune encephalitis (CASE) and the modified Rankin Scale (mRS) have both been applied for evaluating the severity of AE. Here, by combining the two scales and other clinical characteristics, we aimed to investigate risk factors and construct prediction models for early critical care needs of AE patients.MethodsDefinite and probable AE patients who were admitted to the neurology department of Tongji Hospital between 2013 and 2021 were consecutively enrolled. The CASE and mRS scores were used to evaluate the overall symptom severity at the time of hospital admission. Using logistic regression analysis, we analyzed the association between the total scores of the two scales and critical illness individually and then we evaluated this association in combination with other clinical features to predict early intensive care unit (ICU) admission. Finally, we constructed four prediction models and compared their performances.ResultsOf 234 patients enrolled, forty developed critical illness and were early admitted to the ICU (within 14 days of hospitalization). Four prediction models were generated; the models were named CASE, CASE-plus (CASE + prodromal symptoms + elevated fasting blood glucose + elevated cerebrospinal fluid (CSF) white blood cell (WBC) count), mRS and mRS-plus (mRS + prodromal symptoms + abnormal EEG results + elevated fasting blood glucose + elevated CSF WBC count) and had areas under the ROC curve of 0.850, 0.897, 0.695 and 0.833, respectively. All four models had good calibrations. In general, the models containing "CASE" performed better than those including "mRS", and the CASE-plus model demonstrated the best performance.ConclusionOverall, the symptom severity at hospital admission, as defined by CASE or mRS, could predict early ICU admission, especially when assessed by CASE. Adding other clinical findings, such as prodromal symptoms, an increased fasting blood glucose level and an increased CSF WBC count, could improve the predictive efficacy.

Project description:Evidence-based medicine (EBM) has been defined as a process involving five actions: asking, acquiring, appraising, applying, and assessing. Several attempts have been made to create and validate tools to assess EBM aptitude. The newest testing instrument, the ACE tool, which is a 15-question true/false exam, has not been directly compared to the more established Fresno test, which is composed of 12 in-depth short-answer questions. Although both were designed to test Steps 1-4 of the five-step EBM process, it is unclear whether they examine the same things or whether one is superior. To our knowledge there is not a widely used standard for EBM assessment despite the broad requirements for inclusion of EBM in both undergraduate and graduate medical education. It was hypothesized that these instruments do not correlate between one another, based on inherent differences between them, including assessment format, grading method, and scoring range. The authors sought to examine whether a correlation between the results of these two instruments exists in a population of U.S. medical students. A retrospective cohort study of 158 fourth-year U.S. medical students in academic year 2018-2019 was conducted. All students were exposed to a focused EBM curriculum, consisting of three guided discussions of separate journal articles clinically relevant to the practice of emergency medicine. Outcomes measured included scores on both the ACE tool and Fresno test using descriptive statistics. Spearman's rho was used to determine the correlation between the ACE and Fresno scores for each student among the entire group. A subgroup analysis was performed to assess for correlations at more extreme data points. The median scores on the ACE tool and Fresno test were 66.7% and 62.7%. There was no statistically significant correlation between the results of these two assessments (Spearman's rho 0.023, p = 0.774) in our population. The scores from the subgroup of advanced performers on the Fresno test showed a weak statistically significant positive correlation (p = 0.045) to advanced scores on the ACE tool. No other subgroups showed statistically significant correlation. In our population of U.S. medical students, the results of two known EBM assessment instruments do not correlate with one another. The assessments may differ in what categories of learning they measure or in generalizability or perhaps in what depth of understanding they test overall. Further study is needed to determine what each instrument is measuring and whether there is demonstrable variation across populations.

Project description:IntroductionDepression is an affective disorder that contributes to a significant global burden of disease. Measurement-Based Care (MBC) is advocated during the full course management, with symptom assessment being an important component. Rating scales are widely used as convenient and powerful assessment tool, but they are influenced by the subjectivity and consistency of the raters. The assessment of depressive symptoms is usually conducted with a clear purpose and restricted content, such as clinical interviews based on the Hamilton Depression Rating Scale (HAMD), so that the results are easy to obtain and quantify. Artificial Intelligence (AI) techniques are used due to their objective, stable and consistent performance, and are suitable for assessing depressive symptoms. Therefore, this study applied Deep Learning (DL)-based Natural Language Processing (NLP) techniques to assess depressive symptoms during clinical interviews; thus, we proposed an algorithm model, explored the feasibility of the techniques, and evaluated their performance.MethodsThe study included 329 patients with Major Depressive Episode. Clinical interviews based on the HAMD-17 were conducted by trained psychiatrists, whose speech was simultaneously recorded. A total of 387 audio recordings were included in the final analysis. A deeply time-series semantics model for the assessment of depressive symptoms based on multi-granularity and multi-task joint training (MGMT) is proposed.ResultsThe performance of MGMT is acceptable for assessing depressive symptoms with an F1 score (a metric of model performance, the harmonic mean of precision and recall) of 0.719 in classifying the four-level severity of depression and an F1 score of 0.890 in identifying the presence of depressive symptoms.DisscussionThis study demonstrates the feasibility of the DL and the NLP techniques applied to the clinical interview and the assessment of depressive symptoms. However, there are limitations to this study, including the lack of adequate samples, and the fact that using speech content alone to assess depressive symptoms loses the information gained through observation. A multi-dimensional model combing semantics with speech voice, facial expression, and other valuable information, as well as taking into account personalized information, is a possible direction in the future.