Unknown

Dataset Information

0

Imitating manual curation of text-mined facts in biomedicine.


ABSTRACT: Text-mining algorithms make mistakes in extracting facts from natural-language texts. In biomedical applications, which rely on use of text-mined data, it is critical to assess the quality (the probability that the message is correctly extracted) of individual facts--to resolve data conflicts and inconsistencies. Using a large set of almost 100,000 manually produced evaluations (most facts were independently reviewed more than once, producing independent evaluations), we implemented and tested a collection of algorithms that mimic human evaluation of facts provided by an automated information-extraction system. The performance of our best automated classifiers closely approached that of our human evaluators (ROC score close to 0.95). Our hypothesis is that, were we to use a larger number of human experts to evaluate any given sentence, we could implement an artificial-intelligence curator that would perform the classification job at least as accurately as an average individual human evaluator. We illustrated our analysis by visualizing the predicted accuracy of the text-mined relations involving the term cocaine.

SUBMITTER: Rodriguez-Esteban R 

PROVIDER: S-EPMC1560402 | biostudies-literature |

REPOSITORIES: biostudies-literature

Similar Datasets

| S-EPMC3842776 | biostudies-literature
| S-EPMC2768719 | biostudies-literature
| S-EPMC6501925 | biostudies-literature
| S-EPMC6794279 | biostudies-literature
| S-EPMC6917032 | biostudies-literature
| S-EPMC3102890 | biostudies-other
| S-EPMC2516305 | biostudies-literature
| S-EPMC2767227 | biostudies-literature
| S-EPMC8637219 | biostudies-literature
| S-EPMC3506574 | biostudies-literature