Systematic characterizations of text similarity in full text biomedical publications.

ABSTRACT:

Background

Computational methods have been used to find duplicate biomedical publications in MEDLINE. Full text articles are becoming increasingly available, yet the similarities among them have not been systematically studied. Here, we quantitatively investigated the full text similarity of biomedical publications in PubMed Central.

Methodology/principal findings

72,011 full text articles from PubMed Central (PMC) were parsed to generate three different datasets: full texts, sections, and paragraphs. Text similarity comparisons were performed on these datasets using the text similarity algorithm eTBLAST. We measured the frequency of similar text pairs and compared it among different datasets. We found that high abstract similarity can be used to predict high full text simil

SUBMITTER: Sun Z 

PROVIDER: S-EPMC2939881 | biostudies-literature | 2010 Sep

REPOSITORIES: biostudies-literature