Words with Consistent Diachronic Usage Patterns are Learned Earlier: A Computational Analysis Using Temporally Aligned Word Embeddings.
Ontology highlight
ABSTRACT:
SUBMITTER: Cassani G
PROVIDER: S-EPMC8244097 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:By leveraging Phonology-to-Semantics Consistency (PSC), which quantifies form-meaning systematicity as the semantic similarity between a target word and its phonological nearest neighbours, we document a unique effect of systematicity on Age of Acquisition (AoA). This effect is also found after controlling for the effect of neighbourhood density measured for word forms and lexical semantics and several other standard predictors of AoA. Moreover, we show that the effect of systematicity is not reducible to iconicity. Finally, we extensively probe the reliability of this finding by testing different statistical models, analysing systematicity in phonology and orthography and implementing random baselines, reporting a robust, unique negative effect of systematicity on AoA, such that more systematic words tend to be learned earlier. We discuss the findings in the light of studies on non-arbitrary form-meaning mappings and their role in language learning, focusing on the analogical process at the interface of form and meaning upon which PSC is based and how it could help children infer the semantics of novel words when context is scarce or uninformative, ultimately speeding up word learning.
| S-EPMC9245153 | biostudies-literature
Project description:Lexical semantic change (detecting shifts in the meaning and usage of words) is an important task for social and cultural studies as well as for Natural Language Processing applications. Diachronic word embeddings (time-sensitive vector representations of words that preserve their meaning) have become the standard resource for this task. However, given the significant computational resources needed for their generation, very few resources exist that make diachronic word embeddings available to the scientific community. In this paper we present DUKweb, a set of large-scale resources designed for the diachronic analysis of contemporary English. DUKweb was created from the JISC UK Web Domain Dataset (1996-2013), a very large archive which collects resources from the Internet Archive that were hosted on domains ending in '.uk'. DUKweb consists of a series word co-occurrence matrices and two types of word embeddings for each year in the JISC UK Web Domain dataset. We show the reuse potential of DUKweb and its quality standards via a case study on word meaning change detection.
| S-EPMC8520005 | biostudies-literature
Project description:A surprisingly small portion of reading research has been dedicated to investigating how the visual word recognition process is influenced by embedded words (e.g., 'arm' in 'charm'), and no research has yet investigated embedded words in a natural reading setting. Covering this issue, the present work reports analyses of eye-tracking data from the GECO bilingual book reading corpus. Word viewing times were analyzed as a function of the number, frequency and proportional length of embedded words. We anticipated two scenarios: embedded words would either facilitate processing due to increased word-letter feedback, or inhibit processing due to increased lexical competition. A main facilitatory effect of embedded words on the recognition process was established, with an increasing number of embedded words resulting in shorter word viewing times and fewer fixations. This pattern was depicted by readers of Dutch as well as readers of English. Long, high-frequency embedded words formed an exception however, as these led to inhibition (Dutch participants) or a null-effect (English participants). The present results indicate that both scenarios outlined above are at play, but with a theoretical constraint on the role of word-to-word inhibitory connections. Specifically, such connections may predominantly exist among words of similar length. Hence, embedded words generally facilitate processing through word-letter feedback, but this facilitatory effect is countered by word-to-word inhibition if the embedded word's length approximates that of its superset.
| S-EPMC6634368 | biostudies-literature
Project description:MotivationMachine-learning models trained on protein sequences and their measured functions can infer biological properties of unseen sequences without requiring an understanding of the underlying physical or biological mechanisms. Such models enable the prediction and discovery of sequences with optimal properties. Machine-learning models generally require that their inputs be vectors, and the conversion from a protein sequence to a vector representation affects the model's ability to learn. We propose to learn embedded representations of protein sequences that take advantage of the vast quantity of unmeasured protein sequence data available. These embeddings are low-dimensional and can greatly simplify downstream modeling.ResultsThe predictive power of Gaussian process models trained using embeddings is comparable to those trained on existing representations, which suggests that embeddings enable accurate predictions despite having orders of magnitude fewer dimensions. Moreover, embeddings are simpler to obtain because they do not require alignments, structural data, or selection of informative amino-acid properties. Visualizing the embedding vectors shows meaningful relationships between the embedded proteins are captured.Availability and implementationThe embedding vectors and code to reproduce the results are available at https://github.com/fhalab/embeddings_reproduction/.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC6061698 | biostudies-literature
Project description:OBJECTIVE:In this work, we introduce a privacy technique for anonymizing clinical notes that guarantees all private health information is secured (including sensitive data, such as family history, that are not adequately covered by current techniques). MATERIALS AND METHODS:We employ a new "random replacement" paradigm (replacing each token in clinical notes with neighboring word vectors from the embedding space) to achieve 100% recall on the removal of sensitive information, unachievable with current "search-and-secure" paradigms. We demonstrate the utility of this paradigm on multiple corpora in a diverse set of classification tasks. RESULTS:We empirically evaluate the effect of our anonymization technique both on upstream and downstream natural language processing tasks to show that our perturbations, while increasing security (ie, achieving 100% recall on any dataset), do not greatly impact the results of end-to-end machine learning approaches. DISCUSSION:As long as current approaches utilize precision and recall to evaluate deidentification algorithms, there will remain a risk of overlooking sensitive information. Inspired by differential privacy, we sought to make it statistically infeasible to recreate the original data, although at the cost of readability. We hope that the work will serve as a catalyst to further research into alternative deidentification methods that can address current weaknesses. CONCLUSION:Our proposed technique can secure clinical texts at a low cost and extremely high recall with a readability trade-off while remaining useful for natural language processing classification tasks. We hope that our work can be used by risk-averse data holders to release clinical texts to researchers.
| S-EPMC7309261 | biostudies-literature
Project description:Word embeddings are a powerful machine-learning framework that represents each English word by a vector. The geometric relationship between these vectors captures meaningful semantic relationships between the corresponding words. In this paper, we develop a framework to demonstrate how the temporal dynamics of the embedding helps to quantify changes in stereotypes and attitudes toward women and ethnic minorities in the 20th and 21st centuries in the United States. We integrate word embeddings trained on 100 y of text data with the US Census to show that changes in the embedding track closely with demographic and occupation shifts over time. The embedding captures societal shifts-e.g., the women's movement in the 1960s and Asian immigration into the United States-and also illuminates how specific adjectives and occupations became more closely associated with certain populations over time. Our framework for temporal analysis of word embedding opens up a fruitful intersection between machine learning and quantitative social science.
| S-EPMC5910851 | biostudies-literature
Project description:BackgroundThere has been increasing interest in machine learning based natural language processing (NLP) methods in radiology; however, models have often used word embeddings trained on general web corpora due to lack of a radiology-specific corpus.PurposeWe examined the potential of Radiopaedia to serve as a general radiology corpus to produce radiology specific word embeddings that could be used to enhance performance on a NLP task on radiological text.Materials and methodsEmbeddings of dimension 50, 100, 200, and 300 were trained on articles collected from Radiopaedia using a GloVe algorithm and evaluated on analogy completion. A shallow neural network using input from either our trained embeddings or pre-trained Wikipedia 2014 + Gigaword 5 (WG) embeddings was used to label the Radiopaedia articles. Labeling performance was evaluated based on exact match accuracy and Hamming loss. The McNemar's test with continuity and the Benjamini-Hochberg correction and a 5×2 cross validation paired two-tailed t-test were used to assess statistical significance.ResultsFor accuracy in the analogy task, 50-dimensional (50-D) Radiopaedia embeddings outperformed WG embeddings on tumor origin analogies (p < 0.05) and organ adjectives (p < 0.01) whereas WG embeddings tended to outperform on inflammation location and bone vs. muscle analogies (p < 0.01). The two embeddings had comparable performance on other subcategories. In the labeling task, the Radiopaedia-based model outperformed the WG based model at 50, 100, 200, and 300-D for exact match accuracy (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively) and Hamming loss (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively).ConclusionWe have developed a set of word embeddings from Radiopaedia and shown that they can preserve relevant medical semantics and augment performance on a radiology NLP task. Our results suggest that the cultivation of a radiology-specific corpus can benefit radiology NLP models in the future.
| S-EPMC7856086 | biostudies-literature
Project description:Distributed word representations have become an essential foundation for biomedical natural language processing (BioNLP), text mining and information retrieval. Word embeddings are traditionally computed at the word level from a large corpus of unlabeled text, ignoring the information present in the internal structure of words or any information available in domain specific structured resources such as ontologies. However, such information holds potentials for greatly improving the quality of the word representation, as suggested in some recent studies in the general domain. Here we present BioWordVec: an open set of biomedical word vectors/embeddings that combines subword information from unlabeled biomedical text with a widely-used biomedical controlled vocabulary called Medical Subject Headings (MeSH). We assess both the validity and utility of our generated word embeddings over multiple NLP tasks in the biomedical domain. Our benchmarking results demonstrate that our word embeddings can result in significantly improved performance over the previous state of the art in those challenging tasks.
| S-EPMC6510737 | biostudies-literature
Project description:MotivationInter-organ/inter-tissue communication is central to multi-cellular organisms including humans, and mapping inter-tissue interactions can advance system-level whole-body modeling efforts. Large volumes of biomedical literature have fostered studies that map within-tissue or tissue-agnostic interactions, but literature-mining studies that infer inter-tissue relations, such as between hormones and genes are solely missing.ResultsWe present a first study to predict from biomedical literature the hormone-gene associations mediating inter-tissue signaling in the human body. Our BioEmbedS* models use neural network-based Biomedical word Embeddings with a Support Vector Machine classifier to predict if a hormone-gene pair is associated or not, and whether an associated gene is involved in the hormone's production or response. Model training relies on our unified dataset Hormone-Gene version 1 of ground-truth associations between genes and endocrine hormones, which we compiled and carefully balanced in the embedded space to handle data disparities, such as between poorly- versus well-studied hormones. Our BioEmbedS model recapitulates known gene mediators of tissue-tissue signaling with 70.4% accuracy; predicts novel inter-tissue communication genes in humans, which are enriched for hormone-related disorders; and generalizes well to mouse, thereby holding promise for its extension to other multi-cellular organisms as well.Availability and implementationFreely available at https://cross-tissue-signaling.herokuapp.com are our model predictions & datasets; https://github.com/BIRDSgroup/BioEmbedS has all relevant code.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC9563690 | biostudies-literature
Project description:This research evaluates the performance of areal interpolation coupled with dasymetric refinement to estimate different demographic attributes, namely population sub-groups based on race, age structure and urban residence, within consistent census tract boundaries from 1990 to 2010 in Massachusetts. The creation of such consistent estimates facilitates the study of the nuanced micro-scale evolution of different aspects of population, which is impossible using temporally incompatible small-area census geographies from different points in time. Various unexplored ancillary variables, including the Global Human Settlement Layer (GHSL), the National Land-Cover Database (NLCD), parcels, building footprints and the proprietary ZTRAX® dataset are utilized for dasymetric refinement prior to areal interpolation to examine their effectiveness in improving the accuracy of multi-temporal population estimates. Different areal interpolation methods including Areal Weighting (AW), Target Density Weighting (TDW), Expectation Maximization (EM) and its data-extended approach are coupled with different dasymetric refinement scenarios based on these ancillary variables. The resulting consistent small area estimates of white and black subpopulations, people of age 18-65 and urban population show that dasymetrically refined areal interpolation is particularly effective when the analysis spans a longer time period (1990-2010 instead of 2000-2010) and the enumerated population is sufficiently large (e.g., counts of white vs. black). The results also demonstrate that current census-defined urban areas overestimate the spatial distribution of urban population and dasymetrically refined areal interpolation improves estimates of urban population. Refined TDW using building footprints or the ZTRAX® dataset outperforms all other methods. The implementation of areal interpolation enriched by dasymetric refinement represents a promising strategy to create more reliable multi-temporal and consistent estimates of different population subgroups and thus demographic compositions. This methodological foundation has the potential to advance micro-scale modeling of various subpopulations, particularly urban population to inform studies of urbanization and population change over time as well as future population projections.
| S-EPMC6936759 | biostudies-literature