Complexity-entropy analysis at different levels of organisation in written language.
Ontology highlight
ABSTRACT:
SUBMITTER: Estevez-Rams E
PROVIDER: S-EPMC6505741 | biostudies-literature | 2019
REPOSITORIES: biostudies-literature
Publications
Complexity-entropy analysis at different levels of organisation in written language.
Estevez-Rams Ernesto E Mesa-Rodriguez Ania A Estevez-Moya Daniel D
PloS one 20190508 5
Written language is complex. A written text can be considered an attempt to convey a meaningful message which ends up being constrained by language rules, context dependence and highly redundant in its use of resources. Despite all these constraints, unpredictability is an essential element of natural language. Here we present the use of entropic measures to assert the balance between predictability and surprise in written text. In short, it is possible to measure innovation and context preserva ...[more]
Similar Datasets
Project description:The Prosodic Parallelism hypothesis claims adjacent prosodic categories to prefer identical branching of internal adjacent constituents. According to Wiese and Speyer (2015), this preference implies feet contained in the same phonological phrase to display either binary or unary branching, but not different types of branching. The seemingly free schwa-zero alternations at the end of some words in German make it possible to test this hypothesis. The hypothesis was successfully tested by conducting a corpus study which used large-scale bodies of written German. As some open questions remain, and as it is unclear whether Prosodic Parallelism is valid for the spoken modality as well, the present study extends this inquiry to spoken German. As in the previous study, the results of a corpus analysis recruiting a variety of linguistic constructions are presented. The Prosodic Parallelism hypothesis can be demonstrated to be valid for spoken German as well as for written German. The paper thus contributes to the question whether prosodic preferences are similar between the spoken and written modes of a language. Some consequences of the results for the production of language are discussed.
| S-EPMC5069292 | biostudies-literature
Project description:Despite the important role of written language in everyday life, abnormalities in functional written communication have been sparsely investigated in primary progressive aphasia. Prior studies have analysed written language separately in each of the three variants of primary progressive aphasia-but have rarely compared them to each other or to spoken language. Manual analysis of written language can be a time-consuming process. We therefore developed a program that quantifies content units and total units in written or transcribed language samples. We analysed written and spoken descriptions of the Western Aphasia Battery picnic scene, based on a predefined content unit corpus. We calculated the ratio of content units to units as a measure of content density. Our cohort included 115 participants (20 controls for written, 20 controls for spoken, 28 participants with nonfluent variant primary progressive aphasia, 30 for logopenic variant and 17 for semantic variant). Our program identified content units with a validity of 99.7% (95%CI 99.5-99.8). All patients wrote fewer units than controls (P < 0.001). Patients with the logopenic variant (P = 0.013) and the semantic variant (0.004) wrote fewer content units than controls. The content unit-to-unit ratio was higher in the nonfluent and semantic variants than controls (P = 0.019), but no difference in the logopenic variant (P = 0.962). Participants with the logopenic (P < 0.001) and semantic (P = 0.04) variants produced fewer content units in written compared to spoken descriptions. All variants produced fewer units in written samples compared to spoken (P < 0.001). However, due to a relatively smaller decrease in written content units, we observed a larger content unit-to-unit ratio in writing over speech (P < 0.001). Written and spoken content units (r = 0.5, P = 0.009) and total units (r = 0.64, P < 0.001) were significantly correlated in patients with nonfluent variant, but this was not the case for logopenic or semantic. Considering all patients with primary progressive aphasia, fewer content units were produced in those with greater aphasia severity (Progressive Aphasia Severity Scale Sum of Boxes, r = -0.24, P = 0.04) and dementia severity (Clinical Dementia Rating scale Sum of Boxes, r = -0.34, P = 0.004). In conclusion, we observed reduced written content in patients with primary progressive aphasia compared to controls, with a preference for content over non-content units in patients with the nonfluent and semantic variants. We observed a similar 'telegraphic' style in both language modalities in patients with the nonfluent variant. Lastly, we show how our program provides a time-efficient tool, which could enable feedback and tracking of writing as an important feature of language and cognition.
| S-EPMC10396070 | biostudies-literature
Project description:BackgroundThe current method to evaluate the autonomic balance after renal denervation (RDN) relies on heart rate variability (HRV). However, parameters of HRV were not always predictive of response to RDN. Therefore, the complexity and disorder of heart rhythm, measured by entropy of entropy (EoE) and average entropy (AE), have been used to analyze autonomic dysfunction. This study evaluated the dynamic changes in autonomic status after RDN via EoE and AE analysis.MethodsFive patients were prospectively enrolled in the Global SYMPLICITY Registry from 2020 to 2021. 24-h Holter and ambulatory blood pressure monitoring (ABPM) was performed at baseline and 3 months after RDN procedures. The autonomic status was analyzed using the entropy-based AE and EoE analysis and the conventional HRV-based low frequency (LF), high frequency (HF), and LF/HF.ResultsAfter RDN, the ABPM of all patients showed a significant reduction in blood pressure (BP) and heart rate. Only AE and HF values of all patients had consistent changes after RDN (p < 0.05). The spearman rank-order correlation coefficient of AE vs. HF was 0.86, but AE had a lower coefficient of variation than HF.ConclusionsMonitoring the AE and EoE analysis could be an alternative to interpreting autonomic status. In addition, a relative change of autonomic tone, especially an increasing parasympathetic activity, could restore autonomic balance after RDN.
| S-EPMC9131559 | biostudies-literature
Project description:The diversity and complexity of the microbiome's genomic landscape are not always mirrored in its proteomic profile. Despite the anticipated proteomic diversity, observed complexities of microbiome sample are often lower than expected. Two main factors contribute to this discrepancy: limitations in mass spectrometry's detection sensitivity and bioinformatics challenges in metaproteomics identification. This study introduces a novel approach to evaluating sample complexity directly at the full mass spectrum (MS1) level rather than relying on peptide identifications. When analyzing under identical mass spectrometry conditions, microbiome samples displayed significantly higher complexity, as evidenced by the spectral entropy and peptide candidate entropy, compared to single-species samples. The research provides solid evidence for the complexity of microbiome in proteomics indicating the optimization potential of the bioinformatics workflow.
2024-05-27 | PXD050781 | Pride
Project description:One of the fundamental questions about human language is whether all languages are equally complex. Here, we approach this question from an information-theoretic perspective. We present a large scale quantitative cross-linguistic analysis of written language by training a language model on more than 6500 different documents as represented in 41 multilingual text collections consisting of ~ 3.5 billion words or ~ 9.0 billion characters and covering 2069 different languages that are spoken as a native language by more than 90% of the world population. We statistically infer the entropy of each language model as an index of what we call average prediction complexity. We compare complexity rankings across corpora and show that a language that tends to be more complex than another language in one corpus also tends to be more complex in another corpus. In addition, we show that speaker population size predicts entropy. We argue that both results constitute evidence against the equi-complexity hypothesis from an information-theoretic perspective.
| S-EPMC10505229 | biostudies-literature
Project description:When we read printed text, we are continuously predicting upcoming words to integrate information and guide future eye movements. Thus, the Predictability of a given word has become one of the most important variables when explaining human behaviour and information processing during reading. In parallel, the Natural Language Processing (NLP) field evolved by developing a wide variety of applications. Here, we show that using different word embeddings techniques (like Latent Semantic Analysis, Word2Vec, and FastText) and N-gram-based language models we were able to estimate how humans predict words (cloze-task Predictability) and how to better understand eye movements in long Spanish texts. Both types of models partially captured aspects of predictability. On the one hand, our N-gram model performed well when added as a replacement for the cloze-task Predictability of the fixated word. On the other hand, word embeddings were useful to mimic Predictability of the following word. Our study joins efforts from neurolinguistic and NLP fields to understand human information processing during reading to potentially improve NLP algorithms.
| S-EPMC7064512 | biostudies-literature
Project description:The Complexity-entropy causality plane (CECP) is a parsimonious representation space for time series. It has only two dimensions: normalized permutation entropy ([Formula: see text]) and Jensen-Shannon complexity ([Formula: see text]) of a time series. This two-dimensional representation allows for detection of slow or rapid drifts in the condition of mechanical components monitored through sensor measurements. The CECP representation can be used for both predictive analytics and visual monitoring of changes in component condition. This method requires minimal pre-processing of raw signals. Furthermore, it is insensitive to noise, stationarity, and trends. These desirable properties make CECP a good candidate for machine condition monitoring and fault diagnostics. In this work we study the effectiveness of CECP on three rotary component condition assessment applications. We use CECP representation of vibration signals to differentiate various machine component health conditions for rotary machine components, namely roller bearing and gears. The results confirm that the CECP representation is able to detect, with high accuracy, changes in underlying dynamics of machine component degradation states. From class separability perspective, the CECP representation is able to generate linearly separable classes for the classification of different fault states. This classification performance improves with increasing signal length. For signal length of 16,384 data points, the fault classification accuracy varies from 90% to 100% for bearing applications, and from 85% to 100% for gear applications. We observed that the optimum parameter for CECP representatino depends on the application. For bearing applications we found that embedding dimension D = 4, 5, 6, and embedding delay τ = 1, 2, 3 are suitable for good fault classification. For gear applications we find that embedding dimension D = 4, 5, and embedding delay τ = 1, 5 are suitable for fault classification.
| S-EPMC6615599 | biostudies-literature
Project description:Plasma membranes (PMs) contain hundreds of different lipid species that contribute differently to overall bilayer properties. By modulation of these properties, membrane protein function can be affected. Furthermore, inhomogeneous lipid mixing and domains of lipid enrichment/depletion can sort proteins and provide optimal local environments. Recent coarse-grained (CG) Martini molecular dynamics efforts have provided glimpses into lipid organization of different PMs: an "Average" and a "Brain" PM. Their high complexity and large size require long simulations (∼80 μs) for proper sampling. Thus, these simulations are computationally taxing. This level of complexity is beyond the possibilities of all-atom simulations, raising the question-what complexity is needed for "realistic" bilayer properties? We constructed CG Martini PM models of varying complexity (63 down to 8 different lipids). Lipid tail saturations and headgroup combinations were kept as consistent as possible for the "tissues'" (Average/Brain) at three levels of compositional complexity. For each system, we analyzed membrane properties to evaluate which features can be retained at lower complexity and validate eight-component bilayers that can act as reliable mimetics for Average or Brain PMs. Systems of reduced complexity deliver a more robust and malleable tool for computational membrane studies and allow for equivalent all-atom simulations and experiments.
| S-EPMC7553384 | biostudies-literature
Project description:Using both quantitative and qualitative approaches, this study investigated the preference of learners of English as a foreign language (EFL) for four types of written corrective feedback (WCF), which are often discussed in the literature, on grammatical, lexical, orthographic, and pragmatic errors. In particular, it concerned whether such preference is influenced by two learner variables, namely, foreign language enjoyment (FLE) and proficiency level. The preference for selective vs. comprehensive WCF was also examined. The participants in the study were 117 University students in a Thai EFL context. Analysis of questionnaire data revealed a tendency for learners to prefer more explicit types of WCF (i.e., metalinguistic explanation and overt correction) for most error types, irrespective of their proficiency and FLE level. High proficiency level learners rated less explicit WCF types (i.e., underlining and error code) as useful to some degree, whereas their low proficiency level counterparts did not. Similar results were found for the two FLE groups. Besides, the FLE level seemed to play a role in perceiving the value of WCF in terms of scope. The results of follow-up interviews showed that the linguistic features of learners' first language, existing knowledge of the target language, affective feelings, and teacher's role were the main factors contributing to variation in learners' preferences. Possible pedagogical implications are discussed.
| S-EPMC8060554 | biostudies-literature
Project description:Estimating the depth of anaesthesia (DoA) in operations has always been a challenging issue due to the underlying complexity of the brain mechanisms. Electroencephalogram (EEG) signals are undoubtedly the most widely used signals for measuring DoA. In this paper, a novel EEG-based index is proposed to evaluate DoA for 24 patients receiving general anaesthesia with different levels of unconsciousness. Sample Entropy (SampEn) algorithm was utilised in order to acquire the chaotic features of the signals. After calculating the SampEn from the EEG signals, Random Forest was utilised for developing learning regression models with Bispectral index (BIS) as the target. Correlation coefficient, mean absolute error, and area under the curve (AUC) were used to verify the perioperative performance of the proposed method. Validation comparisons with typical nonstationary signal analysis methods (i.e., recurrence analysis and permutation entropy) and regression methods (i.e., neural network and support vector machine) were conducted. To further verify the accuracy and validity of the proposed methodology, the data is divided into four unconsciousness-level groups on the basis of BIS levels. Subsequently, analysis of variance (ANOVA) was applied to the corresponding index (i.e., regression output). Results indicate that the correlation coefficient improved to 0.72 ± 0.09 after filtering and to 0.90 ± 0.05 after regression from the initial values of 0.51 ± 0.17. Similarly, the final mean absolute error dramatically declined to 5.22 ± 2.12. In addition, the ultimate AUC increased to 0.98 ± 0.02, and the ANOVA analysis indicates that each of the four groups of different anaesthetic levels demonstrated significant difference from the nearest levels. Furthermore, the Random Forest output was extensively linear in relation to BIS, thus with better DoA prediction accuracy. In conclusion, the proposed method provides a concrete basis for monitoring patients' anaesthetic level during surgeries.
| S-EPMC5970554 | biostudies-literature