Scaling Laws for Phonotactic Complexity in Spoken English Language Data.
Ontology highlight
ABSTRACT:
SUBMITTER: Baumann A
PROVIDER: S-EPMC8406375 | biostudies-literature | 2021 Sep
REPOSITORIES: biostudies-literature
Publications
Scaling Laws for Phonotactic Complexity in Spoken English Language Data.
Baumann Andreas A Kaźmierski Kamil K Matzinger Theresa T
Language and speech 20200801 3
Two prominent statistical laws in language and other complex systems are Zipf's law and Heaps' law. We investigate the extent to which these two laws apply to the linguistic domain of phonotactics-that is, to sequences of sounds. We analyze phonotactic sequences with different lengths within words and across word boundaries taken from a corpus of spoken English (Buckeye). We demonstrate that the expected relationship between the two scaling laws can only be attested when boundary spanning phonot ...[more]
Similar Datasets
Project description:Representations from transformer-based unidirectional language models are known to be effective at predicting brain responses to natural language. However, most studies comparing language models to brains have used GPT-2 or similarly sized language models. Here we tested whether larger open-source models such as those from the OPT and LLaMA families are better at predicting brain responses recorded using fMRI. Mirroring scaling results from other contexts, we found that brain prediction performance scales logarithmically with model size from 125M to 30B parameter models, with ~15% increased encoding performance as measured by correlation with a held-out test set across 3 subjects. Similar logarithmic behavior was observed when scaling the size of the fMRI training set. We also characterized scaling for acoustic encoding models that use HuBERT, WavLM, and Whisper, and we found comparable improvements with model size. A noise ceiling analysis of these large, high-performance encoding models showed that performance is nearing the theoretical maximum for brain areas such as the precuneus and higher auditory cortex. These results suggest that increasing scale in both models and data will yield incredibly effective models of language processing in the brain, enabling better scientific understanding as well as applications such as decoding.
| S-EPMC11258918 | biostudies-literature
Project description:Although speaking in noisy environments is a common occurrence, few studies have investigated how noise affects language production beyond the acoustic level. In seeking to differentiate between speaker- and listener-oriented modifications, this study examines the effect of noise on the complexity of language production and examines whether cognitive control predicts noise-induced modifications. Participants completed a picture description task via videoconferencing software while both the speaker (the participant) and listener (the experimenter) were exposed to multi-talker babble. Speakers produced fewer T-units, clauses, and words as well as fewer, but longer, unfilled pauses in noise. The degree of reduction in number of clauses, words, and unfilled pauses was significantly associated with weaker cognitive control. Thus, we consider these modifications to be speaker-oriented, driven by the distracting nature of noise. However, participants also produced fewer filled pauses and mazes in noise. These modifications were not significantly correlated with cognitive control, and they diverge from prior work demonstrating that speakers tend to produce more disfluencies when they alone shoulder the burden of a noisy environment. This pattern of results suggests that speakers may alter their speech to alleviate cognitive burden on themselves as well as to facilitate comprehension for their listener.
| S-EPMC10280673 | biostudies-literature
Project description:Twitter has become an important platform for geo-spatial analyses, providing high-volume spatial data on a wide variety of social processes. Understanding the relationship between population density and Twitter activity is therefore of key importance. This study reports a systematic relationship between population density and Twitter use. Number of tweets, number of users and population per unit area are related by power law functions with exponents greater than one. These relations are consistent with each other and hold across a range of spatial scales. This implies that population density can accurately predict Twitter activity, but importantly, it also implies that correct predictions are not given by a naive linear scaling analysis. The observed super-linearity has implications for any spatial analyses performed with Twitter data and is important for understanding the relationship between Twitter use and demographics. For example, the robustness of this relationship means that we can identify 'anomalous' geographic areas that deviate from the observed trend, identifying several towns with high/low usage relative to expectation; using the scaling relationship we are able to show that these anomalies are not caused by age structure, as has been previously proposed. Proper consideration of this scaling relationship will improve robustness in future geo-spatial studies using Twitter.
| S-EPMC6655604 | biostudies-literature
Project description:Phonotactic frequency effects play a crucial role in a number of debates over language processing and representation. It is unclear however, whether these effects reflect prelexical sensitivity to phonotactic frequency, or lexical "gang effects" in speech perception. In this paper, we use Granger causality analysis of MR-constrained MEG/EEG data to understand how phonotactic frequency influences neural processing dynamics during auditory lexical decision. Effective connectivity analysis showed weaker feedforward influence from brain regions involved in acoustic-phonetic processing (superior temporal gyrus) to lexical areas (supramarginal gyrus) for high phonotactic frequency words, but stronger top-down lexical influence for the same items. Low entropy nonwords (nonwords judged to closely resemble real words) showed a similar pattern of interactions between brain regions involved in lexical and acoustic-phonetic processing. These results contradict the predictions of a feedforward model of phonotactic frequency facilitation, but support the predictions of a lexically mediated account.
| S-EPMC4394209 | biostudies-literature
Project description:Code-blends (simultaneous words and signs) are a unique characteristic of bimodal bilingual communication. Using fMRI, we investigated code-blend comprehension in hearing native ASL-English bilinguals who made a semantic decision (edible?) about signs, audiovisual words, and semantically equivalent code-blends. English and ASL recruited a similar fronto-temporal network with expected modality differences: stronger activation for English in auditory regions of bilateral superior temporal cortex, and stronger activation for ASL in bilateral occipitotemporal visual regions and left parietal cortex. Code-blend comprehension elicited activity in a combination of these regions, and no cognitive control regions were additionally recruited. Furthermore, code-blends elicited reduced activation relative to ASL presented alone in bilateral prefrontal and visual extrastriate cortices, and relative to English alone in auditory association cortex. Consistent with behavioral facilitation observed during semantic decisions, the findings suggest that redundant semantic content induces more efficient neural processing in language and sensory regions during bimodal language integration.
| S-EPMC5769874 | biostudies-literature
Project description:During mitosis in higher eukaryotes, each chromosome condenses into a pair of rod-shaped chromatids. This process is co-regulated by the activity of several gene families, and the underlying biophysics remains poorly understood. To better understand the factors regulating chromosome condensation, we compiled a database of mitotic chromosome size and DNA content from the tables and figures of >200 published papers. A comparison across vertebrate species shows that chromosome width, length and volume scale with DNA content to the powers ∼1/4, ∼1/2, and ∼1, respectively. Angiosperms (flowering plants) show a similar length scaling, so this result is not specific to vertebrates. Chromosome shape and size thus satisfy two conditions: (1) DNA content per unit volume is approximately constant and (2) the cross-sectional area increases proportionately with chromosome length. Since viscous drag forces during chromosome movement are expected to scale with length, we hypothesize that the cross-section increase is necessary to limit the occurrence of large chromosome elongations that could slow or stall mitosis. Lastly, we note that individual vertebrate karyotypes typically exhibit a wider range of chromosome lengths as compared with angiosperms.
| S-EPMC8262490 | biostudies-literature
Project description:The success of information technology has clearly demonstrated that miniaturization often leads to unprecedented performance, and unanticipated applications. This hypothesis of "smaller-is-better" has motivated optical engineers to build various nanophotonic devices, although an understanding leading to fundamental scaling behavior for this new class of devices is missing. Here we analyze scaling laws for optoelectronic devices operating at micro and nanometer length-scale. We show that optoelectronic device performance scales non-monotonically with device length due to the various device tradeoffs, and analyze how both optical and electrical constrains influence device power consumption and operating speed. Specifically, we investigate the direct influence of scaling on the performance of four classes of photonic devices, namely laser sources, electro-optic modulators, photodetectors, and all-optical switches based on three types of optical resonators; microring, Fabry-Perot cavity, and plasmonic metal nanoparticle. Results show that while microrings and Fabry-Perot cavities can outperform plasmonic cavities at larger length-scales, they stop working when the device length drops below 100 nanometers, due to insufficient functionality such as feedback (laser), index-modulation (modulator), absorption (detector) or field density (optical switch). Our results provide a detailed understanding of the limits of nanophotonics, towards establishing an opto-electronics roadmap, akin to the International Technology Roadmap for Semiconductors.
| S-EPMC5116679 | biostudies-literature
Project description:Scaling laws relating body mass to species characteristics are among the most universal quantitative patterns in biology. Within major taxonomic groups, the 4 key ecological variables of metabolism, abundance, growth, and mortality are often well described by power laws with exponents near 3/4 or related to that value, a commonality often attributed to biophysical constraints on metabolism. However, metabolic scaling theories remain widely debated, and the links among the 4 variables have never been formally tested across the full domain of eukaryote life, to which prevailing theory applies. Here we present datasets of unprecedented scope to examine these 4 scaling laws across all eukaryotes and link them to test whether their combinations support theoretical expectations. We find that metabolism and abundance scale with body size in a remarkably reciprocal fashion, with exponents near ±3/4 within groups, as expected from metabolic theory, but with exponents near ±1 across all groups. This reciprocal scaling supports "energetic equivalence" across eukaryotes, which hypothesizes that the partitioning of energy in space across species does not vary significantly with body size. In contrast, growth and mortality rates scale similarly both within and across groups, with exponents of ±1/4. These findings are inconsistent with a metabolic basis for growth and mortality scaling across eukaryotes. We propose that rather than limiting growth, metabolism adjusts to the needs of growth within major groups, and that growth dynamics may offer a viable theoretical basis to biological scaling.
| S-EPMC6815163 | biostudies-literature
Project description:Proteins containing prion-like low complexity domains (PLDs) are common drivers of the formation of biomolecular condensates and are prone to misregulation due to amino acid mutations. Here, we exploit the accuracy of our residue-resolution coarse-grained model, Mpipi, to quantify the impact of amino acid mutations on the stability of 140 PLD mutants from six proteins (hnRNPA1, TDP43, FUS, EWSR1, RBM14, and TIA1). Our simulations reveal the existence of scaling laws that quantify the range of change in the critical solution temperature of PLDs as a function of the number and type of amino acid sequence mutations. These rules are consistent with the physicochemical properties of the mutations and extend across the entire family tested, suggesting that scaling laws can be used as tools to predict changes in the stability of PLD condensates. Our work offers a quantitative lens into how the emergent behavior of PLD solutions vary in response to physicochemical changes of single PLD molecules.
| S-EPMC11820118 | biostudies-literature
Project description:Even though people in our contemporary technological society are depending on communication, our understanding of the underlying laws of human communicational behavior continues to be poorly understood. Here we investigate the communication patterns in 2 social Internet communities in search of statistical laws in human interaction activity. This research reveals that human communication networks dynamically follow scaling laws that may also explain the observed trends in economic growth. Specifically, we identify a generalized version of Gibrat's law of social activity expressed as a scaling law between the fluctuations in the number of messages sent by members and their level of activity. Gibrat's law has been essential in understanding economic growth patterns, yet without an underlying general principle for its origin. We attribute this scaling law to long-term correlation patterns in human activity, which surprisingly span from days to the entire period of the available data of more than 1 year. Further, we provide a mathematical framework that relates the generalized version of Gibrat's law to the long-term correlated dynamics, which suggests that the same underlying mechanism could be the source of Gibrat's law in economics, ranging from large firms, research and development expenditures, gross domestic product of countries, to city population growth. These findings are also of importance for designing communication networks and for the understanding of the dynamics of social systems in which communication plays a role, such as economic markets and political systems.
| S-EPMC2722366 | biostudies-literature