Beyond Benford's Law: Distinguishing Noise from Chaos.
Ontology highlight
ABSTRACT:
SUBMITTER: Li Q
PROVIDER: S-EPMC4452586 | biostudies-literature | 2015
REPOSITORIES: biostudies-literature
Publications
Beyond Benford's Law: Distinguishing Noise from Chaos.
Li Qinglei Q Fu Zuntao Z Yuan Naiming N
PloS one 20150601 6
Determinism and randomness are two inherent aspects of all physical processes. Time series from chaotic systems share several features identical with those generated from stochastic processes, which makes them almost undistinguishable. In this paper, a new method based on Benford's law is designed in order to distinguish noise from chaos by only information from the first digit of considered series. By applying this method to discrete data, we confirm that chaotic data indeed can be distinguishe ...[more]
Similar Datasets
Project description:A recently proposed methodology called the Horizontal Visibility Graph (HVG) [Luque et al., Phys. Rev. E., 80, 046103 (2009)] that constitutes a geometrical simplification of the well known Visibility Graph algorithm [Lacasa et al., Proc. Natl. Sci. U.S.A. 105, 4972 (2008)], has been used to study the distinction between deterministic and stochastic components in time series [L. Lacasa and R. Toral, Phys. Rev. E., 82, 036120 (2010)]. Specifically, the authors propose that the node degree distribution of these processes follows an exponential functional of the form [Formula: see text], in which [Formula: see text] is the node degree and [Formula: see text] is a positive parameter able to distinguish between deterministic (chaotic) and stochastic (uncorrelated and correlated) dynamics. In this work, we investigate the characteristics of the node degree distributions constructed by using HVG, for time series corresponding to [Formula: see text] chaotic maps, 2 chaotic flows and [Formula: see text] different stochastic processes. We thoroughly study the methodology proposed by Lacasa and Toral finding several cases for which their hypothesis is not valid. We propose a methodology that uses the HVG together with Information Theory quantifiers. An extensive and careful analysis of the node degree distributions obtained by applying HVG allow us to conclude that the Fisher-Shannon information plane is a remarkable tool able to graphically represent the different nature, deterministic or stochastic, of the systems under study.
| S-EPMC4172653 | biostudies-literature
Project description:Benford's Law describes the finding that the distribution of leading (or leftmost) digits of innumerable datasets follows a well-defined logarithmic trend, rather than an intuitive uniformity. In practice this means that the most common leading digit is 1, with an expected frequency of 30.1%, and the least common is 9, with an expected frequency of 4.6%. Currently, the most common application of Benford's Law is in detecting number invention and tampering such as found in accounting-, tax-, and voter-fraud. We demonstrate that answers to end-of-chapter exercises in physics and chemistry textbooks conform to Benford's Law. Subsequently, we investigate whether this fact can be used to gain advantage over random guessing in multiple-choice tests, and find that while testbank answers in introductory physics closely conform to Benford's Law, the testbank is nonetheless secure against such a Benford's attack for banal reasons.
| S-EPMC4331362 | biostudies-literature
Project description:Integrity and trust in that integrity are fundamental to academic research. However, procedures for monitoring the trustworthiness of research, and for investigating cases where concern about possible data fraud have been raised are not well established. Here we suggest a practical approach for the investigation of work suspected of fraudulent data manipulation using Benford's Law. This should be of value to both individual peer-reviewers and academic institutions and journals. In this, we draw inspiration from well-established practices of financial auditing. We provide synthesis of the literature on tests of adherence to Benford's Law, culminating in advice of a single initial test for digits in each position of numerical strings within a dataset. We also recommend further tests which may prove useful in the event that specific hypotheses regarding the nature of data manipulation can be justified. Importantly, our advice differs from the most common current implementations of tests of Benford's Law. Furthermore, we apply the approach to previously-published data, highlighting the efficacy of these tests in detecting known irregularities. Finally, we discuss the results of these tests, with reference to their strengths and limitations.
| S-EPMC10088595 | biostudies-literature
Project description:Working with a large temporal dataset spanning several decades often represents a challenging task, especially when the record is heterogeneous and incomplete. The use of statistical laws could potentially overcome these problems. Here we apply Benford's Law (also called the "First-Digit Law") to the traveled distances of tropical cyclones since 1842. The record of tropical cyclones has been extensively impacted by improvements in detection capabilities over the past decades. We have found that, while the first-digit distribution for the entire record follows Benford's Law prediction, specific changes such as satellite detection have had serious impacts on the dataset. The least-square misfit measure is used as a proxy to observe temporal variations, allowing us to assess data quality and homogeneity over the entire record, and at the same time over specific periods. Such information is crucial when running climatic models and Benford's Law could potentially be used to overcome and correct for data heterogeneity and/or to select the most appropriate part of the record for detailed studies.
| S-EPMC4496784 | biostudies-other
Project description:Typical responses of cortical neurons to identical sensory stimuli appear highly variable. It has thus been proposed that the cortex primarily uses a rate code. However, other studies have argued for spike-time coding under certain conditions. The potential role of spike-time coding is directly limited by the internally generated variability of cortical circuits, which remains largely unexplored. Here, we quantify this internally generated variability using a biophysical model of rat neocortical microcircuitry with biologically realistic noise sources. We find that stochastic neurotransmitter release is a critical component of internally generated variability, causing rapidly diverging, chaotic recurrent network dynamics. Surprisingly, the same nonlinear recurrent network dynamics can transiently overcome the chaos in response to weak feed-forward thalamocortical inputs, and support reliable spike times with millisecond precision. Our model shows that the noisy and chaotic network dynamics of recurrent cortical microcircuitry are compatible with stimulus-evoked, millisecond spike-time reliability, resolving a long-standing debate.
| S-EPMC6706377 | biostudies-literature
Project description:We develop a Bayesian time-varying model that tracks periods at which conformance to Benford's Law is lower. Our methods are motivated by recent attempts to assess how the quality and homogeneity of large datasets may change over time by using the First-Digit Rule. We resort to a smooth multinomial logistic model which captures the dynamics governing the proportion of first digits, and apply the proposed model to global tropical cyclone tracks over the past two centuries. Our findings indicate that cumulative technological improvements may have only had a moderate influence on the homogeneity of the dataset, and hint that recent heterogeneity could be due to other drivers.
| S-EPMC6461246 | biostudies-literature
Project description:BackgroundBreast cancer is the most frequent cancer in women all over the world, and it is one of the leading causes of cancer-related deaths in women. A pathologist's partiality for the last digit of a patient's name can lead to errors in the measurement of malignancies. This means that, rather than recording the exact measurement of a tumor, a pathologist might round it off to his preferred terminal digit.MethodsIt is a retrospective cross-sectional study in which data on primary tumor resection for 1000 breast cancer patients was obtained from KRL Hospital's patient directory from November 2016 to December 2020. The tumors were measured in cm to one decimal point along their longest dimension. Ki-67 markers were used to categorize the tumors into nine categories. Terminal digit preference was evaluated using Benford's law.ResultsThe recording of the Ki-67 index revealed evidence of pentameric preference. The numbers three, five, and six appeared more frequently in the histogram of the Ki-67 index distribution measured in percentage. The frequency of nine dropped dramatically. However, the influence of tumor size terminal digits on Ki-67 staining scores (low proliferative vs high proliferative) assessed using the Mann-Whitney U Test demonstrated that tumor size terminal digits had no significant effect on Ki-67 staining scores (p = 0.114).ConclusionThe Ki-67 index shows evidence of pentameric preference for digits three, five, and six. The frequency of nine has dropped dramatically. The influence of tumor size on terminal digits on staining scores (low proliferative vs. high proliferative) was assessed using the Mann-Whitney U Test.
| S-EPMC9289319 | biostudies-literature
Project description:Large networks of sparsely coupled, excitatory and inhibitory cells occur throughout the brain. For many models of these networks, a striking feature is that their dynamics are chaotic and thus, are sensitive to small perturbations. How does this chaos manifest in the neural code? Specifically, how variable are the spike patterns that such a network produces in response to an input signal? To answer this, we derive a bound for a general measure of variability-spike-train entropy. This leads to important insights on the variability of multi-cell spike pattern distributions in large recurrent networks of spiking neurons responding to fluctuating inputs. The analysis is based on results from random dynamical systems theory and is complemented by detailed numerical simulations. We find that the spike pattern entropy is an order of magnitude lower than what would be extrapolated from single cells. This holds despite the fact that network coupling becomes vanishingly sparse as network size grows-a phenomenon that depends on "extensive chaos," as previously discovered for balanced networks without stimulus drive. Moreover, we show how spike pattern entropy is controlled by temporal features of the inputs. Our findings provide insight into how neural networks may encode stimuli in the presence of inherently chaotic dynamics.
| S-EPMC4183092 | biostudies-literature
Project description:Indicators of compliance and efficiency in combatting money laundering, collected by EUROSTAT, are plagued with shortcomings. In this paper, I have carried out a forensic analysis on a 2003-2010 dataset of indicators of compliance and efficiency in combatting money laundering, that European Union member states self-reported to EUROSTAT, and on the basis of which, their efforts were evaluated. I used Benford's law to detect any anomalous statistical patterns and found that statistical anomalies were also consistent with strategic manipulation. According to Benford's law, if we pick a random sample of numbers representing natural processes, and look at the distribution of the first digits of these numbers, we see that, contrary to popular belief, digit 1 occurs most often, then digit 2, and so on, with digit 9 occurring in less than 5% of the sample. Without prior knowledge of Benford's law, since people are not intuitively good at creating truly random numbers, deviations thereof can capture strategic alterations. In order to eliminate other sources of deviation, I have compared deviations in situations where incentives and opportunities for manipulation existed and in situations where they did not. While my results are not a conclusive proof of strategic manipulation, they signal that countries that faced incentives and opportunities to misinform the international community about their efforts to combat money laundering may have manipulated these indicators. Finally, my analysis points to the high potential for disruption that the manipulation of national statistics has, and calls for the acknowledgment that strategic manipulation can be an unintended consequence of the international community's pressure on countries to put combatting money laundering on the top of their national agenda.
| S-EPMC5266253 | biostudies-literature
Project description:In this contribution we show how the Lorentz-Lorenz and the Clausius-Mosotti equations are related to Beer's law. Accordingly, the linear concentration dependence of absorbance is a consequence of neglecting the difference between the local and the applied electric field. Additionally, it is necessary to assume that the absorption index and the related refractive index change is small. By connecting the Lorentz-Lorenz equations with dispersion theory, it becomes obvious that the oscillators are coupled via the local field. We investigate this coupling with numerical examples and show that, as a consequence, the integrated absorbance of a single band is in general no longer linearly depending on the concentration. In practice, the deviations from Beer's law usually do not set in before the density reaches about one tenth of that of condensed matter. For solutions, the Lorentz-Lorenz equations predict a strong coupling also between the oscillators of solute and solvent. In particular, in the infrared spectral region, the absorption coefficients are prognosticated to be much higher due to this coupling compared to those in the gas phase.
| S-EPMC7317954 | biostudies-literature