Dynamic Data Selection for Curriculum Learning via Ability Estimation.
Ontology highlight
ABSTRACT:
SUBMITTER: Lalor JP
PROVIDER: S-EPMC7771727 | biostudies-literature | 2020 Nov
REPOSITORIES: biostudies-literature
Publications
Dynamic Data Selection for Curriculum Learning via Ability Estimation.
Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing 20201101
Curriculum learning methods typically rely on heuristics to estimate the difficulty of training examples or the ability of the model. In this work, we propose replacing difficulty heuristics with learned difficulty parameters. We also propose Dynamic Data selection for Curriculum Learning via Ability Estimation (DDaCLAE), a strategy that probes model ability at each training epoch to select the best training examples at that point. We show that models using learned difficulty and/or ability outp ...[more]
Similar Datasets
Project description:Commonly used methods for estimating parameters of a spatial dynamic panel data model include the two-stage least squares, quasi-maximum likelihood, and generalized moments. In this paper, we present an approach that uses the eigenvalues and eigenvectors of a spatial weight matrix to directly construct consistent least-squares estimators of parameters of a general spatial dynamic panel data model. The proposed methodology is conceptually simple and efficient and can be easily implemented. We show that the proposed parameter estimators are consistent and asymptotically normally distributed under mild conditions. We demonstrate the superior performance of our approach via extensive simulation studies. We also provide a real data example.
| S-EPMC7071865 | biostudies-literature
Project description:Dealing with a system of first-order reactions is a recurrent issue in chemometrics, especially in the analysis of data obtained by spectroscopic methods applied on complex biological systems. We argue that global multiexponential fitting, the still common way to solve such problems, has serious weaknesses compared to contemporary methods of sparse modeling. Combining the advantages of group lasso and elastic net-the statistical methods proven to be very powerful in other areas-we created an optimization problem tunable from very sparse to very dense distribution over a large pre-defined grid of time constants, fitting both simulated and experimental multiwavelength spectroscopic data with high computational efficiency. We found that the optimal values of the tuning hyperparameters can be selected by a machine-learning algorithm based on a Bayesian optimization procedure, utilizing widely used or novel versions of cross-validation. The derived algorithm accurately recovered the true sparse kinetic parameters of an extremely complex simulated model of the bacteriorhodopsin photocycle, as well as the wide peak of hypothetical distributed kinetics in the presence of different noise levels. It also performed well in the analysis of the ultrafast experimental fluorescence kinetics data detected on the coenzyme FAD in a very wide logarithmic time window. We conclude that the primary application of the presented algorithms-implemented in available software-covers a wide area of studies on light-induced physical, chemical, and biological processes carried out with different spectroscopic methods. The demand for this kind of analysis is expected to soar due to the emerging ultrafast multidimensional infrared and electronic spectroscopic techniques that provide very large and complex datasets. In addition, simulations based on our methods could help in designing the technical parameters of future experiments for the verification of particular hypothetical models.
| S-EPMC8352076 | biostudies-literature
Project description:Despite the wide application of dynamic models in infectious disease epidemiology, the particular modeling of variability in the different model components is often subjective rather than the result of a thorough model selection process. This is in part because inference for a stochastic transmission model can be difficult since the likelihood is often intractable due to partial observability. In this work, we address the question of adequate inclusion of variability by demonstrating a systematic approach for model selection and parameter inference for dynamic epidemic models. For this, we perform inference for six partially observed Markov process models, which assume the same underlying transmission dynamics, but differ with respect to the amount of variability they allow for. The inference framework for the stochastic transmission models is provided by iterated filtering methods, which are readily implemented in the R package pomp by King and others (2016, Statistical inference for partially observed Markov processes via the R package pomp. Journal of Statistical Software69, 1-43). We illustrate our approach on German rotavirus surveillance data from 2001 to 2008, discuss practical difficulties of the methods used and calculate a model based estimate for the basic reproduction number $R_0$ using these data.
| S-EPMC7307980 | biostudies-literature
Project description:Dynamic Causal Modeling (DCM) can be used to quantify cognitive function in individuals as effective connectivity. However, ambiguity among subjects in the number and location of discernible active regions prevents all candidate models from being compared in all subjects, precluding the use of DCM as an individual cognitive phenotyping tool. This paper proposes a solution to this problem by treating missing regions in the first-level analysis as missing data, and performing estimation of the time course associated with any missing region using one of four candidate methods: zero-filling, average-filling, noise-filling using a fixed stochastic process, or one estimated using expectation-maximization. The effect of this estimation scheme was analyzed by treating it as a preprocessing step to DCM and observing the resulting effects on model evidence. Simulation studies show that estimation using expectation-maximization yields the highest classification accuracy using a simple loss function and highest model evidence, relative to other methods. This result held for various dataset sizes and varying numbers of model choice. In real data, application to Go/No-Go and Simon tasks allowed computation of signals from the missing nodes and the consequent computation of model evidence in all subjects compared to 62 and 48 percent respectively if no preprocessing was performed. These results demonstrate the face validity of the preprocessing scheme and open the possibility of using single-subject DCM as an individual cognitive phenotyping tool.
| S-EPMC4082189 | biostudies-literature
Project description:An accurate step length estimation can provide valuable information to different applications such as indoor positioning systems or it can be helpful when analyzing the gait of a user, which can then be used to detect various gait impairments that lead to a reduced step length (caused by e.g., Parkinson's disease or multiple sclerosis). In this paper, we focus on the estimation of the step length using machine learning techniques that could be used in an indoor positioning system. Previous step length algorithms tried to model the length of a step based on measurements from the accelerometer and some tuneable (user-specific) parameters. Machine-learning-based step length estimation algorithms eliminate these parameters to be tuned. Instead, to adapt these algorithms to different users, it suffices to provide examples of the length of multiple steps for different persons to the machine learning algorithm, so that in the training phase the algorithm can learn to predict the step length for different users. Until now, these machine learning algorithms were trained with features that were chosen intuitively. In this paper, we consider a systematic feature selection algorithm to be able to determine the features from a large collection of features, resulting in the best performance. This resulted in a step length estimator with a mean absolute error of 3.48 cm for a known test person and 4.19 cm for an unknown test person, while current state-of-the-art machine-learning-based step length estimators resulted in a mean absolute error of 4.94 cm and 6.27 cm for respectively a known and unknown test person.
| S-EPMC7038475 | biostudies-literature
Project description:IntroductionPhysical examination skills are receiving less attention in curricula and clinical practice, being supplanted by imaging and other technologies. We developed an online module to introduce auscultation of common cardiac murmurs to second-year medical students.MethodsThe Murmur Online Learning Experience (MOLE) curriculum focused on nine common, unique, or highly testable cardiac murmurs, chosen collaboratively by the authors. The curriculum consisted of (1) a nine-item multiple-choice pretest containing a clinical vignette, a photo of stethoscope location, and an auditory clip; (2) nine modules each containing a several-minute-long auditory clip and a written description (location, quality, radiation, change with exam maneuvers); and (3) a nine-item multiple-choice posttest, identical to the pretest but randomly ordered. All second-year medical students at the University of Louisville were given access to MOLE during their cardiovascular curriculum and given an incentive to complete the ungraded activity.ResultsOne hundred forty-seven (91.8%) students voluntarily completed the pretest and posttest. The mean pretest score was 3.76 out of 9 (SD = 1.77). The mean posttest score was 7.14 out of 9 (SD = 1.78). Paired t-test results demonstrated a p value of <.001.DiscussionAn online murmur curriculum consisting of repetitive auditory murmurs and narrative description of murmurs improved second-year medical students’ ability to correctly identify common cardiac murmurs. This method of learning murmurs via online curriculum is a practical and effective way to hone students’ physical exam skills in the modern era.
| S-EPMC7331965 | biostudies-literature
Project description:BackgroundModeling high-dimensional data involving thousands of variables is particularly important for gene expression profiling experiments, nevertheless,it remains a challenging task. One of the challenges is to implement an effective method for selecting a small set of relevant genes, buried in high-dimensional irrelevant noises. RELIEF is a popular and widely used approach for feature selection owing to its low computational cost and high accuracy. However, RELIEF based methods suffer from instability, especially in the presence of noisy and/or high-dimensional outliers.ResultsWe propose an innovative feature weighting algorithm, called LHR, to select informative genes from highly noisy data. LHR is based on RELIEF for feature weighting using classical margin maximization. The key idea of LHR is to estimate the feature weights through local approximation rather than global measurement, which is typically used in existing methods. The weights obtained by our method are very robust in terms of degradation of noisy features, even those with vast dimensions. To demonstrate the performance of our method, extensive experiments involving classification tests have been carried out on both synthetic and real microarray benchmark datasets by combining the proposed technique with standard classifiers, including the support vector machine (SVM), k-nearest neighbor (KNN), hyperplane k-nearest neighbor (HKNN), linear discriminant analysis (LDA) and naive Bayes (NB).ConclusionExperiments on both synthetic and real-world datasets demonstrate the superior performance of the proposed feature selection method combined with supervised learning in three aspects: 1) high classification accuracy, 2) excellent robustness to noise and 3) good stability using to various classification algorithms.
| S-EPMC4007530 | biostudies-literature
Project description:BackgroundAdvances in modern high-throughput techniques of molecular biology have enabled top-down approaches for the estimation of parameter values in metabolic systems, based on time series data. Special among them is the recent method of dynamic flux estimation (DFE), which uses such data not only for parameter estimation but also for the identification of functional forms of the processes governing a metabolic system. DFE furthermore provides diagnostic tools for the evaluation of model validity and of the quality of a model fit beyond residual errors. Unfortunately, DFE works only when the data are more or less complete and the system contains as many independent fluxes as metabolites. These drawbacks may be ameliorated with other types of estimation and information. However, such supplementations incur their own limitations. In particular, assumptions must be made regarding the functional forms of some processes and detailed kinetic information must be available, in addition to the time series data.ResultsThe authors propose here a systematic approach that supplements DFE and overcomes some of its shortcomings. Like DFE, the approach is model-free and requires only minimal assumptions. If sufficient time series data are available, the approach allows the determination of a subset of fluxes that enables the subsequent applicability of DFE to the rest of the flux system. The authors demonstrate the procedure with three artificial pathway systems exhibiting distinct characteristics and with actual data of the trehalose pathway in Saccharomyces cerevisiae.ConclusionsThe results demonstrate that the proposed method successfully complements DFE under various situations and without a priori assumptions regarding the model representation. The proposed method also permits an examination of whether at all, to what degree, or within what range the available time series data can be validly represented in a particular functional format of a flux within a pathway system. Based on these results, further experiments may be designed to generate data points that genuinely add new information to the structure identification and parameter estimation tasks at hand.
| S-EPMC3495652 | biostudies-literature
Project description:Sensory processing in the brain includes three key operations: multisensory integration-the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations-the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned-but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.
| S-EPMC3630212 | biostudies-literature
Project description:MotivationUnknown parameters of dynamical models are commonly estimated from experimental data. However, while various efficient optimization and uncertainty analysis methods have been proposed for quantitative data, methods for qualitative data are rare and suffer from bad scaling and convergence.ResultsHere, we propose an efficient and reliable framework for estimating the parameters of ordinary differential equation models from qualitative data. In this framework, we derive a semi-analytical algorithm for gradient calculation of the optimal scaling method developed for qualitative data. This enables the use of efficient gradient-based optimization algorithms. We demonstrate that the use of gradient information improves performance of optimization and uncertainty quantification on several application examples. On average, we achieve a speedup of more than one order of magnitude compared to gradient-free optimization. Additionally, in some examples, the gradient-based approach yields substantially improved objective function values and quality of the fits. Accordingly, the proposed framework substantially improves the parameterization of models from qualitative data.AvailabilityThe proposed approach is implemented in the open-source Python Parameter EStimation TOolbox (pyPESTO). pyPESTO is available at https://github.com/ICB-DCM/pyPESTO. All application examples and code to reproduce this study are available at https://doi.org/10.5281/zenodo.4507613.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC8652033 | biostudies-literature