Lab-like findings from online experiments
Ontology highlight
ABSTRACT: Supplementary Information
The online version contains supplementary material available at 10.1007/s40881-021-00114-8.
SUBMITTER: Buso I
PROVIDER: S-EPMC8683827 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:We measure how accurately replication of experimental results can be predicted by black-box statistical models. With data from four large-scale replication projects in experimental psychology and economics, and techniques from machine learning, we train predictive models and study which variables drive predictable replication. The models predicts binary replication with a cross-validated accuracy rate of 70% (AUC of 0.77) and estimates of relative effect sizes with a Spearman ? of 0.38. The accuracy level is similar to market-aggregated beliefs of peer scientists [1, 2]. The predictive power is validated in a pre-registered out of sample test of the outcome of [3], where 71% (AUC of 0.73) of replications are predicted correctly and effect size correlations amount to ? = 0.25. Basic features such as the sample and effect sizes in original papers, and whether reported effects are single-variable main effects or two-variable interactions, are predictive of successful replication. The models presented in this paper are simple tools to produce cheap, prognostic replicability metrics. These models could be useful in institutionalizing the process of evaluation of new findings and guiding resources to those direct replications that are likely to be most informative.
| S-EPMC6894796 | biostudies-literature
Project description:Online labor markets provide new opportunities for behavioral research, but conducting economic experiments online raises important methodological challenges. This particularly holds for interactive designs. In this paper, we provide a methodological discussion of the similarities and differences between interactive experiments conducted in the laboratory and online. To this end, we conduct a repeated public goods experiment with and without punishment using samples from the laboratory and the online platform Amazon Mechanical Turk. We chose to replicate this experiment because it is long and logistically complex. It therefore provides a good case study for discussing the methodological and practical challenges of online interactive experimentation. We find that basic behavioral patterns of cooperation and punishment in the laboratory are replicable online. The most important challenge of online interactive experiments is participant dropout. We discuss measures for reducing dropout and show that, for our case study, dropouts are exogenous to the experiment. We conclude that data quality for interactive experiments via the Internet is adequate and reliable, making online interactive experimentation a potentially valuable complement to laboratory studies.
| S-EPMC5807491 | biostudies-literature
Project description:Use of lab-in-the-field experiments has steadily increased, given benefits of studying relevant populations and their preferences. In the field, researchers must often relinquish the control of a standard laboratory, raising the specter of communication from past to future participants. Little is known about the consequences of such spillovers, and recent literature indicates variation in how authors deal with them. I provide estimates of communication spillovers using existing data from public goods games in Rwanda, leveraging variation in planning the sequence of visiting 147 villages. The resulting order created opportunities for some villages to communicate with past participants. Using ex-post matching of villages with and without these opportunities I find that communication led to substantial increases in cooperation, suggesting that unanticipated spillovers can bias inference. I conclude with advice for creating protocols to deal with communication spillovers.
| S-EPMC9193284 | biostudies-literature
Project description:The COVID-19 pandemic forced many courses to move online, presenting a particular challenge for hands-on laboratory courses. One such course in our Biotechnology track is an advanced Protein Interactions lecture/laboratory course. This 8-week course typically meets for 5 h a week in the laboratory space. For the Fall 2020 version of the course, first-person videos were produced for each of the laboratory experiments, and the corresponding experimental data produced by students in previous semesters were provided for the current students to analyze in their electronic lab notebooks and lab reports. Student perspectives and assessments were collected on course participants from Fall 2019 (in-person laboratories) and Fall 2020 (online laboratories) to compare experiences and outcomes. Analysis of the data shows that the online students appreciated the videos and gained self-confidence in the procedures, but maintained more misconceptions about the material. In addition to being unable to perform the hands-on experiments, other factors such as anxiety could also be interfering with the learning process under the pandemic conditions. The implementation process for the remote labs, student reactions, and lessons learned are discussed.
| S-EPMC10557494 | biostudies-literature
Project description:The use of low temperature thermal detectors for avoiding Darwin-Bragg losses in lab-based ultrafast experiments has begun. An outline of the background of this new development is offered, showing the relevant history and initiative taken by this work.
| S-EPMC5367090 | biostudies-other
Project description:We report the results of a novel protocol for running online experiments using a combination of an online experimental platform in parallel with web-conferencing software in two formats—with and without subject webcams—to improve subjects’ attention and engagement. We compare the results between our online sessions with the offline (lab) sessions of the same experiment. We find that both online formats lead to comparable subject characteristics and performance as the offline (lab) experiment. However, the webcam-on protocol has less noisy data, and hence better statistical power, than the protocol without a webcam. The webcam-on protocol can detect reasonable effect sizes with a comparable sample size as in the offline (lab) protocol. Supplementary Information The online version contains supplementary material available at 10.1007/s40881-021-00112-w.
| S-EPMC8649685 | biostudies-literature
Project description:Humanized monoclonal antibodies (mAbs) are among the most promising modern therapeutics, but defined engineering strategies are still not available. Antibody humanization often leads to a loss of affinity, as it is the case for our model antibody Ab2/3H6 (PDB entry 3BQU). Identifying appropriate back-to-mouse mutations is needed to restore binding affinity, but highly challenging. In order to get more insight, we have applied molecular dynamics simulations and correlated them to antibody binding and expression in wet lab experiments. In this study, we discuss six mAb variants and investigate a tyrosine conglomeration, an isopolar substitution and the improvement of antibody binding towards wildtype affinity. In the 3D structure of the mouse wildtype, residue R94h is surrounded by three tyrosines which form a so-called 'tyrosine cage'. We demonstrate that the tyrosine cage has a supporting function for the CDRh3 loop conformation. The isopolar substitution is not able to mimic the function appropriately. Finally, we show that additional light chain mutations can restore binding to wildtype-comparable level, and also improve the expression of the mAb significantly. We conclude that the variable light chain of Ab2/3H6 is of underestimated importance for the interaction with its antigen mAb 2F5.
| S-EPMC6277173 | biostudies-literature
Project description:For insects, choosing a favorable oviposition site is a type of parental care, as far as it increases the fitness of its offspring. Niche theory predicts that crickets should show a bell-shaped oviposition response to substrate moisture. However, lab experiments with mole crickets showed a linear oviposition response to substrate moisture. Studies with the house cricket Acheta domesticus also showed a linear juvenile body growth response to water availability, thus adult ovipositing females should respond positively to substrate moisture. We used a field experiment to evaluate the relationship between oviposition preference and substrate moisture in forest litter-dwelling cricket species. We also evaluated oviposition responses to substrate moisture level in Ubiquepuella telytokous, the most abundant litter cricket species in our study area, using a laboratory study. We offered cotton substrate for oviposition which varied in substrate moisture level from zero (i.e., dry) to maximum water absorption capacity. We used two complementary metrics to evaluate oviposition preference: (i) presence or absence of eggs in each sampling unit as binary response variable, and (ii) number of eggs oviposited per sampling unit as count response variable. To test for non-linear responses, we adjusted generalized additive models (GAMM) with mixed effects. We found that both cricket oviposition probability and effort (i.e., number of eggs laid) increased linearly with substrate moisture in the field experiment, and for U. telytokous in the lab experiment. We discarded any non-linear responses. Our results demonstrate the importance of substrate moisture as an ecological niche dimension for litter crickets. This work bolsters knowledge of litter cricket life history association with moisture, and suggests that litter crickets may be particularly threatened by changes in climate that favor habitat drying.
| S-EPMC5627918 | biostudies-literature
Project description:Online experiments are growing in popularity. This study aimed to determine the timing accuracy of web technologies and investigate whether they can be used to support high temporal precision psychology experiments. A dynamic sinusoidal grating and flashes were produced by setInterval, CSS3, and requestAnimationFrame (hereafter, rAF) technologies. They were run at normal or real-time priority processing in Chrome, Firefox, Edge, and Internet Explorer on Windows, macOS, and Linux. Timing accuracies were compared with that of Psychtoolbox which was chosen as gold standard. It was found that rAF with real-time priority had the best timing accuracy compared to the other web technologies and had a similar timing accuracy as Psychtoolbox in traditional experiments in most cases. However, rAF exhibited poor timing accuracy on Linux. Therefore, rAF can be used as technical basis for accuracy of millisecond timing sequences in online experiments, thereby benefiting the psychology field.
| S-EPMC7351209 | biostudies-literature
Project description:We apply an online optimization process based on machine learning to the production of Bose-Einstein condensates (BEC). BEC is typically created with an exponential evaporation ramp that is optimal for ergodic dynamics with two-body s-wave interactions and no other loss rates, but likely sub-optimal for real experiments. Through repeated machine-controlled scientific experimentation and observations our 'learner' discovers an optimal evaporation ramp for BEC production. In contrast to previous work, our learner uses a Gaussian process to develop a statistical model of the relationship between the parameters it controls and the quality of the BEC produced. We demonstrate that the Gaussian process machine learner is able to discover a ramp that produces high quality BECs in 10 times fewer iterations than a previously used online optimization technique. Furthermore, we show the internal model developed can be used to determine which parameters are essential in BEC creation and which are unimportant, providing insight into the optimization process of the system.
| S-EPMC4867626 | biostudies-literature