Dataset Information

Optimistic update bias holds firm: Three tests of robustness following Shah et al.

ABSTRACT: A diverse body of research has demonstrated that people update their beliefs to a greater extent when receiving good news compared to bad news. Recently, a paper by Shah et al. claimed that this asymmetry does not exist. Here we carefully examine the experiments and simulations described in Shah et al. and follow their analytic approach on our data sets. After correcting for confounds we identify in the experiments of Shah et al., an optimistic update bias for positive life events is revealed. Contrary to claims made by Shah et al., we observe that participants update their beliefs in a more Bayesian manner after receiving good news than bad. Finally, we show that the parameters Shah et al. pre-selected for simulations are at odds with participants' data, making these simulations irrelevant to the question asked. Together this report makes a strong case for a true optimistic asymmetry in belief updating.

SUBMITTER: Garrett N

PROVIDER: S-EPMC5380127 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Optimistic update bias holds firm: Three tests of robustness following Shah et al.

Garrett Neil N Sharot Tali T

Consciousness and cognition 20161109

A diverse body of research has demonstrated that people update their beliefs to a greater extent when receiving good news compared to bad news. Recently, a paper by Shah et al. claimed that this asymmetry does not exist. Here we carefully examine the experiments and simulations described in Shah et al. and follow their analytic approach on our data sets. After correcting for confounds we identify in the experiments of Shah et al., an optimistic update bias for positive life events is revealed. C ...[more]

PMID: 27836628

Similar Datasets

Project description:BACKGROUND:Individuals tend to have an optimism bias, processing desirable information more frequently than undesirable information. However, people who have been diagnosed with depression often have a more pessimistic view of the future. A recent study suggests that future expectations in individuals with dysphoria become more optimistic when asked to make optimistic future predictions about the future. In the present study, we investigated the differential effects of induced optimism training which making optimistic future prediction to change future beliefs in normal participants with various levels of dysphoria (low, mild, high). METHODS:We recruited normal participants (n?=?69) from a local university students and divided participants into three groups (low, mild, high dysphoria) by measuring dysphoric mood. These three groups were assigned to the induced optimism training or control condition. After the training, participants performed the two-stage belief updating task. In the first stage, participants estimated their personal probability of experiencing adverse events while being presented with the average probability of the event occurring to a living person. This information could be desirable for participants(when presented with a probability that was below their estimation) or undesirable (when presented with a probability that was above their estimation). To assess how desirable versus undesirable information influenced beliefs, participants estimated their personal probability of experiencing the events again in the second stage. The amount of update error was calculated as the difference between the estimates in the first stage and the second stage. The difference between the errors was classified as the update bias. RESULTS:After the induced optimism training, individuals with the mild dysphoria demonstrated a higher update bias than low (p?<?.001) and the high dysphoria (p?<?.05) group in induced optimism condition. Significant differences were not found in control group. Results indicates that individuals in the mild dysphoria group showed an increased update bias after being exposed to the induced optimism training Dysphoric mood and trait optimism remained unchanged in both the experimental and control groups. CONCLUSIONS:Results suggest that induced optimism training has potential to change individuals with mild dysphoria perceptions' about the future.

Project description:Head et al. (2015) provided a large collection of p-values that, from their perspective, indicates widespread statistical significance seeking (i.e., p-hacking). This paper inspects this result for robustness. Theoretically, the p-value distribution should be a smooth, decreasing function, but the distribution of reported p-values shows systematically more reported p-values for .01, .02, .03, .04, and .05 than p-values reported to three decimal places, due to apparent tendencies to round p-values to two decimal places. Head et al. (2015) correctly argue that an aggregate p-value distribution could show a bump below .05 when left-skew p-hacking occurs frequently. Moreover, the elimination of p = .045 and p = .05, as done in the original paper, is debatable. Given that eliminating p = .045 is a result of the need for symmetric bins and systematically more p-values are reported to two decimal places than to three decimal places, I did not exclude p = .045 and p = .05. I conducted Fisher's method .04 < p < .05 and reanalyzed the data by adjusting the bin selection to .03875 < p ≤ .04 versus .04875 < p ≤ .05. Results of the reanalysis indicate that no evidence for left-skew p-hacking remains when we look at the entire range between .04 < p < .05 or when we inspect the second-decimal. Taking into account reporting tendencies when selecting the bins to compare is especially important because this dataset does not allow for the recalculation of the p-values. Moreover, inspecting the bins that include two-decimal reported p-values potentially increases sensitivity if strategic rounding down of p-values as a form of p-hacking is widespread. Given the far-reaching implications of supposed widespread p-hacking throughout the sciences Head et al. (2015), it is important that these findings are robust to data analysis choices if the conclusion is to be considered unequivocal. Although no evidence of widespread left-skew p-hacking is found in this reanalysis, this does not mean that there is no p-hacking at all. These results nuance the conclusion by Head et al. (2015), indicating that the results are not robust and that the evidence for widespread left-skew p-hacking is ambiguous at best.

Dataset Information

Optimistic update bias holds firm: Three tests of robustness following Shah et al.

Publications

Optimistic update bias holds firm: Three tests of robustness following Shah et al.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets