Below is a summary of a preprint featuring an extensive reanalysis of the results Many Labs 4 project (current preprint). ML4 attempted to replicate the mortality salience effect. Following the publication of the preprint a heated debate broke out about data inclusion criteria. In an attempt of conciliation we decided to reanalyze the data using all proposed data inclusion criteria in a multiverse analysis. The figure below shows the results of this analysis.
Abstract
Many Labs projects have become the gold standard for assessing the replicability of key findings in psychological science. The Many Labs 4 project recently failed to replicate the mortality salience effect where being reminded of one’s own death strengthens the own cultural identity. Here, we provide a Bayesian reanalysis of Many Labs 4 using meta-analytic and hierarchical modeling approaches and model comparison with Bayes factors. In a multiverse analysis we assess the robustness of the results with varying data inclusion criteria and prior settings. Bayesian model comparison results largely converge to a common conclusion: We find evidence against a mortality salience effect across the majority of our analyses. Even when ignoring the Bayesian model comparison results we estimate overall effect sizes so small (between d = 0.03 and d = 0.18) that it renders the entire field of mortality salience studies as uninformative.
Results from the Multiverse Analysis
Bayes factors in favor of a mortality salience effect are above the horizontal line, Bayes factors against the mortality salience effect are below the horizontal line. The color of the points refers to the different priors on the overall effect, the size of the points refers to the number of studies included in the analysis, and the x-axis refers to the number of participants the analysis is based on. The majority of analyses provide evidence against the mortality salience effect.
References
Haaf, J. M., Hoogeveen, S., Berkhout, S., Gronau, Q. F., & Wagenmakers, E. (2020). A Bayesian Multiverse Analysis of Many Labs 4: Quantifying the Evidence against Mortality Salience. Retrieved from psyarxiv.com/cb9er
Klein, R. A., Cook, C. L., Ebersole, C. R., Vitiello, C. A., Nosek, B. A., Chartier, C. R., … Ratliff, K. A. (2019). Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement. https://doi.org/10.31234/osf.io/vef2c
About The Author
Julia Haaf
Julia Haaf is postdoc at the Psychological Methods Group at the University of Amsterdam.
