Adjusting for Publication Bias with JASP & R

November 17 - 2021

This post is a synopsis of Bartoš, F., Maier, M., Quintana D. S., & Wagenmakers, E. (2021). Adjusting for Publication Bias in JASP & R — Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis. Preprint available at https://doi.org/10.31234/osf.io/kvsp7

Abstract

Meta-analyses are essential for cumulative science, but their validity can be compromised by publication bias. In order to mitigate the impact of publication bias, one may apply publication bias adjustment techniques such as PET-PEESE and selection models. Implemented in JASP & R, these methods allow researchers without programming experience to conduct state-of-the-art publication bias adjusted meta-analysis. In this tutorial, we demonstrate how to conduct a publication bias adjusted meta-analysis in JASP & R and interpret the results. First, we explain two frequentist bias correction methods: PET-PEESE and selection models. Second, we introduce Robust Bayesian Meta-Analysis (RoBMA), a Bayesian approach that simultaneously considers both PET-PEESE and selection models. We illustrate the methodology on an example data set, provide an instructional video, a R-markdown script, and discuss the interpretation of the results. Finally, we include concrete guidance on reporting the meta-analytic results in an academic article.

Highlights

In the updated preprint we provide an accessible introduction to performing publication bias adjusted meta-analyses with both JASP and R. The JASP portion of the tutorial is recorded by Daniel Quintana and published on the JASP YouTube channel,

while the R portion of the tutorial is accompanied by a commented R-markdown script.

We hope that these resources will facilitate the adoption of state-of-the-art publication bias adjustment methods in the research community.

References

Bartoš, F., Maier, M., Wagenmakers, E. J., Doucouliagos, H., & Stanley, T. D. (2021). No need to choose: Robust Bayesian meta-analysis with competing publication bias adjustment methods. Preprint. https://psyarxiv.com/kvsp7/

Bartoš, F., Maier, M., Quintana D. S., & Wagenmakers, E. (2021). Adjusting for publication bias in JASP & R — Selection models, PET-PEESE, and robust Bayesian meta-analysis. Preprint. https://doi.org/10.31234/osf.io/75bqn

Maier, M., Bartoš, F., & Wagenmakers, E. J. (in press). Robust Bayesian meta-analysis: Addressing publication bias with model-averaging. Psychological Methods. (Preprint available at https://psyarxiv.com/u4cns/)

Stanley, T. D., & Doucouliagos, H. (2014). Meta‐regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60-78. https://doi.org/10.1002/jrsm.1095

Vevea, J. L., & Hedges, L. V. (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika, 60(3), 419–435. https://doi.org/10.1007/BF02294384