Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection

This post is an extended synopsis of a preprint that is available on PsyArXiv. “[…] if you can’t do simple problems, how can you do complicated ones?” — Dennis Lindley (1985, p. 65) Cross-validation (CV) is increasingly popular as a generic method to adjudicate between mathematical models of cognition and behavior. In order to measure model generalizability, CV quantifies out-of-sample…

read more

On the Importance of Avoiding Shortcuts in Applying Cognitive Models to Hierarchical Data

This post summarizes the content of an article that is in press for Behavior Research Methods. The preprint is available on PsyArXiv. Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. First, we considered a modeling strategy that ignores…

read more

Musings on Preregistration: The Case of the Facial Feedback Effect

tl;dr. In 2016, the results of a multi-lab preregistered replication effort cast doubt on the idea, motivated by the “facial feedback hypothesis”, that holding a pen with one’s teeth (instead of with one’s lips) makes cartoons appear more amusing. The effect’s progenitor, Dr. Strack, critiqued the replication effort and suggested that the presence of a camera (put in place to…

read more

Karl Pearson’s Worst Quotation?

The famous statistician Karl Pearson was also a eugenicist, so there are a great many hair-raising quotations to choose from. I nominate the following two for being particularly shocking (for more information see Wikipedia and the Guardian). Brace yourself, here is quotation number one:                       “History shows me one way,…

read more

Karl Pearson’s Best Quotation?

NB. The next post will discuss two of Karl Pearson’s worst quotations.                 “The field of science is unlimited; its material is endless, every group of natural phenomena, every phase of social life, every stage of past or present development is material for science. The unity of all science consists alone in its…

read more

Quantifying Support for the Null Hypothesis in Psychology: An Empirical Investigation

This post summarizes the content of an article that is in press for Advances in Methods and Practices in Psychological Science.1 The preprint is available on PsyArXiv. In the traditional statistical framework, nonsignificant results leave researchers in a state of suspended disbelief. This study examines, empirically, the treatment and evidential impact of nonsignificant results. Our specific goals were twofold: to…

read more

Bayesian Reanalysis of Null Results Reported in Medicine: Strong Yet Variable Evidence for the Absence of Treatment Effects

This post summarizes the content of an article that is in press for PLOS ONE. The preprint is available on PsyArXiv. Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null…

read more

Bayesian Tutorials Galore

This post highlights a recent special issue on Bayesian inference edited by Joachim Vandekerckhove, Jeff Rouder, and John Kruschke for Psychonomic Bulletin & Review. What sets this special issue apart is that most of the 16 contributions (spanning a total of 285 pages!) have a tutorial character. Researchers and students who are new to Bayesian inference –its theoretical underpinnings, its…

read more