Tl;dr In 1847, Augustus De Morgan suggested that researchers could avoid overselling their work if, every time they made a key claim, they reminded the reader (and themselves) of how confident they were in making that claim. In 1971, Eric Minturn went further and proposed that such confidence could be expressed as a wager, with beneficial side-effects: “Replication would be…

“Anything is fair in love and war” — this saying also applies to the eternal struggle between frequentists (those who draw conclusions based on the performance of their procedures in repeated use) and Bayesians (those who quantify uncertainty for the case at hand). One argument that frequentists have hurled at the Bayesian camp is that “Bayesian procedures do not control…

Over the past year or so I’ve been working on a book provisionally titled “Bayesian bedtime stories”. Below is a part of the preface. This post continues the cooking analogy from the previous post. Like cooking, reasoning under uncertainty is not always easy, particularly when the ingredients leave something to be desired. But unlike cooking, reasoning under uncertainty can be…

Over the past year or so I’ve been working on a book provisionally titled “Bayesian bedtime stories”. Below is a part of the preface. The next post continues the cooking analogy by introducing the frequentist chef. Even though the book [Bayesian Bedtime Stories] addresses a large variety of questions, the method of reasoning is always based on the same principle:…

if there could be any mortal who could observe with his mind the interconnection of all causes, nothing indeed would escape him. For he who knows the causes of things that are to be necessarily knows all the things that are going to be. (…) For the things which are going to be do not come into existence suddenly, but…

This post is an extended synopsis of a preprint that is available on PsyArXiv. “[…] if you can’t do simple problems, how can you do complicated ones?” — Dennis Lindley (1985, p. 65) Cross-validation (CV) is increasingly popular as a generic method to adjudicate between mathematical models of cognition and behavior. In order to measure model generalizability, CV quantifies out-of-sample…

This post summarizes the content of an article that is in press for Behavior Research Methods. The preprint is available on PsyArXiv. Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. First, we considered a modeling strategy that ignores…