Since 2017, Dora Matzke and I have been teaching the master course “Bayesian Inference for Psychological Science”. Over the years, the syllabus for this course matured into a book (and an accompanying book of answers) titled “Bayesian inference from the ground up: The theory of common sense”. The current plan is to finish the book in the next few months, and share the pdf publicly online. For now, you can click here or on the cover page below to read the first 271 pages [62 MB!]. Today we are adding two chapters: “A Crack in the Laplacean Edifice”, and “Wrinch and Jeffreys to the Rescue”. These chapters start Part III of the book, “Coherent Learning, Jeffreys Style”.
The first new chapter
exposes the Achilles heel of Laplacean inference: the Principle of Insufficient Reason, also known as the Principle of Indifference. Although this principle appears neutral and innocuous –probability mass is divided evenly across all parameter values and events– it implies a denial without evidence that a general law is ever true. Universal generalizations that involve a necessary cause (e.g., “all AIDS patients have been exposed to HIV”) are deemed false from the outset, in violation of both common sense and scientific practice.
The next chapter outlines how Dorothy Wrinch and Harold Jeffreys solved the Laplacean problem by assigning separate prior mass to the general law. This makes it possible for a finite number of confirmatory instances to provide support in favor of a general law. For instance, when Kate has observed 12 zombies, and all of them are hungry, this provides evidence in favor of the general law “all zombies are hungry”. More specifically, every hungry zombie provides some additional evidence for the general law, making it ever more plausible. When the alternative law assigns a uniform distribution to the proportion of hungry zombies, and n hungry zombies have been observed, the Bayes factor in favor of the general law equals n+1.
I’ll end this post with some purposefully provocative remarks:
- Hipster methods of model selection are often convenient, but they will fail to provide unbounded evidence in favor of the general law as the hoard of hungry zombies grows indefinitely.
- Some statisticians are hesitant to assign mass to a general law, and instead prefer the approach advocated by Laplace. I believe this betrays a lack of familiarity with historical developments, and with the problem that Wrinch and Jeffreys were trying to address.
References
Wagenmakers, E.-J., & Matzke, D. (2023). Bayesian inference from the ground up: The theory of common sense.
Wagenmakers, E.-J., & Matzke, D. (in preparation). Bayesian inference from the ground up: Common sense in practice.
