This is a DeepL-assisted translation of an article for the Dutch magazine Skepter (Wagenmakers, 2023). I am grateful to the editor, Hans van Maanen, for his efforts in rewriting my original draft. The painting is a self-portrait by René Magritte called ‘La Clairvoyance’ (1936 – proposed by van Maanen to accompany the article). 

TLDR; 

Can psychology students really anticipate the appearance of an erotic picture on the computer? No, they can’t (Kekecs et al., 2023, https://royalsocietypublishing.org/doi/10.1098/rsos.191375)

Twelve years ago, American professor Daryl Bem published a startling series of experiments in the highly regarded Journal of Personality and Social Psychology. According to Bem, these experiments provided compelling evidence for the hypothesis that the future can influence the present. In Bem’s first experiment, for example, participants had to guess whether an image would appear on the left or right side of a computer screen. For erotic images, participants guessed the location correctly in 53.1 percent of the time, well above what could be expected of them based on chance. Bem concluded that this presents evidence for what parapsychology terms psi – extrasensory perceptions that cannot be explained by known physical or biological mechanisms.

Bem’s findings caused quite a stir. At the height of his fame, Daryl Bem was even a guest on Stephen Colbert’s popular talk show – where the host pithily described the results of the first experiment as time-traveling porn. Most scientists were unwilling to embrace Bem’s maverick hypothesis; instead, Bem’s work became a textbook example of how scientists can fool themselves by persistent cherry-picking. In the end, Bem’s results didn’t challenge the laws of physics, but rather the modus operandi of many empirical scientists (see also Skepsis, 2018, no. 4).

 

Replication

However, that does not yet answer the question of whether Eros can actually travel back in time. A few initial attempts at replication found no effect, but supporters of Bem were not convinced. 

To clear up any remaining empirical ambiguity, Hungarian psychologist Zoltán Kekecs and colleagues recently conducted another replication experiment, but one that dwarfs all others.

In Kekecs’ replication, both the experimental design and the method of analysis were preregistered based on a consensus procedure featuring fifteen proponents (including Bem) and fourteen skeptics (including me). All research data were immediately made public and analyzed online; three outside supervisors checked that everything was proceeding as planned. The agreement was to stop the study when the statistical evidence (for or against Bem) was sufficiently convincing and consistent.

 

Inside Out

Kekecs’ article was published in January of this year. As in Bem, each of the 2102 subjects was presented with 18 erotic images, for a total of 37,836 decisions. In 18,876 of the cases, the location of the erotic pictures was guessed correctly – a success rate of 49.89%, or just slightly below chance. 

The data were statistically turned inside out in all sorts of ways, but the conclusion was inescapable: the observed minuscule negative effect is not evidence that people can foresee an erotic future. And what about the other way around? Does Kekecs’ article also offer statistical evidence for the absence of erotic premonitions? This depends on the assumptions made about the size of the effect if it were to exist. If we assume that the psi effect is microscopically small – say, 0.01 percent – then Kekecs’ result is inconclusive; but the larger the true effect is thought to be, the greater the evidence that erotic premonitions do not exist. If the effect is hypothesized to be about the same size as in Bem’s original experiment –around 3 percent– then the hypothesis that erotic premonitions do not exist predicts the data about 180 times better than the hypothesis that they do exist.

That’s pretty strong evidence against the psi hypothesis – but can it convince Bem’s supporters? To my surprise, an online forum discussion revealed that it does: several proponents said they had revised their views on “retroactive influences on cognition”. This is admirable: I wonder whether I myself would have changed my opinion in case the replication should have favored Bem; I suspect I would still find a flaw in the protocol much more likely than the idea of time-traveling porn. 

 

Baloney

Of course, one might wonder what exactly is the point of this time-consuming research. The hypothesis under study is patently baloney. Wouldn’t it then be better to invest money and effort in studying a different hypothesis, so that the outcome is not a foregone conclusion? As an expert in the field of baloney, I think there are important lessons to be learned from this replication after all: the importance of adversarial collaboration (where proponents and opponents collaborate on the research design), the importance of preregistration, and the fact that proponents can apparently be convinced by data after all.

This creates high expectations. Would it perhaps be possible to follow a similar procedure for testing hypotheses concerning homeopathy, chiropractic, or faith healing? 

 

References

Kekecs, Z., Palfi, B., Szaszi, B., Szecsi, P., Zrubka, M., Kovacs, M., Bakos, B. E., Cousineau, D., Tressoldi, P., Schmidt, K., Grassi, M., Evans, T. R., Yamada, Y., Miller, J. K., Liu, H., Yonemitsu, F., Dubrov, D., Röer, J. P., Becker, M., Schnepper, R., Ariga, A., Arriaga, P., Oliveira, R., Põldver, N., Kreegipuu, K., Hall, B., Wiechert, S., Verschuere, B., Girán, K., Aczel, B. (2023). Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project. Royal Society Open Science,10:191375. URL: https://royalsocietypublishing.org/doi/10.1098/rsos.191375 

Wagenmakers, E.-J. (2023). De psi van seks. Skepter, 36, 37-38.