Twitter giveth joy, and Twitter taketh it away. This time, Twitter giveth, and in abundance, as I just learned from a tweet of Katie Corker that all APA core titles will require TOP Level 1 transparency for the sharing of data and materials:
The Center for Open Science tweet:
From the COS statement: “The APA said it will officially begin implementing standardized disclosure requirements of data and underlying research materials (TOP Level 1). Furthermore, it encourages editors of core journals to move to Level 2 TOP (required transparency of data and research items when ethically possible). More information on the specific levels of adoption by each of the core journals will be coming in the first half of 2021.”
What does TOP Level 1 for data and materials entail? As outlined in the TOP paper, for “Data, Analytic Methods (Code), and Research Materials Transparency”:
This is close to the kind of transparency required by the PRO initiative, except that the PRO initiative demands that reviewers have early access to data and code in order to assist them in the review process. It is not difficult to predict that adopting TOP Level 1 will greatly enhance the sharing of data and code. Kudos to the APA for taking this step!
”Polderen” with TOP: The Borsboom Levels
The TOP guidelines were developed during a workshop “Journal Standards for Promoting Reproducible Research in the Social-Behavioral Sciences” (November 3-4, 2014) at the Center for Open Science in Charlottesville. The workshop participants were mostly journal editors and Open Science advocates. My colleague Denny Borsboom and I also participated.
The goal of the workshop was to hash out concrete journal guidelines for increasing research transparency. As the workshop progressed, it became increasingly clear that there was little consensus among the journal editors that were present. Some journal editors were very conservative and unwilling to change anything, whereas other editors were much more gung-ho. Progress stalled, and heated arguments ensued. At some point, Denny had a brilliant idea: instead of forcing every editor to adopt a single standard of transparency, we could create different levels of adoption. It would then be the responsibility of every individual editor to choose the level that they felt comfortable with. Moreover, level 0 means “do nothing”, such that even the most conservative editors could get on board with TOP.
Someone else may have thought of this if Denny hadn’t, but regardless I believe that this was a key moment in the development of the TOP guidelines, making them palatable to everyone and allowing editors the freedom they needed. Upon further reflection, I think that the “Borsboom levels” are a nice example of what is known in Dutch as “polderen”, a verb that denotes the kind of political maneuvering whose main purpose is to keep all parties happy. It is said that the desire for “polderen” is motivated by the necessity of the Dutch to cooperate in keeping their country from being flooded.
References
Dahrendorf, M., Hoffmann, T., Mittenbühler, M., Wiechert, S.-M., Sarafoglou, A., Matzke, D., & Wagenmakers, E.-J. (2020). “Because it is the right thing to do”: Taking stock of the Peer Reviewers’ Openness Initiative . Journal of European Psychology Students, 11, 15-20.
Morey, R. D., Chambers, C. D., Etchells, P. J., Harris, C. R., Hoekstra, R., Lakens, D., Lewandowsky, S., Morey, C. C., Newman, D. P., Schönbrodt, F., Vanpaemel, W., Wagenmakers, E.-J., & Zwaan, R. A. (2016). The Peer Reviewers’ Openness Initiative: Incentivising open research practices through peer review . Royal Society Open Science, 3: 150547.
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T. A., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Levy Paluck, E., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T. (2015). Promoting an open research culture. Science, 348, 1422-1425. Details are on the Open Science Framework.
About The Authors
Eric-Jan Wagenmakers
Eric-Jan (EJ) Wagenmakers is professor at the Psychological Methods Group at the University of Amsterdam.
