In Professor Wiens own words:

Stefan Wiens, porträtt. Foto: Niklas Björling
Stefan Wiens. Foto: Niklas Björling

My graduate student Rasmus Eklund and I just received the news that we won the $1000 preregistration challenge by Open Science Framework (https://osf.io/).

Although we are excited to have won this award, we feel that we already won a lot simply by preregistering our study on the neural correlates of visual awareness (http://doi.org/10.3758/s13415-018-0562-z).

To understand our feelings, consider that many apparent truths in Psychology have been challenged. One reason is the flexibility that researchers have in analyzing their data (researcher’s degrees of freedom). Simply put, researchers often decide—consciously or unconsciously—on how to analyze their data after looking at their data. If so, the analyses are not independent of the actual data, and the results are very likely to be biased.

This problem is particularly relevant with functional brain imaging data. For example, recordings of electro-encephalography (EEG) comprise many electrodes that are measured over time. This gives researchers enormous flexibility in deciding which electrodes to analyze in what interval. For our topic, previous studies disagreed on whether a particular EEG response may be the neural correlate of visual awareness.

We decided to conduct a replication and extension of previous research. Critically, we preregistered our study (https://osf.io/4a4ur/). This means that we described in detail what we would do and how we would analyze the data before we actually started collecting any data. This description was locked and time-stamped at OSF. Because we ensured that our analyses were defined a priori and were not data driven, the evidential strength of our findings is strong.

Notably, one common sentiment against preregistration is that it seems like a lot of work and challenging to do before collecting any data. We resolved this by collecting pilot data. These data allowed us to test various aspects of the design before we preregistered our study. Of course, preregistration is demanding because one has to think through every step of the study. However, the benefits are that one will already detect problems with the study before spending specious time and resources on collecting data that may be useless.

Further, the preregistration is like a cookbook recipe: After data collection, just follow the recipe on how to analyze the data. As such, the time spent during preregistration is saved later.

To conclude, any researcher should be collecting data that provide strong evidence. Because preregistration minimizes any doubts that the analyses were driven by the data, it is a simple measure to increase the evidential strength of any findings. If you need help with preregistration, just contact me.

Oh, and here is the article, published in Cognitive, Affective, & Behavioral Neuroscience (published online 18 January 2018):
10.3758/s13415-018-0562-z

Stefan Wiens