Replication is the most important statistic. If an experimental effect does not replicateóeither in your own lab or independentlyóthen trust in that effect should rightly diminish. Thus, assessing the reproducibility of an effect should rightly be among the priorities of psychological scientists. Despite the importance, replication attempts in psychology are rare, likely due to the enhanced incentive (e.g. publications, grant income) to introduce new ideas to the field rather than assess the validity of old ones.
I was involved in the work conducted by the Open Science Collaboration, which was published in the journal Science in 2015. This paper was a large-scale, world-wide collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science. The collaboration attempted to replicate 100 studies published in three prominent psychology journals in 2008, with the aim of calculating an empirical rate of replication, as well as investigating factors that predict reproducibility. The news wasn’t pretty: Reproducibility in psychological science is low.
I have been involved in several published projects working on improving reproducibility in psychological science. I also have ongoing work looking factors that might influence reproducibility in work more closely related to my own, and how to improve these factors.
Representative Publications
Lakens, D., Adolfi, F., Albers, C.,… …Grange, J.A.,… …& Zwaan, R. (2018). Justify your alpha. Nature Human Behavior. (Link)
Anderson, C. J.,Bahnik, S., Barnett-Cowan, M.,… …Grange, J.A.… … & Zuni, K. (2016). Response to a comment on ìEstimating the reproducibility of psychological scienceî. Science, 351, 1037. (Link)
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, 943. (Link)
Open Science Collaboration. (2013). The Reproducibility Project: A model of large-scale collaboration for empirical research on reproducibility. In V. Stodden, F. Leisch, & R. Peng (Eds.), Implementing Reproducible Computational Research (A Volume in the R Series), New York, NY: Taylor & Francis. (Link)
Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7, 657-670. (Link)