Toward more transparent statistical reporting in empirical research
When performing statistical analysis on empirical data, researchers inevitably make a series of arbitrary choices among several options for processing data (i.e. exclusions of participants for various reasonable reasons, aggregation of participants based on one data dimension, data transformations, etc.) and analysing it (i.e. frequentist vs. Bayesian inferential models, corrections, inclusion or exclusion of different variables, etc.). Due to space restrictions and possibly other reasons, researchers typically practice selective reporting of their results, that is, only one particular possibility among the larger landscape of reasonable options, even if they have experimented with multiple paths in their research. But these arbitrary choices may be misleading, in that some choices may result in fluctuations in conclusions. Revealing the outcome of the multiple (reasonable) analysis paths for a data set would inform the reader on the fragility or robustness of the presented results. In this talk, I will discuss novel ways of augmenting the scholarly article medium to convey the detailed results of multiverse analysis to the readership.