What can scientists do?

If you are aware of the practices that often lead to bias, you can make choices in your own work to minimize bias.

  • Be sure to thoroughly report details of your data analyses, including sample sizes, effect sizes, etc.
  • Pre-register your analysis plans. Before you gather data (or in the case of long-term studies, before you begin to examine your data), devise a specific plan for your study design and data analysis and post it in a pre-registration archive (archives typically have templates to make this easier – here is one on the Open Science Framework). Later, when it comes time to publish your results, you will be able to demonstrate that your analyses were planned, and thus not subject to the biases that can accompany incompletely reported exploratory analyses.
  • When you do publish exploratory analyses (and you should – exploratory work is important), be sure to identify analyses as exploratory, and be sure to report all such analyses in full detail (possibly in supplementary materials). Do not just publish the exploratory analyses that produced the most ‘interesting’ results.

When you review papers, check to see that authors have adhered to basic principles of transparency. If they have not, describe how such standards could be met and request such transparency in your review. Some forms of insufficient transparency include:

  • sample sizes not reported from all subsets of data
  • insufficient details of analytical methods
  • failure to describe whether observers were blind to expected outcomes
  • insufficient details of results (what qualifies as sufficient detail varies among types of analysis, but often includes parameter estimates and estimates of variability or uncertainty)