Ensuring Reproducibility in Computational Processes: Automating Data Identification/Citation and Process Documentation

Abstract

Reproducibility of experiments is a key foundation in the empirical sciences. Yet, both the perceived complexity as well as proposed solutions sometimes fail to grasp the full extent of the problem. At the same time, reproducibility is often perceived as a goal in its own right, rather than questioning what precisely we may gain from the investment of effort required to make a specific experiment reproducible. Last, but not least, we need to consider the fact that computational aspects are pervading virtually all scientific disciplines - yet we cannot expect every domain scientist to become an expert in addressing computational reproducibility issues.

In this talk I will review a few examples of reproducibility challenges in computational environments and discuss their potential effects. Based on discussions in a recent Dagstuhl seminar we will identify different types of reproducibility. Here, we will focus specifically on what we gain from them, rather than seeing them merely as means to an end. We subsequently will address two core challenges impacting reproducibility, namely (1) understanding and automatically capturing process context and provenance information, and (2) approaches allowing us to deal with dynamically evolving data sets relying on recommendation of the Research Data Alliance (RDA). The goal is to raise awareness of reproducibility challenges and show ways how these can be addressed with minimal impact on the researchers via research infrastructures offering according services.