Conference site » Proceedings

The Sacred Infrastructure for Computational Research

Klaus Greff
Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA)
Università della Svizzera italiana (USI)
Scuola universitaria professionale della Svizzera italiana (SUPSI)

Aaron Klein
University of Freiburg

Martin Chovanec
Czech Technical University in Prague

Frank Hutter
University of Freiburg

Jürgen Schmidhuber
Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA)
Università della Svizzera italiana (USI)
Scuola universitaria professionale della Svizzera italiana (SUPSI)

Abstract

We present a toolchain for computational research consisting of Sacred and two supporting tools. Sacred is an open source Python framework which aims to provide basic infrastructure for running computational experiments independent of the methods and libraries used. Instead, it focuses on solving universal everyday problems, such as managing configurations, reproducing results, and bookkeeping. Moreover, it provides an extensible basis for other tools, two of which we present here: Labwatch helps with tuning hyperparameters, and Sacredboard provides a web-dashboard for organizing and analyzing runs and results.

Keywords

reproducible research, Python, machine learning, database, hyperparameter optimization

Bibtex entry

Full text PDF