Replicability and reproducibility of neural network simulations
Scientific progress depends on reliable experimental tests of precisely defined hypotheses. Just as ever larger sets of experimental data require increasingly complex data analysis workflows (see the Workshop in Reproducibility for Data Analysis proposed by Thomas Wachtler, Sonja Grün and Michael Denker), we face growing problems in ensuring and controlling the reliability of more and more complex simulation studies in the neurosciences.
A successful assessment of scientific results requires both replicability and reproducibility of simulations. Whereas replicability refers to re-running a simulation (simulation code and software environment) that generated a particular result, reproducibility means that results can be obtained independently, solely based on the given model definition. This requires model details to be provided in an understandable format in addition to simulation code.
As both experimental data and computing resources become increasingly available, computational models grow in size and complexity. Therefore, standards in model definition both on the level of documentation (Nordlie et al., 2009) and software (Davison et al., 2008) in conjunction with tracking of software versions, parameter combinations (Davison et al., 2012) and the simulation workflow (Stevens et al., 2013) are required for the replication and reproduction of results. Well-established workflows increase scientists’ confidence in their simulation results, and hence their motivation to share and commonly develop models on community platforms such as Open Source Brain or the HBP Unified Portal. In this workshop, we will discuss all steps of repeatable and reproducible research from model definition to publication and give a best-practice demonstration of a cortical-network simulation with NEST.
In times of increased computational resources also model complexity grows to a level where replicability and reproducibility are lacking without following proper standards. Although appropriate tools for managing and sharing simulation studies are available, computational neuroscientists still hesitate to make use of them. The promotion of tools and platforms to the community is thus essential for the adaptation of the tools to the scientists’ needs and hence for the progress in computational neuroscience.
||Hans Ekkehard Plesser, Norwegian University of Life Sciences
Intro: What’s the problem?
||Andrew Davison, Centre National de la Recherche Scientifique
Provenance tracking for neuronal network simulations
|15:15||Susanne Kunkel, Jülich Research Centre
Replicable simulations with NEST
|16:15||Jean-Luc R. Stevens, The University of Edinburgh
Reproducible neuroscience with Lancet and HoloViews
|17:00||Jeff Muller, École polytechnique fédérale de Lausanne
HBP Unified Portal: An integrated platform for modeling and analysis
|17:45||David Dahmen, Jülich Research Centre
Best practice demonstration: Simulating a cubic millimeter of cortex with NEST