Latest Version: 0.2.x

What is Testarium?

Testarium is a research tool to perform experiments and store results as in the repository (git or mercurial). It implements scientific template of experiments and uses numpy, colorama, flask, d3, jquery-ui and angular to provide powerful shell and beautiful presentation of your experimental data and scores.

It can be helpful for science researchers and their bosses to monitor the work progress and to optimize parameters.

Setup

$ pip install testarium

Quick start

project/config/config.json
{
	"my.param" : 0.5
}
project/example.py
import testarium
import random, json

@testarium.experiment.set_run
def MyRun(commit):
	# read commit config file 
	c = commit.config
	# get commit directory to store experimental data
	dir = commit.dir
	
	# read some param from config
	p = c['my.param'] 
	
	# open file and write some data
	data = open(dir + '/data.dat', 'w')
	data.write(str(random.random() * p)) 
	return 0

@testarium.experiment.set_score
def MyScore(commit):
	# read experiment data 
	data = float(open(commit.dir + '/data.dat').read())
	# prepare description about the experiment
	desc = { 'score': data * 2.0 }
	return desc

if __name__ == '__main__':
	testarium.testarium.best_score_is_max()
	testarium.main()
$ python example.py run
t> New commit: 20141220.162326
t> 20141220.162326 > branch: default > score: 1.06 > time: 0.0

Testarium paradigm

Typical scientific template of the experiment setup:

Main testarium unit is a commit. The commit consists of two parts: config and result scores & data (experiment description, desc). Each experiment run = new testarium commit.

Contribute

Found a bug? Have a good idea for improving Testarium? Head over to Testarium's github page and create a new ticket or fork. You can also add issues and feature requests directly to the issue tracker. Fork me on GitHub