Latest Version: 0.2.x

What is Testarium?

Testarium is a research tool to perform experiments and store results as in the repository (git or mercurial). It implements scientific template of experiments and uses numpy, colorama, flask, d3, jquery-ui and angular to provide powerful shell and beautiful presentation of your experimental data and scores.

It can be helpful for science researchers and their bosses to monitor the work progress and to optimize parameters.


$ pip install testarium

Quick start

	"a" : 0.5
#!/usr/bin/env python
import random
import testarium
import testarium.score.fafr

def my_run(commit):
    # prepare
    d = commit.dir
    c = commit.config

    # run experiment
    pos = open(d + '/pos.txt', 'w')
    neg = open(d + '/neg.txt', 'w')
    [pos.write(str(random.random() * c['a']) + '\n') for _ in xrange(100)]
    [neg.write(str(random.random() * c['a']) + '\n') for _ in xrange(100)]
    return 0

def my_score(commit):
    d = testarium.score.fafr.Score(commit.dir)
    d['test.param'] = 1
    return d

def my_print(commit):
    score = '%0.2f' % (commit.desc['score'] * 100.0)

    return ['name', 'a', 'test', 'score'], \
           [, commit.config['a'], commit.desc['test.param'], score]

def my_compare(self, other):
    if self._init:
        if self.desc['score'] > other.desc['score']:
            return -1
        elif self.desc['score'] < other.desc['score']:
            return 1
            return 0
        return -1

if __name__ == '__main__':
$ python run
t> New commit: 20141220.162326
t> 20141220.162326 > branch: default > score: 1.06 > time: 0.0

Testarium paradigm

Typical scientific template of the experiment setup:

Main testarium unit is a commit. The commit consists of two parts: config and result scores & data (experiment description, desc). Each experiment run = new testarium commit.


Found a bug? Have a good idea for improving Testarium? Head over to Testarium's github page and create a new ticket or fork. You can also add issues and feature requests directly to the issue tracker. Fork me on GitHub