Latest Version: 0.2.x
Testarium is a research tool to perform experiments and store results as in the repository (git or mercurial). It implements scientific template of experiments and uses numpy, colorama, flask, d3, jquery-ui and angular to provide powerful shell and beautiful presentation of your experimental data and scores.
It can be helpful for science researchers and their bosses to monitor the work progress and to optimize parameters.
$ pip install testarium
{
"a" : 0.5
}
project/example.py
import random
import testarium
import testarium.score.fafr
@testarium.experiment.set_run
def my_run(commit):
# prepare
d = commit.dir
c = commit.config
# run experiment
pos = open(d + '/pos.txt', 'w')
neg = open(d + '/neg.txt', 'w')
[pos.write(str(random.random() * c['a']) + '\n') for _ in xrange(100)]
[neg.write(str(random.random() * c['a']) + '\n') for _ in xrange(100)]
return 0
@testarium.experiment.set_score
def my_score(commit):
d = testarium.score.fafr.Score(commit.dir)
d['test.param'] = 1
return d
@testarium.testarium.set_print
def my_print(commit):
score = '%0.2f' % (commit.desc['score'] * 100.0)
return ['name', 'a', 'test', 'score'], \
[commit.name, commit.config['a'], commit.desc['test.param'], score]
if __name__ == '__main__':
testarium.testarium.best_score_is_max()
testarium.main()
$ python example.py run
t> New commit: 20190123.150408 [default]
t> 20190123.150408 > a: 0.5 > test: 1 > score: 46.00
Typical scientific template of the experiment setup:
Main testarium unit is a commit. The commit consists of two parts: config and result scores & data (experiment description, desc). Each experiment run = new testarium commit.
Found a bug? Have a good idea for improving Testarium? Head over to Testarium's github page and create a new ticket or fork. You can also add issues and feature requests directly to the issue tracker.