Latest Version: 0.2.x
What is Testarium?
Testarium is a research tool to perform experiments and store results as in the repository (git or mercurial). It implements scientific template of experiments and uses numpy, colorama, flask, d3, jquery-ui and angular to provide powerful shell and beautiful presentation of your experimental data and scores.
It can be helpful for science researchers and their bosses to monitor the work progress and to optimize parameters.
Setup
$ pip install testariumQuick start
project/config/config.json{
"a" : 0.5
}
import random
import testarium
import testarium.score.fafr
@testarium.experiment.set_run
def my_run(commit):
# prepare
d = commit.dir
c = commit.config
# run experiment
pos = open(d + '/pos.txt', 'w')
neg = open(d + '/neg.txt', 'w')
[pos.write(str(random.random() * c['a']) + '\n') for _ in xrange(100)]
[neg.write(str(random.random() * c['a']) + '\n') for _ in xrange(100)]
return 0
@testarium.experiment.set_score
def my_score(commit):
d = testarium.score.fafr.Score(commit.dir)
d['test.param'] = 1
return d
@testarium.testarium.set_print
def my_print(commit):
score = '%0.2f' % (commit.desc['score'] * 100.0)
return ['name', 'a', 'test', 'score'], \
[commit.name, commit.config['a'], commit.desc['test.param'], score]
if __name__ == '__main__':
testarium.testarium.best_score_is_max()
testarium.main()
$ python example.py run
t> New commit: 20190123.150408 [default]
t> 20190123.150408 > a: 0.5 > test: 1 > score: 46.00
Testarium paradigm
Typical scientific template of the experiment setup:
Main testarium unit is a commit. The commit consists of two parts: config and result scores & data (experiment description, desc). Each experiment run = new testarium commit.
Contribute
Found a bug? Have a good idea for improving Testarium? Head over to
Testarium's github page and
create a new ticket or fork. You can also add issues and feature
requests directly to the
issue tracker.
