tablix2_benchmark - Tablix benchmark utility

tablix2_benchmark benchmark [ options ] Tablix options file

Tablix is a powerful free software kernel for solving general timetabling problems. It uses a coarse-grained parallel genetic algorithm in combination with other techniques to construct sensible timetables from XML formatted problem descriptions. Tablix can run on a single host as well as on a heterogeneous parallel virtual machine using PVM3.

tablix2_benchmark is used to test the effect of different Tablix command line options on the final result. In most tests Tablix is started multiple times with the same input file to provide more reliable results. Each time Tablix is started, three variables are measured: the fitness value of the resulting timetable, serial number of the last generation and elapsed time (in seconds). When a test is finished, the average for each variable, as well as the maximum and minimum values are printed on standard output. The exact format depends on each test.

The actual timetables that are generated during these tests are stored under a temporary file names in the current directory and are deleted after tablix2_benchmark exits.

In some cases tests can take several days to complete. Because the genetic algorithm can in some cases go into an endless loop, it is always advisable to limit Tablix execution time with the -t option. This way at least the maximum execution time for a test can be calculated. tablix2_benchmark will detect when the time limit set by the -t option was reached. This information will be included in the final statistics.

The following tests are supported:

tablix2_benchmark --single OPTIONS file

Execute a single Tablix run using and print the results. OPTIONS parameter is mandatory and contains options that will be passed to the tablix2 process.

tablix2_benchmark --multiple N OPTIONS FILE

Execute N Tablix runs and print the statistics. OPTIONS parameter is mandatory and contains options that will be passed to the tablix2 process.

tablix2_benchmark --graph START STEP STOP OPTIONS FILE

Execute multiple Tablix runs. The character N in OPTIONS is substituted for an integer that is changed from START to STOP by STEP . Output is gnuplot friendly.

tablix2_benchmark --multi-graph M START STEP STOP OPTIONS FILE

Execute multiple Tablix runs. The character N in OPTIONS is substituted for an integer that is changed from START to STOP by STEP . M Tablix runs are executed for each value of N . Output is gnuplot friendly (use errorbars).

The following test can be for example used to see the minimum grade, that can be reached with this configuration file:

tablix2_benchmark --multiple 5 -t 90 -n 5 sample.xml

Tablix will be started five times. Each time with five computing nodes and with 90 minute time limit.

tablix2_benchmark can also provide useful data for developers. Following test can be used to check the effect of the number of computing nodes on the total time required to obtain the result:

tablix2_benchmark --multi-graph 5 1 1 10 -t 90 -n N sample.xml > stats.txt

Tablix will be started 50 times. 5 times for each value of N (which in this case means the number of nodes). Ten values of N will be probed: from 1 to 10 with steps of 1. The output will be redirected to stats.txt . A graph of average computing time versus number of nodes can then be produced in gnuplot with the following command:

plot test.txt using ($1):($8):($9):($10) with yerrorbars

tablix2_benchmark utility depends on a number of common UNIX programs (mktemp, grep, sed, awk, ...) that may not be available on all systems. The script is not currently capable of properly detecting if any required programs are missing.

Tomaz Solc (tomaz.solc@siol.net)

tablix2(1), Tablix User's Manual, Tablix modules HOWTO, Tablix on Morphix HOWTO