tablix_benchmark - Tablix benchmark utility

tablix_benchmark benchmark [ options ] Tablix options file

Tablix is a high school timetable generator. It uses a coarse-grained parallel genetic algorithm to construct sensible timetables from XML-formated school information files. It can run on a single host as well as on a heterogeneous parallel virtual machine by using PVM3.

tablix_benchmark is used to test the effect of different Tablix command line options on the final result. In most tests Tablix is started multiple times with the same input file to provide more reliable results. Each time Tablix is started, three variables are measured: the grade (fitness) of the resulting timetable, serial number of the last generation and elapsed time (in seconds). When a test is finished, the average for each variable, as well as the maximum and minimum values are printed on standard output. The exact format depends on each test.

The actual timetable that is generated during these tests is stored under the default filename in the current directory. In case the test includes multiple Tablix runs only the last generated timetable remains in the directory after tablix_benchmark exits.

In some cases tests can take several days to complete. Because the genetic algorithm can in some cases go into an endless loop, it is always advisable to limit Tablix execution time with the -t option. This way at least the maximum execution time for a test can be calculated. tablix_benchmark will detect when the time limit set by the -t option was reached. This information will be included in the final statistics.

The following tests are supported:

tablix_benchmark --single OPTIONS file

Execute a single Tablix run using OPTIONS and print the results.

tablix_benchmark --multiple N OPTIONS FILE

Execute N Tablix runs using OPTIONS and print the statistics.

tablix_benchmark --graph START STEP STOP OPTIONS FILE

Execute multiple Tablix runs. N in OPTIONS is changed from START to STOP by STEP . Output is gnuplot friendly.

tablix_benchmark --multi-graph M START STEP STOP OPTIONS FILE

Execute multiple Tablix runs. N in OPTIONS is changed from START to STOP by STEP . M Tablix runs are executed for each value of N . Output is gnuplot friendly (use errorbars).

The following test can be for example used to see the minimum grade, that can be reached with this configuration file:

tablix_benchmark --multiple 5 -t 90 -n 5 sample.xml

Tablix will be started five times. Each time with five computing nodes and with 90 minute time limit.

tablix_benchmark can also provide useful data for developers. Following test can be used to check the effect of the number of computing nodes on the total time required to obtain the result:

tablix_benchmark --multi-graph 5 1 1 10 -t 90 -n N sample.xml > stats.txt

Tablix will be started 50 times. 5 times for each value of N (which in this case means the number of nodes). Ten values of N will be probed: from 1 to 10 with steps of 1. The output will be redirected to stats.txt . A graph of average computing time versus number of nodes can then be produced in gnuplot with the following command:

plot test.txt using ($1):($8):($9):($10) with yerrorbars

Tomaz Solc (tomaz.solc@siol.net)

tablix(1), Tablix User's Manual, Tablix modules HOWTO, Tablix on Morphix HOWTO