TEST DirectivesAuto-pilot can run an arbitrary program to determine if a given benchmark requires more iterations. If the program returns true (zero), then the benchmark is complete, otherwise more iterations are required.
Getstats supports a predicate mode specifically designed to evaluate these conditions. For each quantity (e.g., Elapsed time) that is measured the predicate is evaluated independently. If the predicate is false for any of the quantities, then Getstats returns false (non-zero).
To evaluate a predicate Getstats performs replacement, and then executes
Perl's eval function on the string. The primary replacements that we
use are $name, $mean, and $delta (i.e., the half-width
of a 95% confidence interval).
For example, the following Getstats command ensures that the User,
System, and Elapsed have half-widths that are less than 5% of the mean,
and no test may run more than 30 times: getstats --predicate
'("$name" ne "User" && "$name" ne "System" && "$name" ne "Elapsed") ||
("$delta" < 0.05 * $mean) || ($count > 30)".
See Replacements, for detailed information about available replacements.