Parameters combination¶
Then submission of parametric jobs requires to gather in a parameter file all the combinations of parameters that one wants to run a job against. This list of combination can be described as an explicit array of values of programatically via a Python or shell script or using simple directives.
While the execution of parametric workflows is described here, here are detailed four ways of defining parameters. .
array of values¶
The simplest way to describe the set of parameter combinations that needs to be tested consists in listing them extensively as an array of values. The first row of this array is the name of each parameters and each row is one possible combination.
Here is a parameters file listing all possible combinations for 3 parameters (i,j,k), each of them taking the value 1 or 2.
# array-like description of parameter combinations
i j k
1 1 1
1 1 2
1 2 1
1 2 2
2 1 1
2 1 2
2 2 1
2 2 2
Notice that:
- spaces, void lines are ignored.
- every thing following a # is considered as a comment and ignored
Combined parameter sweep¶
In case of combinations that sweeps all possible set of values based on the domain definition of each variable, a more compact declarative syntax is also available. The same set of parameters can be generated with the following file:
# combine-like description of parameter combinations
#DECIM COMBINE i = [1,2]
#DECIM COMBINE j = [1,2]
#DECIM COMBINE k = [1,2]
Every line starting with #DECIM is parsed as a special command.
Parameters depending on simple formulas¶
Some parameters can also be computed from others using simple arithmetic formulas. Here is a way to declare them:
# combine-like description of parameter combinations
#DECIM COMBINE i = [1,2]
#DECIM COMBINE j = [1,2]
#DECIM COMBINE k = [1,2]
#DECIM p = i*j*k
which is a short way to describe the same 8 combinations as expressed in the following array-like parameter file:
# array-like description of parameter combinations
i j k p
1 1 1 1
1 1 2 2
1 2 1 2
1 2 2 4
2 1 1 2
2 1 2 4
2 2 1 4
2 2 2 8
an additional parameter can also be described by a list of values:
# combine-like description of parameter combinations
#DECIM COMBINE i = [1,2]
#DECIM COMBINE j = [1,2]
#DECIM COMBINE k = [1,2]
#DECIM p = i*j*k
#DECIM t = [1,2,4,8,16,32,64,128,256]
which is a short way to describe the same 8 combinations as expressed in the following array-like parameter file:
# array-like description of parameter combinations
i j k p t
1 1 1 1 1
1 1 2 2 2
1 2 1 2 4
1 2 2 4 8
2 1 1 2 16
2 1 2 4 32
2 2 1 4 64
2 2 2 8 128
For each parameter added via a list of values, the conformance with the existing number of already possible combinations is checked. For example, the following parameter file…
# combine-like description of parameter combinations
#DECIM COMBINE i = [1,2]
#DECIM COMBINE j = [1,2]
#DECIM COMBINE k = [1,2]
#DECIM p = i*j*k
#DECIM t = [1,2,4,8,16,32,64,128,256]
…produces the error:
[ERROR] parameters number mistmatch for expression
[ERROR] t = [1,2,4,8,16,32,64,128,256]
[ERROR] --> expected 8 and got 9 parameters...
More complex Python expressions¶
For a high number of parameters, a portion of code written in Python can also be embedded after a #DECIM PYTHON directive till the end of the file.
# pythonic parameter example file
#DECIM COMBINE nodes = [2,4,8]
#DECIM COMBINE ntasks_per_node = [16,32]
#DECIM k = range(1,7)
#DECIM PYTHON
import math
ntasks = nodes*ntasks_per_node
nthreads = ntasks * 2
NPROC = 2; #Number of processors
t = int(2**(k))
T = 15
which is a short way to describe the same 8 combinations as expressed in the following array-like parameter file:
# array-like description of parameter combinations
nodes ntasks_per_node k ntasks nthreads t NPROC T
2 32 1 64 128 2 2 15
2 64 2 128 256 4 2 15
4 32 3 128 256 8 2 15
4 64 4 256 512 16 2 15
8 32 5 256 512 32 2 15
8 64 6 512 1024 64 2 15
A python section is always evaluated at the end. Each new variables set at the end of the evaluation is added as a new parameter computed against each of the already built combinations. The conformance to the number of combinations already set is also checked if the variable is a set of values.