SST/macro
SST/macro Parameter files

SST/macro Parameter files

A minimal parameter file setting up a 2D-torus topology is shown below. This is meant to be run with the executable compiled in tutorials/sendrecv_cxx.

1 # Launch parameters
2 launch_indexing = block
3 launch_allocation = first_available
4 launch_app1_cmd = aprun -n2 -N1
5 launch_app1 = user_mpiapp_cxx
6 launch_app1_argv =
7 # Network parameters
8 network_name = simple
9 network_bandwidth = 1.0GB/s
10 network_hop_latency = 100ns
11 # Topology - Ring of 4 nodes
12 topology_name = hdtorus
13 topology_geometry = 4,4
14 # Node parameters
15 node_cores = 1
16 node_name = null
17 node_memory_model = null
18 nic_name = null
19 # Application parameters
20 sendrecv_message_size = 128

The input file follows a basic syntax of parameter = value. Parameter names follow C++ variable rules (letters, numbers, underscore) while parameter values can contain spaces. Trailing and leading whitespaces are stripped from parameters. Comments can be included on lines starting with #.

The input file is broken into sections via comments. First, application launch parameters must be chosen determining what application will launch, how nodes will be allocated, how ranks will be indexed, and finally what application will be run. Additionally, you must specify how many processes to launch and how many to spawn per node. We currently recommend using aprun syntax (the launcher for Cray machines), although support is being added for other process management systems. SST/macro can simulate command line parameters by giving a value for launch_app1_argv.

A network must also be chosen. In the simplest possible case, the network is modeled via a simple latency/bandwidth formula. For more complicated network models, many more than two parameters will be required. See Network Model for a brief explanation of SST/macro network congestion models. A topology is also needed for constructing the network. In this case we choose a 2-D 4$$4 torus (16 switches). The topology_geometry parameter takes an arbitrarily long list of numbers as the dimensions to the torus.

Finally, we must construct a node model. In this case, again, we use the simplest possible models (null model) for the node, network interface controller (NIC), and memory. The null model is essentially a no-op, generating the correct control flow but not actually simulating any computation. This is useful for validating program correctness or examining questions only related to the network. More accurate (and complicated) models will require parameters for node frequency, memory bandwidth, injection latency, etc.

Parameter files can be constructed in a more modular way through the include statement. An alternative parameter file would be:

1 include machine.ini
2 # Launch parameters
3 launch_indexing = block
4 launch_allocation = first_available
5 launch_app1_cmd = aprun -n2 -N1
6 launch_app1 = user_mpiapp_cxx
7 launch_app1_argv =
8 # Application parameters
9 sendrecv_message_size = 128

where in the first line we include the file machine.ini. All network, topology, and node parameters would be placed into a machine.ini file. In this way, multiple experiments can be linked to a common machine. Alternatively, multiple machines could be linked to the same application by creating and including an application.ini.

Another feature of input files it the ability to use variables. For example, you may wish to have several components all with identical bandwidth. Values can be linked to an input variable through set var.

1 set var mybw = 1.0GB/s
2 network_bandwidth = $mybw
3 memory_bandwidth = $mybw

The bandwidths can now be tuned through a single variable.

Parameter Sweeping with fork()

Often, an experiment consists of sweeping one or more parameters and observing the effect on performance. This can be accomplished by specifying a set of parameter values in brackets, separated by $|$, like so:

1 ...
2 sendrecv_message_size = {128|256|512|1024}
3 multisim_nproc = 2

SST/macro will automatically fork to run each configuration separately, and a file for each configuration will be created with the simulator output. The multisim_nproc parameter controls how many simulations will be run concurrently. If multiple parameters specify ranges, then SST/macro will run every permutation of them. Sometimes only a subset of all permutations is desired, so to address this you can also apply this mechanism to the include directive, like so:

1 include {machine1.ini | machine2.ini}
2 ...
3 sendrecv_message_size = {128|256|512|1024}
4 multisim_nproc = 10

Note that an included parameter file can also have parameter ranges, but this can get out of hand quickly keeping track of everything, so we recommend sticking to the above methods.

The simulator will number each configuration and print this numbering before it starts anything, and ask you to hit enter to continue. If you want to run a single configuration, usually for debugging without having to change the parameter file, you can specify the -r [number] command line option to run just that configuration.