Getting Started

This page illustrate you how you can download, compile, install and use ROOT-Sim. It is organized in several sections:

Download and Installation Guide
Usage Guide
Writing your First Model
ROOT-Sim Libraries

Download and Installation Guide

ROOT-Sim can be downloaded cloning the official repository, or downloading a prepackaged tarball containing the distributed source code. To compile the library, a C11 compiler is required.

If you decide to clone the repository, there are two main branches of interest: the master branch contains the latest release version, while the develop branch contains the latest development version. At the time of this writing, the latest stable version is 2.0.0.

To clone the repository, which is hosted on GitHub, you can issue the following command:

git checkout https://github.com/HPDCS/ROOT-Sim

Installation Guide

If you have downloaded a tarball, you will find in the main folder a configure script. This script will check the availability of the required dependencies on your machine, and generate a Makefile. To compile the library, run the following commands:

./configure
make
make install

This will install the library system-wide in your machine.

There are several useful options which can be passed to configure, depending on how you want ROOT-Sim to be compiled and installed. Some of these options relate to specific subsystems which are described in other sections. The important options are:

--prefix: by default, make install will install all the files in /usr/local/bin, /usr/local/lib etc. You can specify an installation prefix other than /usr/local using --prefix, for instance --prefix=$HOME. This is particularly useful if you are installing ROOT-Sim on a machine for which you do not have administrator privileges. Be sure to include prefix/bin in your path, otherwise you will not have direct access to the rootsim-cc compiler which is required to generate ROOT-Sim compatible simulation models.
--enable-mpi: if you intend to run simulation models on a cluster, ROOT-Sim can do it transparently by relying on MPI. --enable-mpi instructs configure to look for an available MPI compiler (such as mpicc) to generate the distributed code. We do not require a particular implementation of MPI to be used, although ROOT-Sim has been efficiently used relying on OpenMPI and MPICH. We rely on advanced asynchronous/multithreaded facilities, so an environment compliant with MPI 3.0 specification should be used.
--enable-modules: ROOT-Sim offers advanced facilities when running on Linux. To this end, ad-hoc kernel modules must be compiled and installed. This flag tells configure to look for kernel headers and build the modules if they are available.
--enable-ecs: by definition of PDES, a simulation model can describe interactions across different LPs only by means of message passing. Due to the distributed and optimistic nature of ROOT-Sim, this has the disadvantage that a simulation model is not allowed to pass pointers to the simulation state of any LP in a message, rather all the information should be packed (or linearized) in a buffer for transmission. If the model passes a pointer, the simulation will either crash or generate undefined results. Event Cross State (ECS) is an advanced feature of ROOT-Sim which allows to pass pointers in messages, ensuring correctness also in a distributed environment. To enable ECS, kernel modules are required.
--disable-rebinding: to maximize the performance of the simulation, the runtime environment periodically checks whether the workload on all the available threads in a node is even. If it is not, LPs are moved (rebound) across the different worker threads. This flag disables this feature.
--enable-debug: this flag instructs configure to generate debug symbols and perform extra checks on the source code. Simulations run when this flag has been used in compile mode can be as much as 20x slower.
--enable-coverage: this flag enables code coverage. This is used mostly in development, to see whether the test cases are covering the various possible execution paths in a suitable way.
--enable-extra-checks: this flag tells configure to generate additional checks on the runtime dynamics of the models. This option kills performance drastically, but it can be used to debug models when strange bugs appear, which could be related to the speculative nature of ROOT-Sim. As an example, consistency of messages is always checked at any access, or it is checked whether an event handler has altered some of the internal data structures related to events, which is forbidden in a speculative simulation.
--enable-profile: this flag, which is used in development, relies on gprof to generate performance reports of the internal routines of ROOT-Sim.

These are the requirements to build ROOT-Sim. Note that if you exclude specific subsystems, you can ignore the dependency.

Dependency	Minimum Version
Linux	2.6.5
gcc	6.0.0
OpenMPI(*)	3.0.1
MPICH(*)	3.2.0

(*) Only one between OpenMPI and MPICH are required.

If you have cloned the repository, you will not find the configure script, as this has to be generated locally. To this end, we provide the autogen.sh script which automatically generates configure for you. This script depends on automake, autoconf, and libtool, so they must be installed in your machine.

Usage Guide

To run a simulation model, first it must be compiled in a way which is suitable for the program to interact correctly with the ROOT-Sim runtime environment. ROOT-Sim significantly mangles the code generated by standard compilers, so generating a ROOT-Sim aware executable could be tedious. We provide rootsim-cc, a wrapper of the standard C/MPI compiler, to automatically perform all the required compilation steps.

Currently, rootsim-cc is only able to rely on gcc or on a MPI compiler using in its turn gcc.

Using the compiler

To compile a model, you can simply use the rootsim-cc compiler as you would do with a standard compiler. All flags are passed to rootsim-cc are passed to the backend compiler. After several compilation steps, mangling of the code, and incremental linking of the various libraries forming up the ROOT-Sim runtime environment, an executable is generated.

A standard Makefile to automatize the compilation process can be used. The following is an example which is used also for the example models which are provided in the source tree. This Makefile is intended for debugging purposes, as it generates debug symbols (-g) and performs extra checks on the source code (-Wall -Wextra). The output is an executable called model which is ROOT-Sim aware.

CC = rootsim-cc
CFLAGS = -g -Wall -Wextra
TARGET = model

SRCS = model.c general.c agent.c
OBJS = $(SRCS:.c=.o)

.PHONY: clean

all: $(TARGET)

$(TARGET): $(OBJS)
	$(CC) $(CFLAGS) -o $(TARGET) $(OBJS)

.c.o:
	$(CC) $(CFLAGS) $(INCLUDES) -c $<  -o $@

clean:
	$(RM) *.o $(TARGET)

Running on a Single Node

As it will be discussed later, a simulation model developed for ROOT-Sim only has to implement event handlers: all the runtime support is provided by the library, including the main function. Therefore, it is possible to pass to the executable generated by rootsim-cc many configuration flags to tell the runtime environment how to support the execution. These are the runtime flags which are understood by the runtime environment:

wt: This option accepts a numerical value which corresponds to the number of worker threads which are used on the node. At simulation startup, the runtime environment spawns this number of threads which take care of scheduling LPs and events, according to the specified scheduling policy. It is not possible, for performance reasons, to set this value to a number higher that the available core count on a node.
lp: Number of logical processes to be used in the simulation, across all threads (or across all compute nodes, if running in distributed). Legal values are in the range [1, 65536].
output-dir: This specifies the name of the folder where runtime statistics are dumped. This defaults to outputs.
scheduler: This option specifies what scheduler should be used to determine the next LP to be activated on a worker thread. Legal values are stf (for Smallest Timestamp First, an O(n) scheduler), and star (an O(1)) scheduler. The rule of thumb to choose the scheduling strategy is that if many LPs are handled by a single worker thread, the star scheduler is expected to behave more efficiently. If few LPs are on a single node, the reduced overhead of the stf scheduler might provide better results.
npwd: “Non piece-wise deterministic execution”. This option tells the simulation engine that there is the possibility that replaying an event might lead to different results (note that this is never related to statistical samples drawn by the ROOT-Sim library). If this is the case, this option forces the runtime environment to take a checkpoint after the execution of every simulation event. While this hampers performance, it leads to correct results in that case. This is equivalent to using --p 1.
p: Checkpointing interval. This tells ROOT-Sim to take a snapshot of the simulation state after this number of events. The default value is 10.
full: Full checkpointing. Checkpoints taken by ROOT-Sim are full. It cannot be used in conjunction with --inc. This is the default checkpointing strategy.
inc: Incremental checkpointing. Most of the checkpoints are incremental (periodically, for performance reasons, a full checkpoint is taken anyhow). The performance of executing an event might increase, so this should be used only when it is foreseeable that simulation states are large, but seldom updated (or only small portions of the state are updated). It cannot be used in conjunction with --full.
A: “Autonomic checkpointing”. A runtime performance model is periodically evaluated, telling the runtime library what is the best checkpointing interval, and whether to switch across incremental or full checkpointing. This feature is disabled by default.
gvt: The time period to wait before a new GVT reduction is performed (in msec). The legal range is [500, 5000], the default is 1000.
cktrm-mode: ROOT-Sim offers the possibility to ask the simulation model to determine (by inspecting the simulation state) whether a simulation should be considered completed or not. Two different termination detection modes are allowed: normal and incremental. normal asks every LP in the run to check its state every time a new GVT value is computed (see the option gvt-snapshot-cycles for further information). When using incremental, on the other hand, if a LP told the runtime that it considers the simulation as completed, it will never be queried again in the run. While this allows to reduce the overhead of termination detection, it could be possible that if the termination condition is fluctuating, the correct termination instant is lost, leading to incorrect results. Therefore, incremental should only be used when it is true that a LP which determined that the simulation can be halted, will never change its mind.
gvt-snapshot-cycles: Number of consecutive GVT calculations before rebuilding a state for temination detection. There is no limit on this value, provided that it is non-negative. The default is 2. Consider that, the higher is this number, the longer the user might wait before the simulator notices that the simulation can be stopped.
simulation-time: If the simulation should be halted after a certain amount of simulation time, this value can be passed to this option. A value of zero (the default) means infinity, therefore to halt a simulation LPs must agree on a different condition, specified in the model.
lps-distribution: LPs distribution across worker threads (and nodes, if running in distributed). Legal values are block (the number of LPs is divided across the whole number of worker threads, and this number is assigned in bulk, accounting also for leftovers); circular (a “round robin” policy, in which the first LP is assigned to the first worker thread, the second LP to the second worker thread, and so on). Tinkering with this option might lead to reduced inter-thread and inter-kernel communication, depending on the model, and can affect the performance. The default value is block.
deterministic-seed: Every time that a user runs a model, a new pseudo-random seed is generated. This allows to simulate slightly different scenarios across the runs. If this behaviour is not wanted, namely we want all runs to use the very same pseudo-random sequence of numbers, the option --deterministic-seed can be passed to the executable. This option is disabled by default.
verbose: info and debug can be passed to this option. info prints very reduced information statistics on screen. debug dumps a lot of text, describing message interaction and runtime dynamics, to build execution traces.
stats: This option tells what information should be collected at runtime and dumped in the output dir. global only generates a text file describing average information about all worker threads and nodes performance. performance generates a log of the various interesting performance metrics at each GVT computation. lp dumps punctual information for each LP in the run. all is the most verbose statistic level, which dumps all the above.
seed: It is possible to specify an integer which is used to seed the pseudo-random number generator provided by the ROOT-Sim library.
serial: It runs the simulation sequentially, on a single core of a single node. This can be used to perform initial debugging of the model, or to measure the parallel/distributed speedup.
sequential: It is an alias for --serial.
no-core-binding: By default, to reduce interference, ROOT-Sim sticks worker threads to specific cores. This option disables this behaviour. It can be useful when running a sequential simulation, or when running multiple MPI ranks on a single node for debugging purposes.

The minimal amount of parameters to be passed to a model is the number of threads and the number of logical processes, as follows:

./model --wt 1 --lp 16

This runs the simulation model with 16 LPs on a single worker thread. Note that this is fundamentally different from:

./model --sequential --lp 16

Indeed, the sequential scheduler is a completely different subsystem with respect to ROOT-Sim’s speculative capabilities. The latter command schedules events using a Calendar Queue, which is expected to be much more optimized in a sequential run. The two commands can be used to appreciate the performance overhead induced by the speculative infrastructure, if any.

Running Distributed

ROOT-Sim ultimately relies on MPI for distributed processing. If MPI support has been compiled in the library, a distributed run can be started as:

mpiexec -n 2 --hostfile hosts --map-by node ./model --wt 2 --lp 16

This tells the MPI runtime installed on the machine and used to compile ROOT-Sim that we want to use to compute nodes (-n 2) which can be reached using the information provided in the hosts file. --map-by node ensures that an even amount of computing resources are taken from both nodes.

Any option which is legal for MPI can be passed to mpiexec.

Debugging the core

If you want to debug the core library, you can use gdb attaching to any model compiled against ROOT-Sim. Please note that you must configure ROOT-Sim passing to configure the --enable-debug flag, which disables optimizations and generates debug symbols.

Debugging on a single node, therefore, can be done with the following command:

gdb --args ./model --wt 2 --lp 2

setting the number of worker threads (--wt) and the number of LPs (--lp) to any suitable value.

If you need to debug distributed runs, possibly to find out problems in distributed algorithms, this is a bit trickier. The problem is that you cannot simply run gdb on mpiexec, as you would debug mpiexec rather than the model. Therefore, these are the common steps for debugging the distributed version:

launch the model
get the pid of the model (e.g., by running pidof model in a shell) on each node where there is an instance of the distributed deploy running
launch gdb on each node where there is an instance of the distributed deploy running
attach to the process using attach PID
look at all debugger instances to see what’s going on.

The problem here is that doing all these steps takes time, and the likelihood that you miss the bug is almost 100%. ROOT-Sim, when compiled using --enable-debug at configure time, looks for the presence of the environment variable WGDB (wait for gdb). If this variable is set, it enters an infinite loop right after having entered main(), giving you the time to debug everything that is going on since the simulation startup. Additionally, this global variable prints on screen the PID of the process, to speedup the process of attaching with the debugger.Therefore, to debug the distributed version of the simulation model, you can launch it as:

WGDB=1 mpiexec -n 2 ./model --wt 2 --lp 4

Please note that when you attach to the PIDs in gdb, they are spinning in an infinite loop implemented like this:

if((getenv("WGDB")) != NULL && *(getenv("WGDB")) == '1') {
    ...
    while (i == __wait) {
        sleep(5);
    }
}

Therefore, there is a high likelyhood that you will attach the debugger when it is running the sleep() function.

To continue the execution, you have to issue the following commands in gdb:

(gdb) up
(gdb) set var __wait = 1
(gdb) continue

You might need to issue more than one up function, until you reach the main function.

Writing your first model

The ROOT-Sim Programming Model

ROOT-Sim exposes a reduced-size set of APIs, which can be easily used to build complex simulation models. In particular, there are four main functions which are exposed to simulation model developers. Two of them are callbacks (ProcessEvent() and OnGVT()), which are used to implement event handlers and inspect a committed simulation state, respectively. The third API function is ScheduleNewEvent(), which allows to inject new events into the system, destined to any LP. The fourth, SetState(), allows the simulation model developer to change the simulation state of an LP at any time.

Here we present a quick overview on ROOT-Sim API and on how a simple simulation model can be implemented in C. Everything exposed by ROOT-Sim is defined in the header ROOT-Sim.h, which is installed system-wide (or on a particular target when using --prefix during the configuration phase) when running make. The rootsim-cc compiler is generated automatically to know the location of the installation of the headers, so it will tell the backend compiler where to find this header.

`ProcessEvent()`

ProcessEvent() is the main application-level callback. A simulation model must implement this function to specify the logic associated with the event handlers. ProcessEvent is the sole entry point at application level which is used to schedule the actual events to be simulated. Therefore, it can be seen as the demultiplexer of the various event handlers which should be implemented in the simulation mode. Its full signature is:

void ProcessEvent(int me, simtime_t now, unsigned int event_type, void *content, unsigned int size, void *state);

The meaning of the arguments passed to this callback is:

me: the global id associated with the LP which is being scheduled for event execution. This is automatically assigned by the runtime environment, and is in the range [0, n_prc_tot-1], where n_prc_tot is the value passed at command line using the --nprc flag. The simulation model developer can assume that this exact number of LPs is available in the system, which can be either scheduled by the runtime environment through a call to ProcessEvent(), or to which a new simulation event can be fired, using the ScheduleNewEvent() API function.
now: the current value of the logical time of the LP. This is consistently and transparently managed by the runtime environment. simtime_t is also defined in ROOT-Sim.h. This value, if necessary, can be used as a double, e.g. to compute time intervals or for logging.
event: the numerical code determining the type of the event to be processed. Except for the special INIT event (see below), these values can be safely chosen by the model developer to best suit their needs.
content: the buffer where the event payload will be delivered by the ROOT-Sim kernel (may be NULL in case the event has no payload). Be careful! This buffer must be accessed in read-only mode by the model. Writing to that buffer might yield to undefined behaviour due to the nature of optimistic simulation. Please make your own copy of that buffer’s content if you wish to operate in write-mode on it. The --extra-check configuration flag runs some hashing before and after event execution on this buffer, to be sure that the modeler does not inadverently alter its content (this can be used for debugging your model).
size: the size (in bytes) of the event payload. It is zero if content is set to NULL.
state: the pointer to the top data structure forming the simulation state layout. This is decided by the simulation model’s code, and can be changed at any time.

Upon initialization, ROOT-Sim schedules the special INIT event (with numerical code 0) once to each LP. This means both that the code should handle this event in ProcessEvent(), and that an event with id 0 cannot be used by the application-level code. INIT is defined in ROOT-Sim.h. The purpose of this event is to allow LPs to perform initialization operations (such as allocating space for their states).

A simulation object is not dispatched again, unless a real application-level event is scheduled for it during the simulation run.

`ScheduleNewEvent()`

ScheduleNewEvent() allows the simulation model to generate a new event and inject it into the system, destined at any LP (even itself). Its full signature is:

void ScheduleNewEvent(unsigned int receiver, simtime_t timestamp, unsigned int event, void *content, unsigned int size);

The arguments passed to this function are:

receiver: the global id of the logical processes where the simulation event must be delivered to. This should be in the interval [0, n_prc_tot - 1].
timestamp: the logical time when the recipient of the event must execute it. This makes the simulation time of the receiver advance exactly to that simulation time, once the event is executed. Its value can never be smaller that the value of now passed to ProcessEvent(), as this would make the future affect the past, which is impossible.
event: the numerical code for the event to be injected into the system. This is model-defined, and causes the activation of the corresponding event handler at the recipient.
content: the pointer to the buffer maintaining the application-defined event payload.
size: the size (in bytes) of the event payload.

`OnGVT()`

OnGVT is an application-level callback. All models must implement this function. By using this callback, the runtime environment enforces a (distributed) termination detection procedure. When the GVT is reduced, all LPs are asked whether the simulation (at that particular LP) can be considered as completed. In case that all LPs reply positively, the simulation is halted. Its full signature is:

bool OnGVT(unsigned int me, void *snapshot);

The arguments passed to this callback are:

me: the global id associated to the LP which is being scheduled for termination detection.
snapshot: a consistent simulation state, associated with the GVT value, which can be used by the LP to decide whether the simulation can terminate or not.

When running an optimistic simulation, the state to be inspected is one which can be associated with a timestamp significantly smaller than the current one reached on the speculative boundary. It is therefore meaningless (and unsafe) to alter the content of this state. Similarly, the model cannot send any new event during the execution of OnGVT().

The distributed termination detection can be executed in a normal or incremental fashion. Depending on the cktrm_mode runtime parameter, the platform can be instructed to ask all the LPs if they want to halt the simulation every time the GVT is computed, or if an LP should be excluded from the check once it has replied in a positive way.

This difference can be useful to enhance the simulation performance when dealing with models which can have an oscillating termination condition (i.e., there is a certain phase of simulation where a simulation object wants to terminate, and a subsequent phase where it no longer wants to) or a monotone termination condition (i.e., when a process decides to terminate, it will never change its mind), respectively.

Nevertheless, the same implementation for the termination check can be used in both ways, so that the OnGVT function can be left untouched.

OnGVT() is given a consistent simulation snapshot on a periodic frequency. Therefore, if the simulation model wants to dump on file some statistics, in this function this task can be correctly implemented.

`SetState()`

By definition of the programming model, the simulation state is located into malloc‘d memory, and the platform silently and transparently restores it to previous checkpoint, whenever a rollback operation is performed, due to an inconsistency in the simulation caused by an out-of-order execution of events.

This means that the runtime environment must be aware of the location of the base pointer to the simulation state. This buffer can, at any time, be changed by the model. The programmer must issue a call to SetState(), in order to inform the runtime environment of the user’s will to use a certain memory buffer as the main simulation state for the Logical Process which is currently running. This allows the runtime to correctly track changes in the objects’ states, in order to correctly perform rollback operations, if needed. The full signature of this function is:

void SetState(void *new_state);

new_state is a pointer to the base structure of the simulation state. This structure can then keep any other internal pointer to malloc‘d memory, which is transparently checkpointed by the runtime environment.

Please note that this function can be considered only “syntactic sugar”. Indeed, all buffers allocated via a malloc call by the model are checkpointed and restored transparently. SetState() only tells ROOT-Sim what is the value to be passed to ProcessEvent() as the state parameter, to simplify the development of the model. This value can change at any time, and the runtime environment transparently rolls back its content in case of a rollback operation, ensuring that the value of state passed to ProcessEvent() is always consistent.

A minimal example

We provide here the source of a minimal functioning ROOT-Sim simulation model. In this example, one single event is specified, and a state structure containing an event counter is defined. Each LP in the simulation model schedules an event to a random process according to a Uniform distribution, and the times associated with the events are determined according to the same distribution.

#include <ROOT-Sim.h>
#define EVENT 1
#define TOTAL_NUMBER_OF_EVENTS 1000000

typedef struct _state_type {
    int executed_events;
} state_type;

void ProcessEvent(unsigned int me, simtime_t now, unsigned int event, void *content, int size, state_type *state)
{
    simtime_t timestamp = now + 10 * Random();
    unsigned int receiver = (unsigned int)(n_prc_tot * Random());
    
    switch(event_type) {
        case INIT:
            state = malloc(sizeof(state_type));
            SetState(state);
            state->executed_events = 0;
            ScheduleNewEvent(me, timestamp, EVENT, NULL, 0);
            break;
            
        case EVENT:
            state->executed_events++;
            ScheduleNewEvent(me, timestamp, EVENT, NULL, 0);
            break;
    }
}

bool OnGVT(int me, void *snapshot)
{
    if (snapshot->executed_events >= TOTAL_NUMBER_OF_EVENTS)
        return true;
    return false;
}

This block of code can be used as a skeleton to develop every simulation model. Note that the ProcessEvent() callback relies on a swtich/case construct to demultiplex the value of event to implement the various event handlers. A handler of the special INIT event is provided, which allocates the initial simulation state via a malloc call (this will be transparently rolled back, in case of inconsistencies).SetState() is used to notify ROOT-Sim about the base pointer of the simulation state, only during the execution if INIT (the simulation state never changes).

Random() is a function belonging to the numerical library of ROOT-Sim, which will be later described. It is essentially a pseudo-random number generator, with a per-LP seed which is transparently rolled back upon a rollback operation, ensuring repeatability of random variables draws from numerical distributions upon a rollback operation.

The implementation of OnGVT() describes when the simulation should be halted. In particular, each LP has in its own state the counter executed_events which is incremented any time that the EVENT handler is activated. Once this value reaches a certain threshold (TOTAL_NUMBER_OF_EVENTS), the simulation is considered completed. Since OnGVT() is evaluated at each LP, all LPs must have executed at least TOTAL_NUMBER_OF_EVENTS to halt the simulation.

It is important to note the usage of the >= comparison in OnGVT(). For performance reasons, OnGVT() is called periodically (see the gvt-snapshot-cycles runtime option). Therefore, it is possible that an exact value of executed_events is never evaluated. Using a == operator in OnGVT() can be unsafe, leading to simulations to never terminate. This means that, by the definition of OnGVT(), simulation models return “upper values” to the termination conditions of simulation models.

Additional example models are available in the models subfolders of the ROOT-Sim tarball and on the official repository. They can be used as valuable examples to get started.

Passing Parameters to the Models

It is quite common for a simulation model to be configured at runtime, without having to recompile everything. To this end, ROOT-Sim provides a simple facility to provide runtime parameters to simulation models.

The configuration of ROOT-Sim relies on the standard argp library. A simulation model can define some variables which are intercepted by ROOT-Sim at compile time, making the option parser to take into account also model-specified attributes.

The following code example illustrates how it is possible, for a simulation model, to intercept two options, called --opt-A and --opt-B, the first accepting a floating point value as its argument, the second accepting an integer.

double A;
int B;

enum {
	OPT_A = 128, /// this tells argp to not assign short options
	OPT_B,
};

const struct argp_option model_options[] = {
		{"opt-A", OPT_A, "FLOAT", 0, "This is the A option", 0},
		{"opt-B", OPT_B, "INT", 0, "This is the B option", 0},
		{0}
};

#define HANDLE_CASE(label, fmt, var)      \
	case label:                           \
		if(sscanf(arg, fmt, &var) != 1) { \
			return ARGP_ERR_UNKNOWN;      \
		}                                 \
	break

static error_t model_parse (int key, char *arg, struct argp_state *state){
	switch (key) {
		HANDLE_CASE(OPT_A, "%lf", A);
		HANDLE_CASE(OPT_B, "%d", B);
        default:
			return ARGP_ERR_UNKNOWN;
    }
    return 0;
}

#undef HANDLE_CASE

The code declares an enum to define the numerical codes of the options. argp uses values greater than 127 to identify long options, so setting OPT_A to 128 ensures that it is parsed as --opt-A.

The model_options array defines, according to the argp syntax, what are the options which are handled by the model. A textual description can be provided, which is printed on screen if the following command is issued on the command line:

./model --help

HANDLE_CASE is a commodity macro which allows to simplify the development of the switch case in model_parse, which implements the logic associated with option setting. This is transparently invoked by argp. Note that relying on sscanf in HANDLE_CASE might not be the most secure way to implement the code (it is used here for the sake of brevity).

This configuration infrastructure is called well before LPs are initialized and INIT is scheduled. Therefore, it is important to rely on global variables (such as A and B in the example) to store the values passed from command line. Later, during the execution of INIT, these values can be used to properly initialize the simulation model.

ROOT-Sim Libraries

To simplify the development of simulation models according to the speculative PDES paradigm, ROOT-Sim offers a set of libraries which can be used to implement important portions of simulation models, or to automatize tedious tasks.

In this section, we describe the available libraries, the exposed API, and we show some usage examples.

Numerical Library

ROOT-Sim offers a fully-featured numerical library designed according to the Piece-Wise Determinism paradigm. The main idea behind this library is that if a Logical Process incurs into a Rollback, the seed which is associated with the random number generator associated with that LP must be rolled back as well. The numerical library provided by ROOT-Sim transparently does so, while if you rely on a different numerical library, you must implement this feature by hand, if you want that a logical process is always given the same sequence of pseudo-random numbers, even when the execution is restarted from a previous simulation state.

The following functions are available in the ROOT-Sim numerical library. They can be used to draw samples from random distribution, which are commonly used in many simulation models.

`Random()`

This function has the following signature:

double Random(void);

It returns a floating point number in between [0,1], according to a Uniform Distribution.

`RandomRange()`

This function has the following signature:

int RandomRange(int min, int max)

It returns an integer number in between [min,max], according to a Uniform Distribution.

`RandomRangeNonUniform()`

This function has the following signature:

int RandomRangeNonUniform(int x, int min, int max)

It returns an integer number in between [min,max]. The parameter x determines the incremented probability according to which a number is generated in the range, according to the following formula:

(((RandomRange(0, x) | RandomRange(min, max))) % (max - min + 1)) + min

`Expent()`

The signature of this function is:

double Expent(double mean)

It returns a floating point number according to an Exponential Distribution of mean value mean.

`Normal()`

The signature of this function is:

double Normal(void)

It returns a floating point number according to a Normal Distribution with mean zero.

`Gamma()`

The signature of this function is:

double Gamma(int ia)

It returns a floting point number according to a Gamma Distribution of Integer Order ia, i.e. a waiting time to the ia-th event in a Poisson process of unit mean.

`Poisson()`

The signature of this function is:

double Poisson(void)

It returns the waiting time to the next event in a Poisson process of unit mean.

`Zipf()`

The signature of this function is:

int Zipf(double skew, int limit)

It returns a random sample from a Zipf distribution.

Topology Library

Many simulation models rely on a representation of the physical space. ROOT-Sim offers a library which you can use to instantiate discretized topologies with little effort. To enable the library it is sufficient to declare a struct _topology_settings_t variable in your model which has the following members: TODO

struct _topology_settings_t{
    const char * const topology_path;
    const enum _topology_type_t type;
    const enum _topology_geometry_t default_geometry;
    const unsigned out_of_topology;
    const bool write_enabled;
}

`RegionsCount()`

The signature of this function is:

RegionsCount(void)

It returns the number of regions which are part of the instantiated topology. You can identify an LP as a region at runtime if its id is lower than this value. This value will be always equal or lower than n_prc_tot.

`NeighboursCount()`

The signature of this function is:

NeighboursCount(unsigned int region_id)

It returns the number of valid neighbours of the LP with id region_id in the instantiated topology. This only returns the number of geometrically viable regions, e.g. it doesn’t distinguish obstacles from non obstacles regions.

`DirectionsCount()`

The signature of this function is:

DirectionsCount(void)

It returns the number of valid directions in the instantiated topology. This value coincides with the maximum number of neighbours a region can possibly have in the current topology (e.g. DirectionsCount() = 4 in a square topology).

`GetReceiver()`

The signature of this function is:

GetReceiver(unsigned int from, direction_t direction, bool reachable)

It returns the LP id of the neighbour you would reach going from region from along direction direction. The flag reachable is set if you only want the id of a LP considered reachable. In case the function isn’t able to deliver a correct id it returns DIRECTION_INVALID.

`FindReceiver()`

The signature of this function is:

FindReceiver(void)

It returns the id of an LP selected between the neighbours of the LP it is called in. The selection logic works as follows:

if the current topology type is TOPOLOGY_OBSTACLES, a non obstacle neighbour is uniformly sampled. If there’s no suitable neighbours we return the current lp id
if the current topology type is TOPOLOGY_PROBABILITIES, an LP is selected with a probability proportional with the weight of the outgoing topology edge from the current LP
an error is thrown if you try to use this function in a topology with type TOPOLOGY_COSTS since this operation would have little sense.

`FindReceiverToward()`

The signature of this function is:

FindReceiverToward(unsigned int to)

It returns the id of the next LP you have to visit in order to reach the LP with id to with the smallest possible incurred cost. In case there’s no possible route DIRECTION_INVALID is returned. The cost is calculated as follows:

in a topology having type TOPOLOGY_OBSTACLES, the cost is simply the number of hops needed to reach a certain destination
in a topology having type TOPOLOGY_COSTS, the cost is the sum of the weights of the edges traversed in the path needed to reach the destination
the notion of cost has little sense in a TOPOLOGY_PROBABILITIES type topology so an error is thrown if you try to use this function for this purpose.

`ComputeMinTour()`

The signature of this function is:

ComputeMinTour(unsigned int source, unsigned int dest, unsigned int result[RegionsCount()])

It computes the minimum cost directed path from region source to region dest. You have to pass in result a pointer to a memory location with enough space to store RegionsCount() LP id (clearly it’s the longest possible minimum cost path). The returned value is the cost incurred in the path traversal or -1 if it isn’t possible to find a path at all. TODO further details

`SetValueTopology()`

The signature of this function is:

SetValueTopology(unsigned int from, unsigned int to, double value)

TODO

`GetValueTopology()`

The signature of this function is:

GettValueTopology(unsigned int from, unsigned int to)

TODO

Agent-Based Modeling Library

High level description of ABM library, init details TODO

`SpawnAgent()`

The signature of this function is:

SpawnAgent(unsigned user_data_size)

It returns the id of a newly instantiated agent with additional user_data_size bytes where you can carry around your data.

`DataAgent()`

The signature of this function is:

DataAgent(agent_t agent)

It returns a pointer to the user-editable memory area of the agent agent.

`KillAgent()`

The signature of this function is:

KillAgent(agent_t agent)

It destroys the agent agent. Once killed, an agent disappears from the region.

`CountAgents()`

The signature of this function is:

CountAgents(void)

It returns the number of agents in the current region.

`IterAgents()`

The signature of this function is:

IterAgents(agent_t *agent_p)

TODO

`ScheduleNewLeaveEvent()`

The signature of this function is:

ScheduleNewLeaveEvent(simtime_t time, unsigned int event_type, agent_t agent)

It schedules a leave event at the logical time time for the agent agent with code event_type. That will be the last event the agent can witness before moving to another region. TODO

`TrackNeighbourInfo()`

The signature of this function is:

TrackNeighbourInfo(void *neighbour_data)

It sets the pointer to the data you want to publish to the neighbouring regions. Once set, ROOT-Sim runtime transparently will keep updated the published data with the neighbours.

`GetNeighbourInfo()`

The signature of this function is:

GetNeighbourInfo(direction_t i, unsigned int *region_id, void **data_p)

It retrieves the data published by the neighbouring region you would reach going from the current LP along direction i. region_id must point to a variable which will be set to the LP id of the retrieved neighbour, data_p must point to a pointer variable which will point to the requested data. In case of success 0 is returned otherwise there’s no neighbour along the requested direction which published his data and -1 is returned.

`CountVisits()`

The signature of this function is:

CountVisits(const agent_t agent)

It returns the number of visits scheduled in the future for the agent agent.

`GetVisit()`

The signature of this function is:

GetVisit(const agent_t agent, unsigned *region_p, unsigned *event_type_p, unsigned i)

TODO

`SetVisit()`

The signature of this function is:

SetVisit(const agent_t agent, unsigned *region_p, unsigned *event_type_p, unsigned i)

TODO

`EnqueueVisit()`

The signature of this function is:

EnqueueVisit(agent_t agent, unsigned region, unsigned event_type)

TODO

`AddVisit()`

The signature of this function is:

AddVisit(agent_t agent, unsigned region, unsigned event_type, unsigned i)

TODO

`RemoveVisit()`

The signature of this function is:

RemoveVisit(agent_t agent, unsigned i)

TODO

`CountPastVisits()`

The signature of this function is:

CountPastVisits(const agent_t agent)

It returns the number of visits already completed by the agent agent in the past.

`GetPastVisit()`

The signature of this function is:

GetPastVisit(const agent_t agent, unsigned *region_p, unsigned *event_type_p, simtime_t *time_p, unsigned i)

TODO

Getting Started

Download and Installation Guide

Installation Guide

Usage Guide

Using the compiler

Running on a Single Node

Running Distributed

Debugging the core

Writing your first model

The ROOT-Sim Programming Model

ProcessEvent()

ScheduleNewEvent()

OnGVT()

SetState()

A minimal example

Passing Parameters to the Models

ROOT-Sim Libraries

Numerical Library

Random()

RandomRange()

RandomRangeNonUniform()

Expent()

Normal()

Gamma()

Poisson()

Zipf()

Topology Library

RegionsCount()

NeighboursCount()

DirectionsCount()

GetReceiver()

FindReceiver()

FindReceiverToward()

ComputeMinTour()

SetValueTopology()

GetValueTopology()

Agent-Based Modeling Library

SpawnAgent()

DataAgent()

KillAgent()

CountAgents()

IterAgents()

ScheduleNewLeaveEvent()

TrackNeighbourInfo()

GetNeighbourInfo()

CountVisits()

GetVisit()

SetVisit()

EnqueueVisit()

AddVisit()

RemoveVisit()

CountPastVisits()

GetPastVisit()

JSON Parsing Library

`ProcessEvent()`

`ScheduleNewEvent()`

`OnGVT()`

`SetState()`

`Random()`

`RandomRange()`

`RandomRangeNonUniform()`

`Expent()`

`Normal()`

`Gamma()`

`Poisson()`

`Zipf()`

`RegionsCount()`

`NeighboursCount()`

`DirectionsCount()`

`GetReceiver()`

`FindReceiver()`

`FindReceiverToward()`

`ComputeMinTour()`

`SetValueTopology()`

`GetValueTopology()`

`SpawnAgent()`

`DataAgent()`

`KillAgent()`

`CountAgents()`

`IterAgents()`

`ScheduleNewLeaveEvent()`

`TrackNeighbourInfo()`

`GetNeighbourInfo()`

`CountVisits()`

`GetVisit()`

`SetVisit()`

`EnqueueVisit()`

`AddVisit()`

`RemoveVisit()`

`CountPastVisits()`

`GetPastVisit()`