allinea-ddt.md

Allinea Forge (DDT,MAP)
=======================

Allinea Forge consist of two tools - debugger DDT and profiler MAP.

Allinea DDT, is a commercial debugger primarily for debugging parallel MPI or OpenMP programs. It also has a support for GPU (CUDA) and Intel Xeon Phi accelerators. DDT provides all the standard debugging features (stack trace, breakpoints, watches, view variables, threads etc.) for every thread running as part of your program, or for every process - even if these processes are distributed across a cluster using an MPI implementation.

Allinea MAP is a profiler for C/C++/Fortran HPC codes. It is designed for profiling parallel code, which uses pthreads, OpenMP or MPI.

License and Limitations for Anselm Users
----------------------------------------
On Anselm users can debug OpenMP or MPI code that runs up to 64 parallel processes. In case of debugging GPU or Xeon Phi accelerated codes the limit is 8 accelerators. These limitation means that:

-   1 user can debug up 64 processes, or
-   32 users can debug 2 processes, etc.

In case of debugging on accelerators:

-   1 user can debug on up to 8 accelerators, or
-   8 users can debug on single accelerator.

Compiling Code to run with DDT
------------------------------

### Modules

Load all necessary modules to compile the code. For example:

```bash
    $ module load intel
    $ module load impi   ... or ... module load openmpi/X.X.X-icc
```

Load the Allinea DDT module:

```bash
    $ module load Forge
```

Compile the code:

```bash
$ mpicc -g -O0 -o test_debug test.c

$ mpif90 -g -O0 -o test_debug test.f
```

### Compiler flags

Before debugging, you need to compile your code with theses flags:

!!! Note "Note"
	- **g** : Generates extra debugging information usable by GDB. -g3 includes even more debugging information. This option is available for GNU and INTEL C/C++ and Fortran compilers.

	- **O0** : Suppress all optimizations.

Starting a Job with DDT
-----------------------
Be sure to log in with an  X window forwarding enabled. This could mean using the -X in the ssh:

```bash
    $ ssh -X username@anselm.it4i.cz
```

Other options is to access login node using VNC. Please see the detailed information on how to [use graphic user interface on Anselm](https://docs.it4i.cz/anselm-cluster-documentation/software/debuggers/resolveuid/11e53ad0d2fd4c5187537f4baeedff33)

From the login node an interactive session **with X windows forwarding** (-X option) can be started by following command:

```bash
    $ qsub -I -X -A NONE-0-0 -q qexp -lselect=1:ncpus=16:mpiprocs=16,walltime=01:00:00
```

Then launch the debugger with the ddt command followed by the name of the executable to debug:

```bash
    $ ddt test_debug
```

A submission window that appears have a prefilled path to the executable to debug. You can select the number of MPI processors and/or OpenMP threads on which to run and press run. Command line arguments to a program can be entered to the "Arguments " box.

![](../../../img/ddt1.png)

To start the debugging directly without the submission window, user can specify the debugging and execution parameters from the command line. For example the number of MPI processes is set by option "-np 4". Skipping the dialog is done by "-start" option. To see the list of the "ddt" command line parameters, run "ddt --help".

```bash
    ddt -start -np 4 ./hello_debug_impi
```

Documentation
-------------
Users can find original User Guide after loading the DDT module:

```bash
    $DDTPATH/doc/userguide.pdf
```

[1] Discipline, Magic, Inspiration and Science: Best Practice Debugging with Allinea DDT, Workshop conducted at LLNL by Allinea on May 10, 2013, [link](https://computing.llnl.gov/tutorials/allineaDDT/index.html)