Commit 9957468e authored by Lukáš Krupčík's avatar Lukáš Krupčík

change directory tree

parent 74f28f43
# Bioinformatics Applications
## Introduction
In addition to the many applications available through modules (deployed through EasyBuild packaging system) we provide an alternative source of applications on our clusters inferred from [Gentoo Linux](https://www.gentoo.org/). The user's environment is setup through a script which returns a bash instance to the user (you can think of it a starting a whole virtual machine but inside your current namespace) . The applications were optimized by gcc compiler for the SandyBridge and IvyBridge platforms. The binaries use paths from /apps/gentoo prefix to find the required runtime dependencies, config files, etc. The Gentoo Linux is a standalone installation not even relying on the glibc provided by host operating system (Redhat). The trick which allowed us to install Gentoo Linux on the host Redhat system is called Gentoo::RAP and uses a modified loader with a hardcoded path ([links](https://wiki.gentoo.org/wiki/Prefix/libc)).
## Starting the Environment
```console
mmokrejs@login2~$ /apps/gentoo/startprefix
```
## Starting PBS Jobs Using the Applications
Create a template file which can be used and an argument to qsub command. Notably, the 'PBS -S' line specifies full PATH to the Bourne shell of the Gentoo Linux environment.
```console
mmokrejs@login2~$ cat myjob.pbs
#PBS -S /apps/gentoo/bin/sh
#PBS -l nodes=1:ppn=16,walltime=12:00:00
#PBS -q qfree
#PBS -M my_email@foo.bar
#PBS -m ea
#PBS -N sample22
#PBS -A DD-13-5
#source ~/.bashrc
cd $PBS_O_WORKDIR || exit 255
myscript.sh foo 1>myjob.log 2>&1
$ head -n 1 myscript.sh
#! /apps/gentoo/bin/sh
$ qsub myjob.pbs
$ qstat
```
## Reading Manual Pages for Installed Applications
```console
mmokrejs@login2~$ man -M /apps/gentoo/usr/share/man bwa
mmokrejs@login2~$ man -M /apps/gentoo/usr/share/man samtools
```
## Listing of Bioinformatics Applications
```console
mmokrejs@login2~$ grep biology /scratch/mmokrejs/gentoo_rap/installed.txt
sci-biology/ANGLE-bin-20080813-r1
sci-biology/AlignGraph-9999
sci-biology/Atlas-Link-0.01-r1
sci-biology/BRANCH-9999
sci-biology/EBARDenovo-1.2.2
sci-biology/FLASH-1.2.9
sci-biology/GAL-0.2.2
sci-biology/Gambit-0.4.145
sci-biology/HTSeq-0.6.1
sci-biology/InterMine-0.98
sci-biology/MochiView-1.45
sci-biology/MuSeqBox-5.4
sci-biology/ONTO-PERL-1.41
sci-biology/ORFcor-20130507
sci-biology/Rcorrector-9999
sci-biology/SSAKE-3.8.2
sci-biology/STAR-9999
sci-biology/YASRA-2.33
sci-biology/abacas-1.3.1
sci-biology/align_to_scf-1.06
sci-biology/assembly-stats-9999
sci-biology/bambus-2.33
sci-biology/bamtools-9999
sci-biology/bcftools-1.2
sci-biology/bedtools-2.22.1
sci-biology/bfast-0.7.0a
sci-biology/biobambam2-9999
sci-biology/bismark-0.13.0
sci-biology/blat-34-r1
sci-biology/blue-1.1.3
sci-biology/bowtie-2.2.9
sci-biology/brat-1.2.4
sci-biology/bwa-0.7.13
sci-biology/bx-python-9999
sci-biology/cast-bin-20080813
sci-biology/cd-hit-4.6.5
sci-biology/cdbfasta-0.1
sci-biology/clover-2011.10.24
sci-biology/clustalw-2.1
sci-biology/cnrun-2.0.3
sci-biology/codonw-1.4.4-r2
sci-biology/conform-gt-1174
sci-biology/conifer-0.2.2
sci-biology/coral-1.4
sci-biology/cross_genome-20140822
sci-biology/cutadapt-9999
sci-biology/dawg-1.1.2
sci-biology/dna2pep-1.1
sci-biology/edena-3.131028
sci-biology/epga-9999
sci-biology/erpin-5.5b
sci-biology/estscan-3.0.3
sci-biology/eugene-4.1d
sci-biology/exonerate-gff3-9999
sci-biology/fastx_toolkit-0.0.14
sci-biology/gemini-9999
sci-biology/geneid-1.4.4
sci-biology/genepop-4.2.1
sci-biology/glimmerhmm-3.0.1-r1
sci-biology/gmap-2015.12.31.5
sci-biology/hexamer-19990330
sci-biology/hts-python-9999
sci-biology/jellyfish-2.1.4
sci-biology/jigsaw-3.2.10
sci-biology/kallisto-9999
sci-biology/karect-1.0.0
sci-biology/lastz-1.03.66
sci-biology/libgtextutils-0.6.1
sci-biology/lucy-1.20
sci-biology/megahit-9999
sci-biology/merlin-1.1.2
sci-biology/miranda-3.3a
sci-biology/mreps-2.5
sci-biology/mrfast-2.6.0.1
sci-biology/mummer-3.22-r1
sci-biology/muscle-3.8.31
sci-biology/nrcl-110625
sci-biology/nwalign-0.3.1
sci-biology/oases-9999
sci-biology/parafly-20130121
sci-biology/phrap-1.080812-r1
sci-biology/phred-071220
sci-biology/phylip-3.696-r1
sci-biology/plinkseq-0.10
sci-biology/primer3-2.3.7
sci-biology/prinseq-lite-0.20.4
sci-biology/proda-1.0
sci-biology/pybedtools-0.6.9
sci-biology/pysam-0.9.0
sci-biology/pysamstats-0.24.2
sci-biology/quast-2.3
sci-biology/quorum-1.0.0
sci-biology/reaper-15348
sci-biology/repeatmasker-libraries-20150807
sci-biology/reptile-1.1
sci-biology/samstat-20130708
sci-biology/samtools-0.1.20-r2
sci-biology/samtools-1.3-r1
sci-biology/scaffold_builder-20131122-r1
sci-biology/scan_for_matches-20121220
sci-biology/screed-0.7.1
sci-biology/scythe-0.992
sci-biology/seqan-2.1.1
sci-biology/seqtools-4.34.5
sci-biology/sff_dump-1.04
sci-biology/sgp2-1.1
sci-biology/shrimp-2.2.3
sci-biology/sickle-9999
sci-biology/smalt-0.7.6
sci-biology/snpomatic-9999
sci-biology/ssaha2-bin-2.5.5
sci-biology/stampy-1.0.28
sci-biology/stringtie-1.2.2
sci-biology/subread-1.4.6
sci-biology/swissknife-1.72
sci-biology/tagdust-20101028
sci-biology/tclust-110625
sci-biology/tigr-foundation-libs-2.0-r1
sci-biology/trans-abyss-1.4.8
sci-biology/trf-4.07b
sci-biology/uchime-4.2.40
sci-biology/velvet-1.2.10
sci-biology/velvetk-20120606
sci-biology/zmsort-110625
```
```console
mmokrejs@login2~$ grep sci-libs /scratch/mmokrejs/gentoo_rap/installed.txt
sci-libs/amd-2.3.1
sci-libs/blas-reference-20151113-r1
sci-libs/camd-2.3.1
sci-libs/cbflib-0.9.3.3
sci-libs/ccolamd-2.8.0
sci-libs/cholmod-2.1.2
sci-libs/coinor-cbc-2.8.9
sci-libs/coinor-cgl-0.58.6
sci-libs/coinor-clp-1.15.6-r1
sci-libs/coinor-dylp-1.9.4
sci-libs/coinor-osi-0.106.6
sci-libs/coinor-utils-2.9.11
sci-libs/coinor-vol-1.4.4
sci-libs/colamd-2.8.0
sci-libs/cxsparse-3.1.2
sci-libs/dcmtk-3.6.0
sci-libs/gsl-2.1
sci-libs/hdf5-1.8.15_p1
sci-libs/htslib-1.3
sci-libs/io_lib-1.14.7
sci-libs/lapack-reference-3.6.0-r1
sci-libs/lemon-1.3-r2
sci-libs/libmaus2-9999
sci-libs/qrupdate-1.1.2-r1
sci-libs/scikits-0.1-r1
sci-libs/suitesparseconfig-4.2.1
sci-libs/umfpack-5.6.2
```
## Classification of Applications
| Applications for bioinformatics at IT4I | |
| --------------------------------------- | ------ |
| error-correctors | 6 |
| aligners | 20 |
| clusterers | 5 |
| assemblers | 9 |
| scaffolders | 6 |
| motif searching | 6 |
| ORF/gene prediction/genome annotation | 13 |
| genotype/haplotype/popullation genetics | 3 |
| phylogenetics | 1 |
| transcriptome analysis | 2 |
| utilities | 15 |
| GUI | 3 |
| libraries | 4 |
| **Total** | **93** |
![graphs](../img/bio-graphs.png)
## Other Applications Available Through Gentoo Linux
Gentoo Linux is a allows compilation of its applications from source code while using compiler and optimize flags set to user's wish. This facilitates creation of optimized binaries for the host platform. Users maybe also use several versions of gcc, python and other tools.
```console
mmokrejs@login2~$ gcc-config -l
mmokrejs@login2~$ java-config -L
mmokrejs@login2~$ eselect
```
# COMSOL Multiphysics
## Introduction
[COMSOL](http://www.comsol.com) is a powerful environment for modelling and solving various engineering and scientific problems based on partial differential equations. COMSOL is designed to solve coupled or multiphysics phenomena. For many standard engineering problems COMSOL provides add-on products such as electrical, mechanical, fluid flow, and chemical applications.
* [Structural Mechanics Module](http://www.comsol.com/structural-mechanics-module),
* [Heat Transfer Module](http://www.comsol.com/heat-transfer-module),
* [CFD Module](http://www.comsol.com/cfd-module),
* [Acoustics Module](http://www.comsol.com/acoustics-module),
* and [many others](http://www.comsol.com/products)
COMSOL also allows an interface support for equation-based modelling of partial differential equations.
## Execution
On the clusters COMSOL is available in the latest stable version. There are two variants of the release:
* **Non commercial** or so called **EDU variant**, which can be used for research and educational purposes.
* **Commercial** or so called **COM variant**, which can used also for commercial activities. **COM variant** has only subset of features compared to the **EDU variant** available. More about licensing [here](licensing-and-available-versions/).
To load the of COMSOL load the module
```console
$ ml COMSOL
```
By default the **EDU variant** will be loaded. If user needs other version or variant, load the particular version. To obtain the list of available versions use
```console
$ ml av COMSOL
```
If user needs to prepare COMSOL jobs in the interactive mode it is recommend to use COMSOL on the compute nodes via PBS Pro scheduler. In order run the COMSOL Desktop GUI on Windows is recommended to use the [Virtual Network Computing (VNC)](../../general/accessing-the-clusters/graphical-user-interface/x-window-system/).
Example for Salomon:
```console
$ xhost +
$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ppn=24
$ ml COMSOL
$ comsol
```
To run COMSOL in batch mode, without the COMSOL Desktop GUI environment, user can utilized the default (comsol.pbs) job script and execute it via the qsub command.
```bash
#!/bin/bash
#PBS -l select=3:ppn=24
#PBS -q qprod
#PBS -N JOB_NAME
#PBS -A PROJECT_ID
cd /scratch/work/user/$USER/ || exit # on Anselm use: /scratch/$USER
echo Time is `date`
echo Directory is `pwd`
echo '**PBS_NODEFILE***START*******'
cat $PBS_NODEFILE
echo '**PBS_NODEFILE***END*********'
text_nodes < cat $PBS_NODEFILE
module load COMSOL
# module load COMSOL/51-EDU
ntask=$(wc -l $PBS_NODEFILE)
comsol -nn ${ntask} batch -configuration /tmp –mpiarg –rmk –mpiarg pbs -tmpdir /scratch/.../$USER/ -inputfile name_input_f.mph -outputfile name_output_f.mph -batchlog name_log_f.log
```
Working directory has to be created before sending the (comsol.pbs) job script into the queue. Input file (name_input_f.mph) has to be in working directory or full path to input file has to be specified. The appropriate path to the temp directory of the job has to be set by command option (-tmpdir).
## LiveLink for MATLAB
COMSOL is the software package for the numerical solution of the partial differential equations. LiveLink for MATLAB allows connection to the COMSOL API (Application Programming Interface) with the benefits of the programming language and computing environment of the MATLAB.
LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On the clusters 1 commercial (**COM**) license and the 5 educational (**EDU**) licenses of LiveLink for MATLAB (please see the [ISV Licenses](../isv_licenses/)) are available. Following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode (on Anselm use 16 threads).
```console
$ xhost +
$ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ppn=24
$ ml MATLAB
$ ml COMSOL
$ comsol server MATLAB
```
At the first time to launch the LiveLink for MATLAB (client-MATLAB/server-COMSOL connection) the login and password is requested and this information is not requested again.
To run LiveLink for MATLAB in batch mode with (comsol_matlab.pbs) job script you can utilize/modify the following script and execute it via the qsub command.
```bash
#!/bin/bash
#PBS -l select=3:ppn=24
#PBS -q qprod
#PBS -N JOB_NAME
#PBS -A PROJECT_ID
cd /scratch/work/user/$USER || exit # on Anselm use: /scratch/$USER
echo Time is `date`
echo Directory is `pwd`
echo '**PBS_NODEFILE***START*******'
cat $PBS_NODEFILE
echo '**PBS_NODEFILE***END*********'
text_nodes < cat $PBS_NODEFILE
module load MATLAB
module load COMSOL/51-EDU
ntask=$(wc -l $PBS_NODEFILE)
comsol -nn ${ntask} server -configuration /tmp -mpiarg -rmk -mpiarg pbs -tmpdir /scratch/work/user/$USER/work &
cd /apps/cae/COMSOL/51/mli
matlab -nodesktop -nosplash -r "mphstart; addpath /scratch/work/user/$USER/work; test_job"
```
This example shows how to run Livelink for MATLAB with following configuration: 3 nodes and 24 cores per node. Working directory has to be created before submitting (comsol_matlab.pbs) job script into the queue. Input file (test_job.m) has to be in working directory or full path to input file has to be specified. The Matlab command option (-r ”mphstart”) created a connection with a COMSOL server using the default port number.
# Licensing and Available Versions
## Comsol Licence Can Be Used By:
* all persons in the carrying out of the CE IT4Innovations Project (In addition to the primary licensee, which is VSB - Technical University of Ostrava, users are CE IT4Innovations third parties - CE IT4Innovations project partners, particularly the University of Ostrava, the Brno University of Technology - Faculty of Informatics, the Silesian University in Opava, Institute of Geonics AS CR.)
* all persons who have a valid license
* students of the Technical University
## Comsol EDU Network Licence
The licence intended to be used for science and research, publications, students’ projects, teaching (academic licence).
## Comsol COM Network Licence
The licence intended to be used for science and research, publications, students’ projects, commercial research with no commercial use restrictions. Enables the solution of at least one job by one user in one program start.
## Available Versions
* ver. 51
# GPI-2
## Introduction
Programming Next Generation Supercomputers: GPI-2 is an API library for asynchronous interprocess, cross-node communication. It provides a flexible, scalable and fault tolerant interface for parallel applications.
The GPI-2 library ([www.gpi-site.com/gpi2/](http://www.gpi-site.com/gpi2/)) implements the GASPI specification (Global Address Space Programming Interface, [www.gaspi.de](http://www.gaspi.de/en/project.html)). GASPI is a Partitioned Global Address Space (PGAS) API. It aims at scalable, flexible and failure tolerant computing in massively parallel environments.
## Modules
The GPI-2, version 1.0.2 is available on Anselm via module gpi2:
```console
$ ml gpi2
$ ml av GPI-2 # Salomon
```
The module sets up environment variables, required for linking and running GPI-2 enabled applications. This particular command loads the default module, which is gpi2/1.0.2
## Linking
!!! note
Link with -lGPI2 -libverbs
Load the gpi2 module. Link using **-lGPI2** and **-libverbs** switches to link your code against GPI-2. The GPI-2 requires the OFED infinband communication library ibverbs.
### Compiling and Linking With Intel Compilers
```console
$ ml intel
$ ml gpi2
$ icc myprog.c -o myprog.x -Wl,-rpath=$LIBRARY_PATH -lGPI2 -libverbs
```
### Compiling and Linking With GNU Compilers
```console
$ ml gcc
$ ml gpi2
$ gcc myprog.c -o myprog.x -Wl,-rpath=$LIBRARY_PATH -lGPI2 -libverbs
```
## Running the GPI-2 Codes
!!! note
gaspi_run starts the GPI-2 application
The gaspi_run utility is used to start and run GPI-2 applications:
```console
$ gaspi_run -m machinefile ./myprog.x
```
A machine file (** machinefile **) with the hostnames of nodes where the application will run, must be provided. The machinefile lists all nodes on which to run, one entry per node per process. This file may be hand created or obtained from standard $PBS_NODEFILE:
```console
$ cut -f1 -d"." $PBS_NODEFILE > machinefile
```
machinefile:
```console
cn79
cn80
```
This machinefile will run 2 GPI-2 processes, one on node cn79 other on node cn80.
machinefle:
```console
cn79
cn79
cn80
cn80
```
This machinefile will run 4 GPI-2 processes, 2 on node cn79 o 2 on node cn80.
!!! note
Use the **mpiprocs**to control how many GPI-2 processes will run per node
Example:
```console
$ qsub -A OPEN-0-0 -q qexp -l select=2:ncpus=16:mpiprocs=16 -I
```
This example will produce $PBS_NODEFILE with 16 entries per node.
### Gaspi_logger
!!! note
gaspi_logger views the output form GPI-2 application ranks
The gaspi_logger utility is used to view the output from all nodes except the master node (rank 0). The gaspi_logger is started, on another session, on the master node - the node where the gaspi_run is executed. The output of the application, when called with gaspi_printf(), will be redirected to the gaspi_logger. Other I/O routines (e.g. printf) will not.
## Example
Following is an example GPI-2 enabled code:
```cpp
#include <GASPI.h>
#include <stdlib.h>
void success_or_exit ( const char* file, const int line, const int ec)
{
if (ec != GASPI_SUCCESS)
{
gaspi_printf ("Assertion failed in %s[%i]:%dn", file, line, ec);
exit (1);
}
}
#define ASSERT(ec) success_or_exit (__FILE__, __LINE__, ec);
int main(int argc, char *argv[])
{
gaspi_rank_t rank, num;
gaspi_return_t ret;
/* Initialize GPI-2 */
ASSERT( gaspi_proc_init(GASPI_BLOCK) );
/* Get ranks information */
ASSERT( gaspi_proc_rank(&rank) );
ASSERT( gaspi_proc_num(&num) );
gaspi_printf("Hello from rank %d of %dn",
rank, num);
/* Terminate */
ASSERT( gaspi_proc_term(GASPI_BLOCK) );
return 0;
}
```
Load modules and compile:
```console
$ ml gcc gpi2
$ gcc helloworld_gpi.c -o helloworld_gpi.x -Wl,-rpath=$LIBRARY_PATH -lGPI2 -libverbs
```
Submit the job and run the GPI-2 application
```console
$ qsub -q qexp -l select=2:ncpus=1:mpiprocs=1,place=scatter,walltime=00:05:00 -I
qsub: waiting for job 171247.dm2 to start
qsub: job 171247.dm2 ready
cn79 $ ml gpi2
cn79 $ cut -f1 -d"." $PBS_NODEFILE > machinefile
cn79 $ gaspi_run -m machinefile ./helloworld_gpi.x
Hello from rank 0 of 2
```
At the same time, in another session, you may start the gaspi logger:
```console
$ ssh cn79
cn79 $ gaspi_logger
GASPI Logger (v1.1)
[cn80:0] Hello from rank 1 of 2
```
In this example, we compile the helloworld_gpi.c code using the **gnu compiler**(gcc) and link it to the GPI-2 and ibverbs library. The library search path is compiled in. For execution, we use the qexp queue, 2 nodes 1 core each. The GPI module must be loaded on the master compute node (in this example the cn79), gaspi_logger is used from different session to view the output of the second process.
# Intel Advisor
is tool aiming to assist you in vectorization and threading of your code. You can use it to profile your application and identify loops, that could benefit from vectorization and/or threading parallelism.
## Installed Versions
The following versions are currently available on Salomon as modules:
2016 Update 2 - Advisor/2016_update2
## Usage
Your program should be compiled with -g switch to include symbol names. You should compile with -O2 or higher to see code that is already vectorized by the compiler.
Profiling is possible either directly from the GUI, or from command line.
To profile from GUI, launch Advisor:
```console
$ advixe-gui
```
Then select menu File -> New -> Project. Choose a directory to save project data to. After clicking OK, Project properties window will appear, where you can configure path to your binary, launch arguments, working directory etc. After clicking OK, the project is ready.
In the left pane, you can switch between Vectorization and Threading workflows. Each has several possible steps which you can execute by clicking Collect button. Alternatively, you can click on Command Line, to see the command line required to run the analysis directly from command line.
## References
1. [Intel® Advisor 2015 Tutorial: Find Where to Add Parallelism - C++ Sample](https://software.intel.com/en-us/intel-advisor-tutorial-vectorization-windows-cplusplus)
1. [Product page](https://software.intel.com/en-us/intel-advisor-xe)
1. [Documentation](https://software.intel.com/en-us/intel-advisor-2016-user-guide-linux)
# Intel Compilers
The Intel compilers in multiple versions are available, via module intel. The compilers include the icc C and C++ compiler and the ifort fortran 77/90/95 compiler.
```console
$ ml intel
$ icc -v
$ ifort -v
```
The intel compilers provide for vectorization of the code, via the AVX2 instructions and support threading parallelization via OpenMP
For maximum performance on the Salomon cluster compute nodes, compile your programs using the AVX2 instructions, with reporting where the vectorization was used. We recommend following compilation options for high performance
```console
$ icc -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec myprog.c mysubroutines.c -o myprog.x
$ ifort -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec myprog.f mysubroutines.f -o myprog.x
```
In this example, we compile the program enabling interprocedural optimizations between source files (-ipo), aggresive loop optimizations (-O3) and vectorization (-xCORE-AVX2)
The compiler recognizes the omp, simd, vector and ivdep pragmas for OpenMP parallelization and AVX2 vectorization. Enable the OpenMP parallelization by the **-openmp** compiler switch.
```console
$ icc -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec -openmp myprog.c mysubroutines.c -o myprog.x
$ ifort -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec -openmp myprog.f mysubroutines.f -o myprog.x
```
Read more at <https://software.intel.com/en-us/intel-cplusplus-compiler-16.0-user-and-reference-guide>
## Sandy Bridge/Ivy Bridge/Haswell Binary Compatibility
Anselm nodes are currently equipped with Sandy Bridge CPUs, while Salomon compute nodes are equipped with Haswell based architecture. The UV1 SMP compute server has Ivy Bridge CPUs, which are equivalent to Sandy Bridge (only smaller manufacturing technology). The new processors are backward compatible with the Sandy Bridge nodes, so all programs that ran on the Sandy Bridge processors, should also run on the new Haswell nodes. To get optimal performance out of the Haswell processors a program should make use of the special AVX2 instructions for this processor. One can do this by recompiling codes with the compiler flags designated to invoke these instructions. For the Intel compiler suite, there are two ways of doing this:
* Using compiler flag (both for Fortran and C): -xCORE-AVX2. This will create a binary with AVX2 instructions, specifically for the Haswell processors. Note that the executable will not run on Sandy Bridge/Ivy Bridge nodes.
* Using compiler flags (both for Fortran and C): -xAVX -axCORE-AVX2. This will generate multiple, feature specific auto-dispatch code paths for Intel® processors, if there is a performance benefit. So this binary will run both on Sandy Bridge/Ivy Bridge and Haswell processors. During runtime it will be decided which path to follow, dependent on which processor you are running on. In general this will result in larger binaries.
# Intel Debugger
IDB is no longer available since Intel Parallel Studio 2015
## Debugging Serial Applications
The intel debugger version is available, via module intel/13.5.192. The debugger works for applications compiled with C and C++ compiler and the ifort fortran 77/90/95 compiler. The debugger provides java GUI environment. Use [X display](../../general/accessing-the-clusters/graphical-user-interface/x-window-system/) for running the GUI.
```console
$ ml intel/13.5.192
$ ml Java
$ idb
```
The debugger may run in text mode. To debug in text mode, use
```console
$ idbc
```
To debug on the compute nodes, module intel must be loaded. The GUI on compute nodes may be accessed using the same way as in [the GUI section](../../general/accessing-the-clusters/graphical-user-interface/x-window-system/)
Example:
```console
$ qsub -q qexp -l select=1:ncpus=24 -X -I # use 16 threads for Anselm
qsub: waiting for job 19654.srv11 to start
qsub: job 19654.srv11 ready
$ ml intel
$ ml Java
$ icc -O0 -g myprog.c -o myprog.x
$ idb ./myprog.x
```
In this example, we allocate 1 full compute node, compile program myprog.c with debugging options -O0 -g and run the idb debugger interactively on the myprog.x executable. The GUI access is via X11 port forwarding provided by the PBS workload manager.
## Debugging Parallel Applications
Intel debugger is capable of debugging multithreaded and MPI parallel programs as well.
### Small Number of MPI Ranks
For debugging small number of MPI ranks, you may execute and debug each rank in separate xterm terminal (do not forget the [X display](../../general/accessing-the-clusters/graphical-user-interface/x-window-system/)). Using Intel MPI, this may be done in following way:
```console
$ qsub -q qexp -l select=2:ncpus=24 -X -I
qsub: waiting for job 19654.srv11 to start
qsub: job 19655.srv11 ready
$ ml intel
$ mpirun -ppn 1 -hostfile $PBS_NODEFILE --enable-x xterm -e idbc ./mympiprog.x
```
In this example, we allocate 2 full compute node, run xterm on each node and start idb debugger in command line mode, debugging two ranks of mympiprog.x application. The xterm will pop up for each rank, with idb prompt ready. The example is not limited to use of Intel MPI
### Large Number of MPI Ranks
Run the idb debugger from within the MPI debug option. This will cause the debugger to bind to all ranks and provide aggregated outputs across the ranks, pausing execution automatically just after startup. You may then set break points and step the execution manually. Using Intel MPI:
```console
$ qsub -q qexp -l select=2:ncpus=24 -X -I
qsub: waiting for job 19654.srv11 to start
qsub: job 19655.srv11 ready
$ ml intel
$ mpirun -n 48 -idb ./mympiprog.x
```
### Debugging Multithreaded Application
Run the idb debugger in GUI mode. The menu Parallel contains number of tools for debugging multiple threads. One of the most useful tools is the **Serialize Execution** tool, which serializes execution of concurrent threads for easy orientation and identification of concurrency related bugs.
## Further Information
Exhaustive manual on idb features and usage is published at Intel website, <https://software.intel.com/sites/products/documentation/doclib/iss/2013/compiler/cpp-lin/>
# Intel Inspector
Intel Inspector is a dynamic memory and threading error checking tool for C/C++/Fortran applications. It can detect issues such as memory leaks, invalid memory references, uninitalized variables, race conditions, deadlocks etc.
## Installed Versions
The following versions are currently available on Salomon as modules:
2016 Update 1 - Inspector/2016_update1
## Usage
Your program should be compiled with -g switch to include symbol names. Optimizations can be turned on.
Debugging is possible either directly from the GUI, or from command line.
### GUI Mode
To debug from GUI, launch Inspector:
```console
$ inspxe-gui &
```
Then select menu File -> New -> Project. Choose a directory to save project data to. After clicking OK, Project properties window will appear, where you can configure path to your binary, launch arguments, working directory etc. After clicking OK, the project is ready.
In the main pane, you can start a predefined analysis type or define your own. Click Start to start the analysis. Alternatively, you can click on Command Line, to see the command line required to run the analysis directly from command line.
### Batch Mode
Analysis can be also run from command line in batch mode. Batch mode analysis is run with command inspxe-cl. To obtain the required parameters, either consult the documentation or you can configure the analysis in the GUI and then click "Command Line" button in the lower right corner to the respective command line.