Skip to content
Snippets Groups Projects
Commit e0d02628 authored by Lukáš Krupčík's avatar Lukáš Krupčík
Browse files

fix links, remove duplicate environment-and-modules and create new environment-and-modules

parent 9606be77
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs.it4i/anselm/environment-and-modules.md deleted 100644 → 0
+ 0
− 78
View file @ 9606be77
# Environment and Modules
## Environment Customization
After logging in, you may want to configure the environment. Write your preferred path definitions, aliases, functions and module loads in the .bashrc file
```console
$ cat ./bashrc
# ./bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
alias qs='qstat -a'
module load PrgEnv-gnu
# Display information to standard output - only in interactive ssh session
if [ -n "$SSH_TTY" ]
then
module list # Display loaded modules
fi
```
!!! note
Do not run commands outputting to standard output (echo, module list, etc) in .bashrc for non-interactive SSH sessions. It breaks fundamental functionality (SCP, PBS) of your account! Consider utilization of SSH session interactivity for such commands as stated in the previous example.
## Application Modules
In order to configure your shell for running particular application on Anselm we use Module package interface.
!!! note
The modules set up the application paths, library paths and environment variables for running particular application.
We have also second modules repository. This modules repository is created using tool called EasyBuild. On Salomon cluster, all modules will be build by this tool. If you want to use software from this modules repository, please follow instructions in section [Application Modules Path Expansion](environment-and-modules/#application-modules-path-expansion).
The modules may be loaded, unloaded and switched, according to momentary needs.
To check available modules use
```console
$ ml av
```
To load a module, for example the octave module use
```console
$ ml octave
```
loading the octave module will set up paths and environment variables of your active shell such that you are ready to run the octave software
To check loaded modules use
```console
$ ml
```
To unload a module, for example the octave module use
```console
$ ml -octave
```
Following modules set up the development environment
PrgEnv-gnu sets up the GNU development environment in conjunction with the bullx MPI library
PrgEnv-intel sets up the INTEL development environment in conjunction with the Intel MPI library
## Application Modules Path Expansion
All application modules on Anselm cluster (and further) will be build using tool called [EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild").
This command expands your searched paths to modules. You can also add this command to the .bashrc file to expand paths permanently. After this command, you can use same commands to list/add/remove modules as is described above.
docs.it4i/environment-and-modules.md
+ 11
− 71
View file @ e0d02628
...@@ -7,19 +7,19 @@ After logging in, you may want to configure the environment. Write your preferre ...@@ -7,19 +7,19 @@ After logging in, you may want to configure the environment. Write your preferre
```console ```console
# ./bashrc # ./bashrc
# Source global definitions # users compilation path
if [ -f /etc/bashrc ]; then export MODULEPATH=${MODULEPATH}:/home/$USER/.local/easybuild/modules/all
. /etc/bashrc
fi
# User specific aliases and functions # User specific aliases and functions
alias qs='qstat -a' alias qs='qstat -a'
module load intel/2015b
# load default intel compilator !!! is not recommended !!!
ml intel
# Display information to standard output - only in interactive ssh session # Display information to standard output - only in interactive ssh session
if [ -n "$SSH_TTY" ] if [ -n "$SSH_TTY" ]
then then
module list # Display loaded modules ml # Display loaded modules
fi fi
``` ```
...@@ -28,9 +28,9 @@ fi ...@@ -28,9 +28,9 @@ fi
### Application Modules ### Application Modules
In order to configure your shell for running particular application on Salomon we use Module package interface. In order to configure your shell for running particular application on clusters we use Module package interface.
Application modules on Salomon cluster are built using [EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild"). The modules are divided into the following structure: Application modules on clusters are built using [EasyBuild](software/tools/easybuild/). The modules are divided into the following structure:
```console ```console
base: Default module class base: Default module class
...@@ -54,71 +54,11 @@ Application modules on Salomon cluster are built using [EasyBuild](http://hpcuge ...@@ -54,71 +54,11 @@ Application modules on Salomon cluster are built using [EasyBuild](http://hpcuge
toolchain: EasyBuild toolchains toolchain: EasyBuild toolchains
tools: General purpose tools tools: General purpose tools
vis: Visualization, plotting, documentation and typesetting vis: Visualization, plotting, documentation and typesetting
OS: singularity image
python: python packages
``` ```
!!! note !!! note
The modules set up the application paths, library paths and environment variables for running particular application. The modules set up the application paths, library paths and environment variables for running particular application.
The modules may be loaded, unloaded and switched, according to momentary needs. The modules may be loaded, unloaded and switched, according to momentary needs. For details see [here](software/modules/lmod/).
To check available modules use
```console
$ ml av
```
To load a module, for example the Open MPI module use
```console
$ ml OpenMPI
```
loading the Open MPI module will set up paths and environment variables of your active shell such that you are ready to run the Open MPI software
To check loaded modules use
```console
$ ml
```
To unload a module, for example the Open MPI module use
```console
$ ml -OpenMPI
```
Learn more on modules by reading the module man page
```console
$ man module
```
### EasyBuild Toolchains
As we wrote earlier, we are using EasyBuild for automatized software installation and module creation.
EasyBuild employs so-called **compiler toolchains** or, simply toolchains for short, which are a major concept in handling the build and installation processes.
A typical toolchain consists of one or more compilers, usually put together with some libraries for specific functionality, e.g., for using an MPI stack for distributed computing, or which provide optimized routines for commonly used math operations, e.g., the well-known BLAS/LAPACK APIs for linear algebra routines.
For each software package being built, the toolchain to be used must be specified in some way.
The EasyBuild framework prepares the build environment for the different toolchain components, by loading their respective modules and defining environment variables to specify compiler commands (e.g., via `$F90`), compiler and linker options (e.g., via `$CFLAGS` and `$LDFLAGS`), the list of library names to supply to the linker (via `$LIBS`), etc. This enables making easyblocks largely toolchain-agnostic since they can simply rely on these environment variables; that is, unless they need to be aware of, for example, the particular compiler being used to determine the build configuration options.
Recent releases of EasyBuild include out-of-the-box toolchain support for:
* various compilers, including GCC, Intel, Clang, CUDA
* common MPI libraries, such as Intel MPI, MPICH, MVAPICH2, Open MPI
* various numerical libraries, including ATLAS, Intel MKL, OpenBLAS, ScaLAPACK, FFTW
On Salomon, we have currently following toolchains installed:
| Toolchain | Module(s) |
| --------- | ---------------------------------------------- |
| GCC | GCC |
| ictce | icc, ifort, imkl, impi |
| intel | GCC, icc, ifort, imkl, impi |
| gompi | GCC, OpenMPI |
| goolf | BLACS, FFTW, GCC, OpenBLAS, OpenMPI, ScaLAPACK |
| iompi | OpenMPI, icc, ifort |
| iccifort | icc, ifort |
docs.it4i/salomon/environment-and-modules.md deleted 100644 → 0
+ 0
− 124
View file @ 9606be77
# Environment and Modules
## Environment Customization
After logging in, you may want to configure the environment. Write your preferred path definitions, aliases, functions and module loads in the .bashrc file
```console
# ./bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
alias qs='qstat -a'
module load intel/2015b
# Display information to standard output - only in interactive ssh session
if [ -n "$SSH_TTY" ]
then
module list # Display loaded modules
fi
```
!!! note
Do not run commands outputting to standard output (echo, module list, etc) in .bashrc for non-interactive SSH sessions. It breaks fundamental functionality (SCP, PBS) of your account! Take care for SSH session interactivity for such commands as stated in the previous example.
### Application Modules
In order to configure your shell for running particular application on Salomon we use Module package interface.
Application modules on Salomon cluster are built using [EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild"). The modules are divided into the following structure:
```console
base: Default module class
bio: Bioinformatics, biology and biomedical
cae: Computer Aided Engineering (incl. CFD)
chem: Chemistry, Computational Chemistry and Quantum Chemistry
compiler: Compilers
data: Data management & processing tools
debugger: Debuggers
devel: Development tools
geo: Earth Sciences
ide: Integrated Development Environments (e.g. editors)
lang: Languages and programming aids
lib: General purpose libraries
math: High-level mathematical software
mpi: MPI stacks
numlib: Numerical Libraries
perf: Performance tools
phys: Physics and physical systems simulations
system: System utilities (e.g. highly depending on system OS and hardware)
toolchain: EasyBuild toolchains
tools: General purpose tools
vis: Visualization, plotting, documentation and typesetting
```
!!! note
The modules set up the application paths, library paths and environment variables for running particular application.
The modules may be loaded, unloaded and switched, according to momentary needs.
To check available modules use
```console
$ ml av
```
To load a module, for example the Open MPI module use
```console
$ ml OpenMPI
```
loading the Open MPI module will set up paths and environment variables of your active shell such that you are ready to run the Open MPI software
To check loaded modules use
```console
$ ml
```
To unload a module, for example the Open MPI module use
```console
$ ml -OpenMPI
```
Learn more on modules by reading the module man page
```console
$ man module
```
### EasyBuild Toolchains
As we wrote earlier, we are using EasyBuild for automatized software installation and module creation.
EasyBuild employs so-called **compiler toolchains** or, simply toolchains for short, which are a major concept in handling the build and installation processes.
A typical toolchain consists of one or more compilers, usually put together with some libraries for specific functionality, e.g., for using an MPI stack for distributed computing, or which provide optimized routines for commonly used math operations, e.g., the well-known BLAS/LAPACK APIs for linear algebra routines.
For each software package being built, the toolchain to be used must be specified in some way.
The EasyBuild framework prepares the build environment for the different toolchain components, by loading their respective modules and defining environment variables to specify compiler commands (e.g., via `$F90`), compiler and linker options (e.g., via `$CFLAGS` and `$LDFLAGS`), the list of library names to supply to the linker (via `$LIBS`), etc. This enables making easyblocks largely toolchain-agnostic since they can simply rely on these environment variables; that is, unless they need to be aware of, for example, the particular compiler being used to determine the build configuration options.
Recent releases of EasyBuild include out-of-the-box toolchain support for:
* various compilers, including GCC, Intel, Clang, CUDA
* common MPI libraries, such as Intel MPI, MPICH, MVAPICH2, Open MPI
* various numerical libraries, including ATLAS, Intel MKL, OpenBLAS, ScaLAPACK, FFTW
On Salomon, we have currently following toolchains installed:
| Toolchain | Module(s) |
| --------- | ---------------------------------------------- |
| GCC | GCC |
| ictce | icc, ifort, imkl, impi |
| intel | GCC, icc, ifort, imkl, impi |
| gompi | GCC, OpenMPI |
| goolf | BLACS, FFTW, GCC, OpenBLAS, OpenMPI, ScaLAPACK |
| iompi | OpenMPI, icc, ifort |
| iccifort | icc, ifort |
mkdocs.yml
+ 1
− 2
View file @ e0d02628
...@@ -31,7 +31,6 @@ pages: ...@@ -31,7 +31,6 @@ pages:
- Introduction: salomon/introduction.md - Introduction: salomon/introduction.md
- Hardware Overview: salomon/hardware-overview.md - Hardware Overview: salomon/hardware-overview.md
- Accessing the Cluster: salomon/shell-and-data-access.md - Accessing the Cluster: salomon/shell-and-data-access.md
- Environment and Modules: salomon/environment-and-modules.md
- Resource Allocation and Job Execution: - Resource Allocation and Job Execution:
- Resources Allocation Policy: salomon/resources-allocation-policy.md - Resources Allocation Policy: salomon/resources-allocation-policy.md
- Job Scheduling: salomon/job-priority.md - Job Scheduling: salomon/job-priority.md
...@@ -47,7 +46,6 @@ pages: ...@@ -47,7 +46,6 @@ pages:
- Introduction: anselm/introduction.md - Introduction: anselm/introduction.md
- Hardware Overview: anselm/hardware-overview.md - Hardware Overview: anselm/hardware-overview.md
- Accessing the Cluster: anselm/shell-and-data-access.md - Accessing the Cluster: anselm/shell-and-data-access.md
- Environment and Modules: anselm/environment-and-modules.md
- Resource Allocation and Job Execution: - Resource Allocation and Job Execution:
- Resource Allocation Policy: anselm/resources-allocation-policy.md - Resource Allocation Policy: anselm/resources-allocation-policy.md
- Job Priority: anselm/job-priority.md - Job Priority: anselm/job-priority.md
...@@ -57,6 +55,7 @@ pages: ...@@ -57,6 +55,7 @@ pages:
- Storage: anselm/storage.md - Storage: anselm/storage.md
- Network: anselm/network.md - Network: anselm/network.md
- Software: - Software:
- Environment and Modules: environment-and-modules.md
- Modules: - Modules:
- Lmod Environment: software/modules/lmod.md - Lmod Environment: software/modules/lmod.md
- Intel Xeon Phi Environment: software/mic/mic_environment.md - Intel Xeon Phi Environment: software/mic/mic_environment.md
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment