From 3e5a711d6de0b633bf8716c5b2b6648ee3640440 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Thu, 26 Jan 2017 08:04:16 +0100
Subject: [PATCH] add lmod and modify menu
---
docs.it4i/software/lmod.md | 331 +++++++++++++++++++++++++++++++++++++
mkdocs.yml | 43 ++---
2 files changed, 353 insertions(+), 21 deletions(-)
create mode 100644 docs.it4i/software/lmod.md
diff --git a/docs.it4i/software/lmod.md b/docs.it4i/software/lmod.md
new file mode 100644
index 000000000..4d8c377ac
--- /dev/null
+++ b/docs.it4i/software/lmod.md
@@ -0,0 +1,331 @@
+Lmod
+====
+
+Lmod as a modules tool, a modern alternative to the oudated & no longer actively maintained Tcl-based environment modules tool.
+
+Detailed documentation on Lmod is available at [here](http://lmod.readthedocs.io).
+
+Benefits
+--------
+
+ * significantly more responsive module commands, in particular module avail (ml av)
+ * easier to use interface
+ * module files can be written in either Tcl or Lua syntax (and both types of modules can be mixed together)
+
+Introduction
+------------
+
+Below you will find more details and examples.
+
+|command|equivalent|
+|---|---|
+|ml|module list|
+|ml GCC/6.2.0-2.27|module load GCC/6.2.0-2.27|
+|ml -GCC| module unload GCC|
+|ml av|module avail|
+|ml show GCC/6.2.0-2.27|module show GCC|
+|ml spider|gcc searches (case-insensitive) for gcc in all available modules|
+|ml spider GCC/6.2.0-2.27|show all information about the module GCC/6.2.0-2.27|
+|ml save mycollection|stores the currently loaded modules to a collection|
+|ml restore mycollection|restores a previously stored collection of modules|
+
+Listing loaded modules: ml (module load)
+-------------------------------------------
+
+To get an overview of the currently loaded modules, use module list or ml (without specifying extra arguments).
+
+```bash
+$ ml
+Currently Loaded Modules:
+ 1) EasyBuild/3.0.0 (S) 2) lmod/7.2.2
+ Where:
+ S: Module is Sticky, requires --force to unload or purge
+```
+
+!!! tip
+ for more details on sticky modules, see the section on [ml purge](#resetting-by-unloading-all-modules-ml-purge-module-purge)
+
+Searching for available modules: ml av (module avail) and ml spider
+----------------------------------------------------------------------
+
+To get an overview of all available modules, you can use module avail or simply ml av:
+
+```bash
+$ ml av
+---------------------------------------- /apps/modules/compiler ----------------------------------------------
+ GCC/5.2.0 GCCcore/6.2.0 (D) icc/2013.5.192 ifort/2013.5.192 LLVM/3.9.0-intel-2017.00 (D)
+ ... ...
+
+---------------------------------------- /apps/modules/devel -------------------------------------------------
+ Autoconf/2.69-foss-2015g CMake/3.0.0-intel-2016.01 M4/1.4.17-intel-2016.01 pkg-config/0.27.1-foss-2015g
+ Autoconf/2.69-foss-2016a CMake/3.3.1-foss-2015g M4/1.4.17-intel-2017.00 pkg-config/0.27.1-intel-2015b
+ ... ...
+```
+
+!!! note
+ In the current module naming scheme, each module name consists of two parts:
+ * the part before the first /, corresponding to the software name; and
+ * the remainder, corresponding to the software version, the compiler toolchain that was used to install the software, and a possible version suffix
+
+
+!!! tip
+ The (D) indicates that this particular version of the module is the default, but we strongly recommend to not rely on this as the default can change at any point. Usuall, the default will point to the latest version available.
+
+Searching for modules: ml spider
+--------------------------------
+
+If you just provide a software name, for example gcc, it prints on overview of all available modules for GCC.
+
+```bash
+$ ml spider gcc
+---------------------------------------------------------------------------------
+ GCC:
+---------------------------------------------------------------------------------
+ Description:
+ The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...). - Homepage: http://gcc.gnu.org/
+
+ Versions:
+ GCC/4.4.7-system
+ GCC/4.7.4
+ GCC/4.8.3
+ GCC/4.9.2-binutils-2.25
+ GCC/4.9.2
+ GCC/4.9.3-binutils-2.25
+ GCC/4.9.3
+ GCC/4.9.3-2.25
+ GCC/5.1.0-binutils-2.25
+ GCC/5.2.0
+ GCC/5.3.0-binutils-2.25
+ GCC/5.3.0-2.25
+ GCC/5.3.0-2.26
+ GCC/5.3.1-snapshot-20160419-2.25
+ GCC/5.4.0-2.26
+ GCC/6.2.0-2.27
+
+ Other possible modules matches:
+ GCCcore
+---------------------------------------------------------------------------------
+ To find other possible module matches do:
+ module -r spider '.*GCC.*'
+---------------------------------------------------------------------------------
+ For detailed information about a specific "GCC" module (including how to load the modules) use the module's full name.
+ For example:
+ $ module spider GCC/6.2.0-2.27
+---------------------------------------------------------------------------------
+```
+
+!!! tip
+ spider is case-insensitive.
+
+If you use spider on a full module name like GCC/6.2.0-2.27 it will tell on which cluster(s) that module available:
+
+```bash
+$ module spider GCC/6.2.0-2.27
+--------------------------------------------------------------------------------------------------------------
+ GCC: GCC/6.2.0-2.27
+--------------------------------------------------------------------------------------------------------------
+ Description:
+ The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...). - Homepage: http://gcc.gnu.org/
+
+ This module can be loaded directly: module load GCC/6.2.0-2.27
+
+ Help:
+ The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada,
+ as well as libraries for these languages (libstdc++, libgcj,...). - Homepage: http://gcc.gnu.org/
+```
+
+This tells you what the module contains and a URL to the homepage of the software.
+
+Available modules for a particular software package: ml av <name>
+------------------------------------------------------------------------------------------
+
+To check which modules are available for a particular software package, you can provide the software name to ml av.
+For example, to check which versions of git are available:
+
+```bash
+$ ml av git
+
+-------------------------------------- /apps/modules/tools ----------------------------------------
+ git/2.8.0-GNU-4.9.3-2.25 git/2.8.0-intel-2017.00 git/2.9.0 git/2.9.2 git/2.11.0 (D)
+
+ Where:
+ D: Default Module
+
+Use "module spider" to find all possible modules.
+Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
+```
+!!! tip
+ the specified software name is treated case-insensitively.
+
+Lmod does a partial match on the module name, so sometimes you need to use / to indicate the end of the software name you are interested in:
+
+```bash
+$ ml av GCC/
+
+------------------------------------------ /apps/modules/compiler -------------------------------------------
+GCC/4.4.7-system GCC/4.8.3 GCC/4.9.2 GCC/4.9.3 GCC/5.1.0-binutils-2.25 GCC/5.3.0-binutils-2.25 GCC/5.3.0-2.26 GCC/5.4.0-2.26 GCC/4.7.4 GCC/4.9.2-binutils-2.25 GCC/4.9.3-binutils-2.25 GCC/4.9.3-2.25 GCC/5.2.0 GCC/5.3.0-2.25 GCC/6.2.0-2.27 (D)
+
+ Where:
+ D: Default Module
+
+Use "module spider" to find all possible modules.
+Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
+
+```
+
+Inspecting a module using ml show
+--------------------------------------------------
+
+To see how a module would change the environment, use module show or ml show:
+
+```bash
+$ ml show Python/3.5.2
+
+help([[Python is a programming language that lets you work more quickly and integrate your systems more effectively. - Homepage: http://python.org/]])
+whatis("Description: Python is a programming language that lets you work more quickly and integrate your systems more effectively. - Homepage: http://python.org/")
+conflict("Python")
+load("bzip2/1.0.6")
+load("zlib/1.2.8")
+load("libreadline/6.3")
+load("ncurses/5.9")
+load("SQLite/3.8.8.1")
+load("Tk/8.6.3")
+load("GMP/6.0.0a")
+load("XZ/5.2.2")
+prepend_path("CPATH","/apps/all/Python/3.5.2/include")
+prepend_path("LD_LIBRARY_PATH","/apps/all/Python/3.5.2/lib")
+prepend_path("LIBRARY_PATH","/apps/all/Python/3.5.2/lib")
+prepend_path("MANPATH","/apps/all/Python/3.5.2/share/man")
+prepend_path("PATH","/apps/all/Python/3.5.2/bin")
+prepend_path("PKG_CONFIG_PATH","/apps/all/Python/3.5.2/lib/pkgconfig")
+setenv("EBROOTPYTHON","/apps/all/Python/3.5.2")
+setenv("EBVERSIONPYTHON","3.5.2")
+setenv("EBDEVELPYTHON","/apps/all/Python/3.5.2/easybuild/Python-3.5.2-easybuild-devel")
+setenv("EBEXTSLISTPYTHON","setuptools-20.1.1,pip-8.0.2,nose-1.3.7")
+
+```
+
+!!! tip
+ Note that both the direct changes to the environment as well as other modules that will be loaded are shown.
+
+If you're not sure what all of this means: don't worry, you don't have to know; just try loading the module as try using the software.
+
+## Loading modules: ml <modname(s)> (module load <modname(s)>)
+
+The effectively apply the changes to the environment that are specified by a module, use module load or ml and specify the name of the module.
+For example, to set up your environment to use intel:
+
+```bash
+$ ml intel/2017.00
+$ ml
+Currently Loaded Modules:
+ 1) GCCcore/5.4.0
+ 2) binutils/2.26-GCCcore-5.4.0 (H)
+ 3) icc/2017.0.098-GCC-5.4.0-2.26
+ 4) ifort/2017.0.098-GCC-5.4.0-2.26
+ 5) iccifort/2017.0.098-GCC-5.4.0-2.26
+ 6) impi/2017.0.098-iccifort-2017.0.098-GCC-5.4.0-2.26
+ 7) iimpi/2017.00-GCC-5.4.0-2.26
+ 8) imkl/2017.0.098-iimpi-2017.00-GCC-5.4.0-2.26
+ 9) intel/2017.00
+
+ Where:
+ H: Hidden Module
+```
+
+!!! tip
+ Note that even though we only loaded a single module, the output of ml shows that a whole bunch of modules were loaded, which are required dependencies for intel/2017.00.
+
+Conflicting modules
+-------------------
+
+!!! warning
+ It is important to note that **only modules that are compatible with each other can be loaded together. In particular, modules must be installed either with the same toolchain as the modules that** are already loaded, or with a compatible (sub)toolchain.
+
+For example, once you have loaded one or more modules that were installed with the intel/2017.00 toolchain, all other modules that you load should have been installed with the same toolchain.
+
+In addition, only **one single version** of each software package can be loaded at a particular time. For example, once you have the Python/3.5.2-intel-2017.00 module loaded, you can not load a different version of Python in the same session/job script; neither directly, nor indirectly as a dependency of another module you want to load.
+
+Unloading modules: ml -<modname(s)> (module unload <modname(s)>)
+
+To revert the changes to the environment that were made by a particular module, you can use module unload or ml -<modname>.
+For example:
+
+```bash
+$ ml
+Currently Loaded Modules:
+ 1) EasyBuild/3.0.0 (S) 2) lmod/7.2.2
+$ which gcc
+/usr/bin/gcc
+$ ml GCC/
+$ ml
+Currently Loaded Modules:
+ 1) EasyBuild/3.0.0 (S) 2) lmod/7.2.2 3) GCCcore/6.2.0 4) binutils/2.27-GCCcore-6.2.0 (H) 5) GCC/6.2.0-2.27
+$ which gcc
+/apps/all/GCCcore/6.2.0/bin/gcc
+$ ml -GCC
+$ ml
+Currently Loaded Modules:
+ 1) EasyBuild/3.0.0 (S) 2) lmod/7.2.2 3) GCCcore/6.2.0 4) binutils/2.27-GCCcore-6.2.0 (H)
+$ which gcc
+/usr/bin/gcc
+```
+
+Resetting by unloading all modules: ml purge (module purge)
+-----------------------------------------------------------
+
+To reset your environment back to a clean state, you can use module purge or ml purge:
+
+```bash
+$ ml
+Currently Loaded Modules:
+ 1) EasyBuild/3.0.0 (S) 2) lmod/7.2.2 3) GCCcore/6.2.0 4) binutils/2.27-GCCcore-6.2.0 (H)
+$ ml purge
+The following modules were not unloaded:
+ (Use "module --force purge" to unload all):
+ 1) EasyBuild/3.0.0
+$ ml
+Currently Loaded Modules:
+ 1) EasyBuild/3.0.0 (S)
+$ ml purge --force
+$ ml
+No modules loaded
+```
+
+As such, you should not (re)load the cluster module anymore after running ml purge. See also here.
+
+Module collections: ml save, ml restore
+---------------------------------------
+
+If you have a set of modules that you need to load often, you can save these in a collection (only works with Lmod).
+
+First, load all the modules you need, for example:
+
+```bash
+ml intel/2017.00 Python/3.5.2-intel-2017.00
+```
+
+Now store them in a collection using ml save:
+
+```bash
+$ ml save my-collection
+```
+
+Later, for example in a job script, you can reload all these modules with ml restore:
+
+```bash
+$ ml restore my-collection
+```
+
+With ml savelist can you gets a list of all saved collections:
+
+```bash
+$ ml savelist
+Named collection list:
+ 1) my-collection
+ 2) my-test-collection
+```
+
+To inspect a collection, use ml describe.
+
+To remove a module collection, remove the corresponding entry in $HOME/.lmod.d.
diff --git a/mkdocs.yml b/mkdocs.yml
index 476a365fc..883a4d8d4 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -44,7 +44,26 @@ pages:
- IB Single-plane Topology: salomon/ib-single-plane-topology.md
- 7D Enhanced Hypercube: salomon/7d-enhanced-hypercube.md
- Storage: salomon/storage.md
- - Software:
+ - PRACE User Support: salomon/prace.md
+ - Anselm Cluster:
+ - Introduction: anselm-cluster-documentation/introduction.md
+ - Hardware Overview: anselm-cluster-documentation/hardware-overview.md
+ - Accessing the Cluster: anselm-cluster-documentation/shell-and-data-access.md
+ - Environment and Modules: anselm-cluster-documentation/environment-and-modules.md
+ - Resource Allocation and Job Execution:
+ - Resource Allocation Policy: anselm-cluster-documentation/resources-allocation-policy.md
+ - Job Priority: anselm-cluster-documentation/job-priority.md
+ - Job Submission and Execution: anselm-cluster-documentation/job-submission-and-execution.md
+ - Capacity Computing: anselm-cluster-documentation/capacity-computing.md
+ - Compute Nodes: anselm-cluster-documentation/compute-nodes.md
+ - Storage: anselm-cluster-documentation/storage.md
+ - Network: anselm-cluster-documentation/network.md
+ - Remote Visualization: anselm-cluster-documentation/remote-visualization.md
+ - PRACE User Support: anselm-cluster-documentation/prace.md
+ - 'Software':
+ - Environment Lmod: software/lmod.md
+ - Modules Matrix: modules-matrix.md
+ - Salomon Software:
- Available Modules: modules-salomon.md
- Available Modules on UV: modules-salomon-uv.md
- 'ANSYS':
@@ -98,21 +117,7 @@ pages:
- Octave: salomon/software/numerical-languages/octave.md
- R: salomon/software/numerical-languages/r.md
- Operating System: salomon/software/operating-system.md
- - PRACE User Support: salomon/prace.md
- - Anselm Cluster:
- - Introduction: anselm-cluster-documentation/introduction.md
- - Hardware Overview: anselm-cluster-documentation/hardware-overview.md
- - Accessing the Cluster: anselm-cluster-documentation/shell-and-data-access.md
- - Environment and Modules: anselm-cluster-documentation/environment-and-modules.md
- - Resource Allocation and Job Execution:
- - Resource Allocation Policy: anselm-cluster-documentation/resources-allocation-policy.md
- - Job Priority: anselm-cluster-documentation/job-priority.md
- - Job Submission and Execution: anselm-cluster-documentation/job-submission-and-execution.md
- - Capacity Computing: anselm-cluster-documentation/capacity-computing.md
- - Compute Nodes: anselm-cluster-documentation/compute-nodes.md
- - Storage: anselm-cluster-documentation/storage.md
- - Network: anselm-cluster-documentation/network.md
- - Software:
+ - Anselm Software:
- Available Modules: modules-anselm.md
- 'ANSYS':
- Introduction: anselm-cluster-documentation/software/ansys/ansys.md
@@ -181,11 +186,7 @@ pages:
- Operating System: anselm-cluster-documentation/software/operating-system.md
- ParaView: anselm-cluster-documentation/software/paraview.md
- Virtualization: anselm-cluster-documentation/software/kvirtualization.md
- - Remote Visualization: anselm-cluster-documentation/remote-visualization.md
- - PRACE User Support: anselm-cluster-documentation/prace.md
-# - 'Software':
-# - Modules Matrix: modules-matrix.md
- - Modules Matrix: modules-matrix.md
+# - Modules Matrix: modules-matrix.md
- PBS Pro Documentation: pbspro.md
# - Testing:
# - Colors: colors.md
--
GitLab