From 9492d76339055e5d3b4954c02a2d218b93e2bba2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 10:40:03 +0200
Subject: [PATCH 01/58] Update comsol-multiphysics.md
---
.../cae/comsol/comsol-multiphysics.md | 35 +++++++++++++++----
1 file changed, 29 insertions(+), 6 deletions(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index 0ef8b15cd..d59f91018 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -32,17 +32,40 @@ By default, the **EDU variant** will be loaded. If you needs other version or va
$ ml av COMSOL
```
-To prepare COMSOL jobs in the interactive mode, we recommend using COMSOL on the compute nodes via the PBS Pro scheduler. To run the COMSOL Desktop GUI on Windows, we recommend using the [Virtual Network Computing (VNC)][2].
+To prepare COMSOL jobs in the interactive mode, we recommend using COMSOL on the compute nodes via the PBS Pro scheduler. To run the COMSOL Desktop GUI on Windows.
-Example for Salomon:
+!!! Note
+ We recommend using the [Virtual Network Computing (VNC)][2].
+
+Example for Barbora:
```console
-$ xhost +
-$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ppn=24
-$ ml COMSOL
-$ comsol
+$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=36
+$ ml av COMSOL
+
+------------------------------ /apps/modules/phys ------------------------------
+ COMSOL/5.2.0-COM COMSOL/5.2.0-EDU (D)
+
+$ ml COMSOL/5.2.0-EDU
+$ comsol -3drend sw
```
+!!! info
+ If you receive errors at startup, for example, of the following type:
+
+ ```console
+FL3D: error at line 814 in file fl3dglcontext_x11common.c:
+ PBuffers are not supported by the system.
+FL3D: error at line 235 in file fl3dglcontext_x11common.c:
+ assert: x11Handle
+ ```
+
+ you need to run COMSOL with additional parameters:
+
+ ```console
+$ comsol -3drend sw
+ ```
+
To run COMSOL in batch mode without the COMSOL Desktop GUI environment, utilize the default (comsol.pbs) job script and execute it via the `qsub` command:
```bash
--
GitLab
From c0eb116cb14de02880e45e5f4a314ef322fdcc88 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 10:42:16 +0200
Subject: [PATCH 02/58] Update comsol-multiphysics.md
---
.../software/cae/comsol/comsol-multiphysics.md | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index d59f91018..d2b60a4c9 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -35,7 +35,7 @@ $ ml av COMSOL
To prepare COMSOL jobs in the interactive mode, we recommend using COMSOL on the compute nodes via the PBS Pro scheduler. To run the COMSOL Desktop GUI on Windows.
!!! Note
- We recommend using the [Virtual Network Computing (VNC)][2].
+ We recommend using the [Virtual Network Computing (VNC)][2].
Example for Barbora:
@@ -52,19 +52,16 @@ $ comsol -3drend sw
!!! info
If you receive errors at startup, for example, of the following type:
-
- ```console
+```console
FL3D: error at line 814 in file fl3dglcontext_x11common.c:
PBuffers are not supported by the system.
FL3D: error at line 235 in file fl3dglcontext_x11common.c:
assert: x11Handle
- ```
-
- you need to run COMSOL with additional parameters:
-
- ```console
+```
+you need to run COMSOL with additional parameters:
+```console
$ comsol -3drend sw
- ```
+```
To run COMSOL in batch mode without the COMSOL Desktop GUI environment, utilize the default (comsol.pbs) job script and execute it via the `qsub` command:
--
GitLab
From 985000538da80d5d46303076a29e7375f2de68a6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 10:42:42 +0200
Subject: [PATCH 03/58] Update comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index d2b60a4c9..f6308ad76 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -51,7 +51,7 @@ $ comsol -3drend sw
```
!!! info
- If you receive errors at startup, for example, of the following type:
+ If you receive errors at startup, for example, of the following type:
```console
FL3D: error at line 814 in file fl3dglcontext_x11common.c:
PBuffers are not supported by the system.
--
GitLab
From 0e828eaf71610492260e0fc36bfff9b84fd2b70a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 10:44:55 +0200
Subject: [PATCH 04/58] Update comsol-multiphysics.md
---
.../cae/comsol/comsol-multiphysics.md | 23 +++++++++++--------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index f6308ad76..828ab3077 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -35,7 +35,7 @@ $ ml av COMSOL
To prepare COMSOL jobs in the interactive mode, we recommend using COMSOL on the compute nodes via the PBS Pro scheduler. To run the COMSOL Desktop GUI on Windows.
!!! Note
- We recommend using the [Virtual Network Computing (VNC)][2].
+ We recommend using the [Virtual Network Computing (VNC)][2].
Example for Barbora:
@@ -51,17 +51,20 @@ $ comsol -3drend sw
```
!!! info
- If you receive errors at startup, for example, of the following type:
-```console
-FL3D: error at line 814 in file fl3dglcontext_x11common.c:
- PBuffers are not supported by the system.
-FL3D: error at line 235 in file fl3dglcontext_x11common.c:
+ If you receive errors at startup, for example, of the following type:
+
+ ```console
+FL3D: error at line 814 in file fl3dglcontext_x11common.c:
+ PBuffers are not supported by the system.
+FL3D: error at line 235 in file fl3dglcontext_x11common.c:
assert: x11Handle
-```
-you need to run COMSOL with additional parameters:
-```console
+ ```
+
+ you need to run COMSOL with additional parameters:
+
+ ```console
$ comsol -3drend sw
-```
+ ```
To run COMSOL in batch mode without the COMSOL Desktop GUI environment, utilize the default (comsol.pbs) job script and execute it via the `qsub` command:
--
GitLab
From 3a884265f4aea36b2b76fccb00885e7503ddcb5d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 10:46:32 +0200
Subject: [PATCH 05/58] Update comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index 828ab3077..38b7f0464 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -52,7 +52,7 @@ $ comsol -3drend sw
!!! info
If you receive errors at startup, for example, of the following type:
-
+
```console
FL3D: error at line 814 in file fl3dglcontext_x11common.c:
PBuffers are not supported by the system.
@@ -60,11 +60,11 @@ FL3D: error at line 235 in file fl3dglcontext_x11common.c:
assert: x11Handle
```
- you need to run COMSOL with additional parameters:
+ you need to run COMSOL with additional parameters:
- ```console
+ ```console
$ comsol -3drend sw
- ```
+ ```
To run COMSOL in batch mode without the COMSOL Desktop GUI environment, utilize the default (comsol.pbs) job script and execute it via the `qsub` command:
--
GitLab
From 1e3719b0a2632dc9399d3969e5cb341169edd0f2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 10:58:52 +0200
Subject: [PATCH 06/58] Update comsol-multiphysics.md
---
.../cae/comsol/comsol-multiphysics.md | 50 +++++++++----------
1 file changed, 23 insertions(+), 27 deletions(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index 38b7f0464..db2daadf8 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -20,7 +20,7 @@ On the clusters, COMSOL is available in the latest stable version. There are two
* **Commercial** or so called **COM variant**, which can used also for commercial activities. **COM variant** has only subset of features compared to the **EDU variant** available. More about licensing [here][1].
-To load the `COMSOL` module, use:
+To load the default `COMSOL` module ([**not** recommended][4]), use:
```console
$ ml COMSOL
@@ -40,7 +40,7 @@ To prepare COMSOL jobs in the interactive mode, we recommend using COMSOL on the
Example for Barbora:
```console
-$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=36
+$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=36:mpiprocs=36
$ ml av COMSOL
------------------------------ /apps/modules/phys ------------------------------
@@ -50,32 +50,31 @@ $ ml COMSOL/5.2.0-EDU
$ comsol -3drend sw
```
-!!! info
- If you receive errors at startup, for example, of the following type:
+* If you receive errors at startup, for example, of the following type:
- ```console
+```console
FL3D: error at line 814 in file fl3dglcontext_x11common.c:
PBuffers are not supported by the system.
FL3D: error at line 235 in file fl3dglcontext_x11common.c:
assert: x11Handle
- ```
+```
- you need to run COMSOL with additional parameters:
+you need to run COMSOL with additional parameters:
- ```console
+```console
$ comsol -3drend sw
- ```
+```
To run COMSOL in batch mode without the COMSOL Desktop GUI environment, utilize the default (comsol.pbs) job script and execute it via the `qsub` command:
```bash
#!/bin/bash
-#PBS -l select=3:ppn=24
+#PBS -l select=3:ncpus=36:mpiprocs=36
#PBS -q qprod
#PBS -N JOB_NAME
#PBS -A PROJECT_ID
-cd /scratch/work/user/$USER/ || exit
+cd /scratch/project/PROJECT_ID/ || exit
echo Time is `date`
echo Directory is `pwd`
@@ -85,12 +84,11 @@ echo '**PBS_NODEFILE***END*********'
text_nodes < cat $PBS_NODEFILE
-ml COMSOL
-# ml COMSOL/51-EDU
+ml COMSOL/5.2.0-EDU
ntask=$(wc -l $PBS_NODEFILE)
-comsol -nn ${ntask} batch -configuration /tmp –mpiarg –rmk –mpiarg pbs -tmpdir /scratch/.../$USER/ -inputfile name_input_f.mph -outputfile name_output_f.mph -batchlog name_log_f.log
+comsol -3drend sw -nn ${ntask} batch -configuration /tmp –mpiarg –rmk –mpiarg pbs -tmpdir /scratch/project/PROJECT_ID/ -inputfile name_input_f.mph -outputfile name_output_f.mph -batchlog name_log_f.log
```
A working directory has to be created before sending the (comsol.pbs) job script into the queue. The input file (name_input_f.mph) has to be in the working directory or a full path to the input file has to be specified. The appropriate path to the temp directory of the job has to be set by the `-tmpdir` command option.
@@ -102,11 +100,9 @@ COMSOL is a software package for the numerical solution of partial differential
LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On the clusters there is 1 commercial (**COM**) and 5 educational (**EDU**) licenses of LiveLink for MATLAB (see the [ISV Licenses][3]). The following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode.
```console
-$ xhost +
-$ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ppn=24
-$ ml MATLAB
-$ ml COMSOL
-$ comsol server MATLAB
+$ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ncpus=36:mpiprocs=36
+$ ml MATLAB/R2015b COMSOL/5.2.0-EDU
+$ comsol -3drend sw server MATLAB
```
On the first start of the LiveLink for MATLAB (client-MATLAB/server-COMSOL connection), the login and password is requested; this information is not requested again.
@@ -115,12 +111,12 @@ To run LiveLink for MATLAB in batch mode with (comsol_matlab.pbs) job script, yo
```bash
#!/bin/bash
-#PBS -l select=3:ppn=24
+#PBS -l select=3:ncpus=36:mpiprocs=36
#PBS -q qprod
#PBS -N JOB_NAME
#PBS -A PROJECT_ID
-cd /scratch/work/user/$USER || exit
+cd /scratch/project/PROJECT_ID || exit
echo Time is `date`
echo Directory is `pwd`
@@ -130,21 +126,21 @@ echo '**PBS_NODEFILE***END*********'
text_nodes < cat $PBS_NODEFILE
-ml MATLAB
-ml COMSOL/51-EDU
+ml MATLAB/R2015b COMSOL/5.2.0-EDU
ntask=$(wc -l $PBS_NODEFILE)
-comsol -nn ${ntask} server -configuration /tmp -mpiarg -rmk -mpiarg pbs -tmpdir /scratch/work/user/$USER/work &
-cd /apps/cae/COMSOL/51/mli
-matlab -nodesktop -nosplash -r "mphstart; addpath /scratch/work/user/$USER/work; test_job"
+comsol -3drend sw -nn ${ntask} server -configuration /tmp -mpiarg -rmk -mpiarg pbs -tmpdir /scratch/project/PROJECT_ID &
+cd $EBROOTCOMSOL/mli
+matlab -nodesktop -nosplash -r "mphstart; addpath /scratch/project/PROJECT_ID; test_job"
```
-This example shows how to run LiveLink for MATLAB with the following configuration: 3 nodes and 16 cores per node. A working directory has to be created before submitting (comsol_matlab.pbs) job script into the queue. The input file (test_job.m) has to be in the working directory or a full path to the input file has to be specified. The Matlab command option (`-r ”mphstart”`) created a connection with a COMSOL server using the default port number.
+This example shows how to run LiveLink for MATLAB with the following configuration: 3 nodes and 36 cores per node. A working directory has to be created before submitting (comsol_matlab.pbs) job script into the queue. The input file (test_job.m) has to be in the working directory or a full path to the input file has to be specified. The Matlab command option (`-r ”mphstart”`) created a connection with a COMSOL server using the default port number.
[1]: licensing-and-available-versions.md
[2]: ../../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
[3]: ../../isv_licenses.md
+[4]: ../modules/lmod/
[a]: http://www.comsol.com
[b]: http://www.comsol.com/structural-mechanics-module
--
GitLab
From 27ba01c7aefea7475acfcf4bffc879fc8babf6b9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:01:43 +0200
Subject: [PATCH 07/58] Update comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index db2daadf8..a7d69694f 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -20,7 +20,7 @@ On the clusters, COMSOL is available in the latest stable version. There are two
* **Commercial** or so called **COM variant**, which can used also for commercial activities. **COM variant** has only subset of features compared to the **EDU variant** available. More about licensing [here][1].
-To load the default `COMSOL` module ([**not** recommended][4]), use:
+To load the default `COMSOL` module ([**not** recommended][4]), use:
```console
$ ml COMSOL
--
GitLab
From ea2022ed3be31733b93c36627f6d483dfc88370e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:03:04 +0200
Subject: [PATCH 08/58] Update comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index a7d69694f..0892d150a 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -140,7 +140,7 @@ This example shows how to run LiveLink for MATLAB with the following configurati
[1]: licensing-and-available-versions.md
[2]: ../../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
[3]: ../../isv_licenses.md
-[4]: ../modules/lmod/
+[4]: ../../../modules/lmod/
[a]: http://www.comsol.com
[b]: http://www.comsol.com/structural-mechanics-module
--
GitLab
From 884a1a459995c1a3bdbfdc0f504c12c3afde1ebb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:04:03 +0200
Subject: [PATCH 09/58] Update comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index 0892d150a..0d642ae53 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -43,7 +43,7 @@ Example for Barbora:
$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=36:mpiprocs=36
$ ml av COMSOL
------------------------------- /apps/modules/phys ------------------------------
+------------------------------ /apps/modules/phys -----------------------------
COMSOL/5.2.0-COM COMSOL/5.2.0-EDU (D)
$ ml COMSOL/5.2.0-EDU
--
GitLab
From 7926213e1c0ef9bec3bb4fc97149a907a0900b80 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:05:46 +0200
Subject: [PATCH 10/58] Update comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index 0d642ae53..eacd2054b 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -140,7 +140,7 @@ This example shows how to run LiveLink for MATLAB with the following configurati
[1]: licensing-and-available-versions.md
[2]: ../../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
[3]: ../../isv_licenses.md
-[4]: ../../../modules/lmod/
+[4]: ../../../modules/lmod/#loading-modules
[a]: http://www.comsol.com
[b]: http://www.comsol.com/structural-mechanics-module
--
GitLab
From 3832e7f61f872b8c110cf318c27d561fabdbce78 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:17:01 +0200
Subject: [PATCH 11/58] Update docs.it4i/software/chemistry/gaussian.md
---
docs.it4i/software/chemistry/gaussian.md | 27 ++++++++++++------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/docs.it4i/software/chemistry/gaussian.md b/docs.it4i/software/chemistry/gaussian.md
index 52f84971d..a0c682461 100644
--- a/docs.it4i/software/chemistry/gaussian.md
+++ b/docs.it4i/software/chemistry/gaussian.md
@@ -27,17 +27,20 @@ uid=1000(user) gid=1000(user) groups=1000(user),1234(open-0-0),7310(gaussian)
## Installed Version
-Gaussian is available on Salomon, Barbora, and DGX-2 systems in the latest version Gaussian 16 rev. c0.
-
-| Module | CPU support | GPU support | Parallelization | Note | Barbora | Salomon | DGX-2 |
-|--------------------------------------|-------------|--------------|-----------------|---------------------|---------|---------|-------|
-| Gaussian/16_rev_c0-binary | AVX2 | Yes | SMP | Binary distribution | Yes | No | Yes |
-| Gaussian/16_rev_c0-binary-Linda | AVX2 | Yes | SMP + Linda | Binary distribution | Yes | Yes | No |
-| Gaussian/16_rev_c0-CascadeLake | AVX-512 | No | SMP | IT4I compiled | Yes | No | No |
-| Gaussian/16_rev_c0-CascadeLake-Linda | AVX-512 | No | SMP + Linda | IT4I compiled | Yes | No | No |
-| Gaussian/16_rev_c0-GPU-Linda | AVX-512 | Yes | SMP + Linda | IT4I compiled | Yes | No | No |
-| Gaussian/16_rev_c0-GPU | AVX-512 | Yes | SMP | IT4I compiled | No | No | Yes |
-| Gaussian/16_rev_c0-Linda | AVX | No | SMP + Linda | IT4I compiled | No | No | No |
+Gaussian is available on Karolina, Salomon, Barbora, and DGX-2 systems in the latest version Gaussian 16 rev. c0 and versions 09-a02, a09-d01.
+
+| Module | CPU support | GPU support | Parallelization | Note | Barbora | Salomon | DGX-2 | Karolina |
+|--------------------------------------|-------------|--------------|-----------------|---------------------|---------|---------|-------| -------- |
+| Gaussian/09-a02 | AVX2 | No | SMP + Linda | IT4I compiled | Yes | No | No | Yes |
+| Gaussian/09-d01 | AVX2 | No | SMP + Linda | IT4I compiled | Yes | No | No | Yes |
+| Gaussian/16_rev_c0-binary | AVX2 | Yes | SMP | Binary distribution | Yes | No | Yes | No |
+| Gaussian/16_rev_c0-binary-Linda | AVX2 | Yes | SMP + Linda | Binary distribution | Yes | Yes | No | No |
+| Gaussian/16_rev_c0-CascadeLake | AVX-512 | No | SMP | IT4I compiled | Yes | No | No | No |
+| Gaussian/16_rev_c0-CascadeLake-Linda | AVX-512 | No | SMP + Linda | IT4I compiled | Yes | No | No | No |
+| Gaussian/16_rev_c0-GPU-Linda | AVX2 | Yes | SMP + Linda | IT4I compiled | Yes | No | No | No |
+| Gaussian/16_rev_c0-GPU | AVX2 | Yes | SMP | IT4I compiled | No | No | Yes | No |
+| Gaussian/16_rev_c0-Linda | AVX2 | No | SMP + Linda | IT4I compiled | No | No | No | Yes |
+| Gaussian/16_rev_c0 | AVX2 | No | SMP + Linda | IT4I compiled | No | No | No | Yes |
Speedup may be observed on Barbora and DGX-2 systems when using the `CascadeLake` and `GPU` modules compared to the `binary` module.
@@ -255,8 +258,6 @@ rm -rf $GAUSS_SCRDIR
exit
```
-[1]: ../../salomon/storage.md
-
[a]: https://gaussian.com/gaussian16/
[b]: mailto:support@it4i.cz
[c]: https://gaussian.com/man/
--
GitLab
From 5f16d7f69e2964397ab17d37b1ac609e1c70f117 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:26:50 +0200
Subject: [PATCH 12/58] Update docs.it4i/software/chemistry/molpro.md
---
docs.it4i/software/chemistry/molpro.md | 22 ++++++++++------------
1 file changed, 10 insertions(+), 12 deletions(-)
diff --git a/docs.it4i/software/chemistry/molpro.md b/docs.it4i/software/chemistry/molpro.md
index db4169a44..92b9fe558 100644
--- a/docs.it4i/software/chemistry/molpro.md
+++ b/docs.it4i/software/chemistry/molpro.md
@@ -32,40 +32,38 @@ Compilation parameters are default:
## Running
-MOLPRO is compiled for parallel execution using MPI and OpenMP. By default, MOLPRO reads the number of allocated nodes from PBS and launches a data server on one node. On the remaining allocated nodes, compute processes are launched, one process per node, each with 16 threads. You can modify this behavior by using the `-n`, `-t`, and `helper-server` options. For more details, see the [MOLPRO documentation][c].
+MOLPRO is compiled for parallel execution using MPI and OpenMP. By default, MOLPRO reads the number of allocated nodes from PBS and launches a data server on one node. On the remaining allocated nodes, compute processes are launched, one process per node. You can modify this behavior by using the `-n`, `-t`, and `helper-server` options. For more details, see the [MOLPRO documentation][c].
!!! note
- The OpenMP parallelization in MOLPRO is limited and has been observed to produce limited scaling. We therefore recommend to use MPI parallelization only. This can be achieved by passing the `mpiprocs=16:ompthreads=1` option to PBS.
+ The OpenMP parallelization in MOLPRO is limited and has been observed to produce limited scaling. We therefore recommend to use MPI parallelization only. This can be achieved by passing the `mpiprocs=128:ompthreads=1` option to PBS.
-You are advised to use the `-d` option to point to a directory in [SCRATCH file system - Salomon][1]. MOLPRO can produce a large amount of temporary data during its run, so it is important that these are placed in the fast scratch file system.
+You are advised to use the `-d` option to point to a directory in SCRATCH file system. MOLPRO can produce a large amount of temporary data during its run, so it is important that these are placed in the fast scratch file system.
### Example jobscript
```bash
#PBS -A IT4I-0-0
#PBS -q qprod
-#PBS -l select=1:ncpus=16:mpiprocs=16:ompthreads=1
+#PBS -l select=1:ncpus=128:mpiprocs=128:ompthreads=1
cd $PBS_O_WORKDIR
# load Molpro module
-module add molpro
+ml Molpro/mpp-2021.1.0.linux_x86_64_openmp
# create a directory in the SCRATCH filesystem
-mkdir -p /scratch/$USER/$PBS_JOBID
+mkdir -p /scratch/project/IT4I-0-0/$PBS_JOBID
+cd /scratch/project/IT4I-0-0/$PBS_JOBID
# copy an example input
-cp /apps/chem/molpro/2010.1/molprop_2010_1_Linux_x86_64_i8/examples/caffeine_opt_diis.com .
+cp $EBROOTMOLPRO/examples/caffeine_opt_diis.com .
# run Molpro with default options
-molpro -d /scratch/$USER/$PBS_JOBID caffeine_opt_diis.com
+molpro -d /scratch/project/IT4I-0-0/$PBS_JOBID caffeine_opt_diis.com
-# delete scratch directory
-rm -rf /scratch/$USER/$PBS_JOBID
+exit 0
```
-[1]: ../../salomon/storage.md
-
[a]: http://www.molpro.net/
[b]: https://www.molpro.net/licensee/?portal=licensee
[c]: http://www.molpro.net/info/2010.1/doc/manual/node9.html
--
GitLab
From b2279748476374a85bc40de8f6c3bfae2e13f602 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:34:00 +0200
Subject: [PATCH 13/58] Update docs.it4i/software/chemistry/nwchem.md
---
docs.it4i/software/chemistry/nwchem.md | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/docs.it4i/software/chemistry/nwchem.md b/docs.it4i/software/chemistry/nwchem.md
index 0fec079e0..129a4df24 100644
--- a/docs.it4i/software/chemistry/nwchem.md
+++ b/docs.it4i/software/chemistry/nwchem.md
@@ -14,15 +14,15 @@ $ ml av NWChem
## Running
-NWChem is compiled for parallel MPI execution. A standard procedure for MPI jobs applies. A sample jobscript (for Salomon on 24 threads) using a sample input file [h2o.nw][1]:
+NWChem is compiled for parallel MPI execution. A standard procedure for MPI jobs applies. A sample jobscript (for Karolina on 128 threads) using a sample input file [h2o.nw][1]:
```bash
#PBS -A IT4I-0-0
#PBS -q qprod
-#PBS -l select=1:ncpus=24:mpiprocs=24
+#PBS -l select=1:ncpus=128:mpiprocs=128
cd $PBS_O_WORKDIR
-ml NWChem
+ml NWChem/7.0.2-intel-2020a-Python-3.8.2-karolina
mpirun nwchem h2o.nw
```
@@ -31,10 +31,9 @@ mpirun nwchem h2o.nw
Refer to [the documentation][b] and set the following directives in the input file:
* MEMORY : controls the amount of memory NWChem will use
-* SCRATCH_DIR : set this to a directory in [SCRATCH filesystem - Salomon][2] (or run the calculation completely in a scratch directory). For certain calculations, it might be advisable to reduce I/O by forcing the `direct` mode, e.g. `scf direct`
+* SCRATCH_DIR : set this to a directory in SCRATCH filesystem (or run the calculation completely in a scratch directory). For certain calculations, it might be advisable to reduce I/O by forcing the `direct` mode, e.g. `scf direct`
[1]: ./files-nwchem/h2o.nw
-[2]: ../../salomon/storage.md
[a]: https://www.nwchem-sw.org/index.php/Main_Page
[b]: https://github.com/nwchemgit/nwchem/wiki
--
GitLab
From 6737c1e5d0dd3b9ee37498700e5bf6ef2543dc15 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:41:52 +0200
Subject: [PATCH 14/58] Update docs.it4i/software/chemistry/orca.md
---
docs.it4i/software/chemistry/orca.md | 43 ++++++++++++++--------------
1 file changed, 22 insertions(+), 21 deletions(-)
diff --git a/docs.it4i/software/chemistry/orca.md b/docs.it4i/software/chemistry/orca.md
index ca8cedd6d..9d3f6d033 100644
--- a/docs.it4i/software/chemistry/orca.md
+++ b/docs.it4i/software/chemistry/orca.md
@@ -26,17 +26,17 @@ You can test a serial computation with this simple input file. Create a file cal
*
```
-Next, create a PBS submission file (interactive job can be used too):
+Next, create a PBS submission file for Karolina cluster (interactive job can be used too):
```bash
#!/bin/bash
#PBS -S /bin/bash
#PBS -N ORCA_SERIAL
-#PBS -l select=1
+#PBS -l select=1:ncpus=128
#PBS -q qexp
#PBS -A OPEN-0-0
-ml ORCA/4.0.1.2
+ml ORCA/5.0.1-OpenMPI-4.1.1
orca orca_serial.inp
```
@@ -44,11 +44,11 @@ Submit the job to the queue and wait before it ends. Then you can find an output
```console
$ qsub submit_serial.pbs
-1417552.dm2
+1417552.infra-pbs
$ ll ORCA_SERIAL.*
--rw------- 1 hra0031 hra0031 0 Aug 21 12:24 ORCA_SERIAL.e1417552
--rw------- 1 hra0031 hra0031 20715 Aug 21 12:25 ORCA_SERIAL.o1417552
+-rw------- 1 user user 0 Aug 21 12:24 ORCA_SERIAL.e1417552
+-rw------- 1 user user 20715 Aug 21 12:25 ORCA_SERIAL.o1417552
$ cat ORCA_SERIAL.o1417552
@@ -71,7 +71,7 @@ $ cat ORCA_SERIAL.o1417552
#######################################################
- Program Version 4.0.1.2 - RELEASE -
+ Program Version 5.0.1 - RELEASE -
...
@@ -81,7 +81,7 @@ TOTAL RUN TIME: 0 days 0 hours 0 minutes 1 seconds 47 msec
## Running ORCA in Parallel
-Your serial computation can be easily converted to parallel. Simply specify the number of parallel processes by the `%pal` directive. In this example, 4 nodes, 16 cores each are used.
+Your serial computation can be easily converted to parallel. Simply specify the number of parallel processes by the `%pal` directive. In this example, 4 nodes, 128 cores each are used.
!!! warning
Do not use the `! PAL` directive as only PAL2 to PAL8 is recognized.
@@ -91,7 +91,7 @@ Your serial computation can be easily converted to parallel. Simply specify the
# https://orcaforum.cec.mpg.de/OrcaManual.pdf
! HF SVP
%pal
- nprocs 64 # 4 nodes, 16 cores each
+ nprocs 512 # 4 nodes, 128 cores each
end
* xyz 0 1
C 0 0 0
@@ -105,12 +105,12 @@ You also need to edit the previously used PBS submission file. You have to speci
#!/bin/bash
#PBS -S /bin/bash
#PBS -N ORCA_PARALLEL
-#PBS -l select=4:ncpus=16:mpiprocs=16
+#PBS -l select=4:ncpus=128:mpiprocs=128
#PBS -q qexp
#PBS -A OPEN-0-0
-ml ORCA/4.0.1.2
-/apps/all/ORCA/4.0.1.2/orca orca_parallel.inp > output.out
+ml ORCA/5.0.1-OpenMPI-4.1.1
+orca orca_parallel.inp > output.out
```
!!! note
@@ -120,11 +120,11 @@ Submit this job to the queue and see the output file.
```console
$ qsub submit_parallel.pbs
-1417598.dm2
+1417598.infra-pbs
$ ll ORCA_PARALLEL.*
--rw------- 1 hra0031 hra0031 0 Aug 21 13:12 ORCA_PARALLEL.e1417598
--rw------- 1 hra0031 hra0031 23561 Aug 21 13:13 ORCA_PARALLEL.o1417598
+-rw------- 1 user user 0 Aug 21 13:12 ORCA_PARALLEL.e1417598
+-rw------- 1 user user 23561 Aug 21 13:13 ORCA_PARALLEL.o1417598
$ cat ORCA_PARALLEL.o1417598
@@ -147,7 +147,7 @@ $ cat ORCA_PARALLEL.o1417598
#######################################################
- Program Version 4.0.1.2 - RELEASE -
+ Program Version 5.0.1 - RELEASE -
...
************************************************************
@@ -160,7 +160,7 @@ $ cat ORCA_PARALLEL.o1417598
TOTAL RUN TIME: 0 days 0 hours 0 minutes 11 seconds 859 msec
```
-You can see, that the program was running with 64 parallel MPI-processes. In version 4.0.1.2, only the following modules are parallelized:
+You can see, that the program was running with 512 parallel MPI-processes. In version 5.0.1, only the following modules are parallelized:
* ANOINT
* CASSCF / NEVPT2
@@ -182,9 +182,9 @@ You can see, that the program was running with 64 parallel MPI-processes. In ver
## Running ORCA 5.0.0 and Higher in Parallel
-On Barbora cluster, version 5.0.1 is available. However, to run it in parallel you need to specify execution nodes via `inputfilename.nodes` file. Additionally, all calculations **must** be run on SCRATCH.
+On Barbora cluster and Karolina cluster, version 5.0.1 is available. However, to run it in parallel you need to specify execution nodes via `inputfilename.nodes` file. Additionally, all calculations **must** be run on SCRATCH.
-Example submission script would look like this:
+Example submission script (Karolina cluster) would look like this:
```
#!/bin/bash
@@ -192,7 +192,7 @@ Example submission script would look like this:
#PBS -A OPEN-00-00
#PBS -N jobname
#PBS -q qprod
-#PBS -l select=2
+#PBS -l select=2:ncpus=128:mpiprocs=128
#PBS -l walltime=00:05:00
ml purge
@@ -240,7 +240,8 @@ cp -r $PBS_O_WORKDIR/* .
# copy output files to home, delete the rest
cp * $PBS_O_WORKDIR/ && cd $PBS_O_WORKDIR
rm -rf $SCRDIR
-exit
+
+exit 0
```
## Register as a User
--
GitLab
From b66738881b41eb4c445d99225670cc2e8f5624a2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:43:53 +0200
Subject: [PATCH 15/58] Update docs.it4i/software/chemistry/vasp.md
---
docs.it4i/software/chemistry/vasp.md | 5 -----
1 file changed, 5 deletions(-)
diff --git a/docs.it4i/software/chemistry/vasp.md b/docs.it4i/software/chemistry/vasp.md
index f780b36cc..43e7de234 100644
--- a/docs.it4i/software/chemistry/vasp.md
+++ b/docs.it4i/software/chemistry/vasp.md
@@ -33,14 +33,9 @@ VASP can be ran using several different binaries, each being compiled for a spec
* `vasp_std` is the standard VASP binary for regular jobs
* `vasp_ncl` must be used when running non-collinear (spin-orbit coupling) jobs
* `vasp_gam` is optimized for calculations at the Gamma point
-
-<!---
-
* `vasp_gpu_std` is a compilation for GPU calculations
* `vasp_gpu_ncl` has to be used when running a non-collinear job on GPU
--->
-
## Tutorials and Examples
Tutorials and examples can be found at the [offical VASP wiki](https://www.vasp.at/wiki/index.php/The_VASP_Manual), specifically in the sections [Tutorials](https://www.vasp.at/wiki/index.php/Category:Tutorials) and [Examples](https://www.vasp.at/wiki/index.php/Category:Examples). These should provide you with enough information to write and run your own jobs.
--
GitLab
From 77de965029e1bd83ade15f22f218755425c55586 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:46:59 +0200
Subject: [PATCH 16/58] Update mkdocs.yml
---
mkdocs.yml | 1 -
1 file changed, 1 deletion(-)
diff --git a/mkdocs.yml b/mkdocs.yml
index 7f5e9f59b..ae9be3b89 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -146,7 +146,6 @@ nav:
- Dask: software/data-science/dask.md
- Debuggers:
- Introduction: software/debuggers/introduction.md
- - Aislinn: software/debuggers/aislinn.md
- Allinea Forge (DDT,MAP): software/debuggers/allinea-ddt.md
- Allinea Performance Reports: software/debuggers/allinea-performance-reports.md
- CUBE: software/debuggers/cube.md
--
GitLab
From 8e8295f772812cdc766595ce4702ecb17da8951b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:47:07 +0200
Subject: [PATCH 17/58] Deleted docs.it4i/software/debuggers/aislinn.md
---
docs.it4i/software/debuggers/aislinn.md | 111 ------------------------
1 file changed, 111 deletions(-)
delete mode 100644 docs.it4i/software/debuggers/aislinn.md
diff --git a/docs.it4i/software/debuggers/aislinn.md b/docs.it4i/software/debuggers/aislinn.md
deleted file mode 100644
index 2e418091f..000000000
--- a/docs.it4i/software/debuggers/aislinn.md
+++ /dev/null
@@ -1,111 +0,0 @@
-# Aislinn
-
-## Introduction
-
-Aislinn is a dynamic verifier for MPI programs. For a fixed input, it covers all possible runs with respect to nondeterminism introduced by MPI. It allows to detect bugs that very rarely occur in normal runs. It detects problems like invalid memory accesses, deadlocks, misuse of MPI, and resource leaks. Aislinn is an open-source software, so you can use it without any licensing limitations. For more information, visit the project's [web page][a].
-
-!!! note
- Aislinn is developed at IT4Innovations and some parts are still considered experimental. If you have any questions or experience any problems, contact the author: <mailto:stanislav.bohm@vsb.cz>.
-
-## Installed Versions
-
-For the current list of installed versions, use:
-
-```console
-$ ml av Aislinn
-```
-
-## Usage
-
-Let us have the following program that contains a bug that is not manifested in all runs:
-
-```cpp
-#include <mpi.h>
-#include <stdlib.h>
-
-int main(int argc, char **argv) {
- int rank;
-
- MPI_Init(&argc, &argv);
- MPI_Comm_rank(MPI_COMM_WORLD, &rank);
-
- if (rank == 0) {
- int *mem1 = (int*) malloc(sizeof(int) * 2);
- int *mem2 = (int*) malloc(sizeof(int) * 3);
- int data;
- MPI_Recv(&data, 1, MPI_INT, MPI_ANY_SOURCE, 1,
- MPI_COMM_WORLD, MPI_STATUS_IGNORE);
- mem1[data] = 10; // <---------- Possible invalid memory write
- MPI_Recv(&data, 1, MPI_INT, MPI_ANY_SOURCE, 1,
- MPI_COMM_WORLD, MPI_STATUS_IGNORE);
- mem2[data] = 10;
- free(mem1);
- free(mem2);
- }
-
- if (rank == 1 || rank == 2) {
- MPI_Send(&rank, 1, MPI_INT, 0, 1, MPI_COMM_WORLD);
- }
-
- MPI_Finalize();
- return 0;
-}
-```
-
-The program does the following: process 0 receives two messages from anyone and processes 1 and 2 send a message to process 0. If a message from process 1 is received first, then the run does not expose the error. If a message from process 2 is received first, then invalid memory write occurs at line 16.
-
-To verify this program by Aislinn, we first load Aislinn itself:
-
-```console
-$ ml aislinn
-```
-
-Now we compile the program by Aislinn implementation of MPI. There are `mpicc` for C programs and `mpicxx` for C++ programs. Only MPI parts of the verified application has to be recompiled; non-MPI parts may remain untouched. Let us assume that our program is in `test.cpp`.
-
-```console
-$ mpicc -g test.cpp -o test
-```
-
-The `-g` flag is not necessary, but it puts more debugging information into the program, hence Aislinn may provide a more detailed report. The command produces an executable file `test`.
-
-Now we run the Aislinn itself. The argument `-p 3` specifies that we want to verify our program for the case of three MPI processes:
-
-```console
-$ aislinn -p 3 ./test
-==AN== INFO: Aislinn v0.3.0
-==AN== INFO: Found error 'Invalid write'
-==AN== INFO: 1 error(s) found
-==AN== INFO: Report written into 'report.html'
-```
-
-Aislinn found an error and produced an HTML report. To view it, we can use any browser, for example:
-
-```console
-$ firefox report.html
-```
-
-At the beginning of the report, there are some basic summaries of the verification. In the second part (depicted in the following picture), the error is described.
-
-
-
-It shows us:
-
-* Error occurs in process 0 in `test.cpp` on line 16.
-* Stdout and stderr streams are empty. (The program does not write anything).
-* The last part shows MPI calls for each process that occurs in the invalid run. More detailed information about each call can be obtained by hovering the mouse cursor over the respective MPI call.
-
-### Limitations
-
-Since the verification is a non-trivial process, there are some of limitations.
-
-* The verified process has to terminate in all runs, i.e. we cannot answer the halting problem.
-* The verification is a computationally and memory demanding process. We put an effort to make it efficient and it is an important point for further research. However covering all runs will always be more demanding than techniques that examine only a single run. The good practice is to start with small instances and when it is feasible, make them bigger. Aislinn is good for finding bugs that are hard to find because they occur very rarely (only in a rare scheduling). Such bugs often do not need big instances.
-* Aislinn expects that your program is a "standard MPI" program, i.e. processes communicate only through MPI, the verified program does not interacts with the system in some unusual ways (e.g. opening sockets).
-
-There are also some limitations bounded to the current version and they will be removed in the future:
-
-* All files containing MPI calls have to be recompiled by MPI implementation provided by Aislinn. The files that do not contain MPI calls, do not have to be recompiled. Aislinn MPI implementation supports many commonly used calls from MPI-2 and MPI-3 related to point-to-point communication, collective communication, and communicator management. Unfortunately, MPI-IO and one-side communication is not implemented yet.
-* Each MPI can use only one thread (if you use OpenMP, set `OMP_NUM_THREADS` to 1).
-* There are some limitations for using files, but if the program just reads inputs and writes results, it is OK.
-
-[a]: http://verif.cs.vsb.cz/aislinn/
--
GitLab
From e95fb930afb71fba2008ba01bb83cf294a408344 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:53:00 +0200
Subject: [PATCH 18/58] Update docs.it4i/software/debuggers/allinea-ddt.md
---
docs.it4i/software/debuggers/allinea-ddt.md | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/docs.it4i/software/debuggers/allinea-ddt.md b/docs.it4i/software/debuggers/allinea-ddt.md
index ddd754850..aaeaff2d0 100644
--- a/docs.it4i/software/debuggers/allinea-ddt.md
+++ b/docs.it4i/software/debuggers/allinea-ddt.md
@@ -33,6 +33,9 @@ Load the Allinea DDT module:
$ ml Forge
```
+!!! note
+ Loading default modules is [**not** recommended][2].
+
Compile the code:
```console
@@ -54,15 +57,15 @@ Before debugging, you need to compile your code with these flags:
Be sure to log in with an X window forwarding enabled. This could mean using the `-X` in the `ssh`:
```console
-$ ssh -X username@anselm.it4i.cz
+$ ssh -X username@login1.karolina.it4i.cz
```
-Another options is to access the login node using VNC. See the detailed information on how to [use graphic user interface on Anselm][1].
+Another options is to access the login node using VNC. See the detailed information on how to [use graphic user interface][1].
From the login node, an interactive session **with X windows forwarding** (the `-X` option) can be started by the following command:
```console
-$ qsub -I -X -A NONE-0-0 -q qexp -lselect=1:ncpus=16:mpiprocs=16,walltime=01:00:00
+$ qsub -I -X -A NONE-0-0 -q qexp -l select=1:ncpus=128:mpiprocs=128,walltime=01:00:00
```
Then launch the debugger with the `ddt` command followed by the name of the executable to debug:
@@ -92,5 +95,6 @@ $DDTPATH/doc/userguide.pdf
[1] Discipline, Magic, Inspiration and Science: Best Practice Debugging with Allinea DDT, Workshop conducted at LLNL by Allinea on May 10, 2013, [link][a]
[1]: ../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
+[2]: ../../../modules/lmod/#loading-modules
[a]: https://computing.llnl.gov/tutorials/allineaDDT/index.html
--
GitLab
From 368c1ff5e3ef289f4f7a483b59f34dc3d3abb6b4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 11:59:48 +0200
Subject: [PATCH 19/58] Update
docs.it4i/software/debuggers/allinea-performance-reports.md
---
.../software/debuggers/allinea-performance-reports.md | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/docs.it4i/software/debuggers/allinea-performance-reports.md b/docs.it4i/software/debuggers/allinea-performance-reports.md
index 9af12e8da..81d6f3cfe 100644
--- a/docs.it4i/software/debuggers/allinea-performance-reports.md
+++ b/docs.it4i/software/debuggers/allinea-performance-reports.md
@@ -13,7 +13,7 @@ Our license is limited to 64 MPI processes.
For the current list of installed versions, use:
```console
-$ ml av PerformanceReports
+$ ml av Forge
```
The module sets up environment variables, required for using the Allinea Performance Reports.
@@ -35,15 +35,15 @@ In this example, we will be profiling the mympiprog.x MPI program, using Allinea
First, we allocate nodes via the express queue:
```console
-$ qsub -q qexp -l select=2:ppn=24:mpiprocs=24:ompthreads=1 -I
- qsub: waiting for job 262197.dm2 to start
- qsub: job 262197.dm2 ready
+$ qsub -q qexp -l select=2:ncpus=128:mpiprocs=128:ompthreads=1 -I
+ qsub: waiting for job 262197.infra-pbs to start
+ qsub: job 262197.infra-pbs ready
```
Then we load the modules and run the program the usual way:
```console
-$ ml intel
+$ ml intel/2020b Forge/20.1.1
$ ml PerfReports/6.0
$ mpirun ./mympiprog.x
```
--
GitLab
From e9e8a45ea1359f44afdc147c01a499a794da5298 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:13:33 +0200
Subject: [PATCH 20/58] Update mkdocs.yml
---
mkdocs.yml | 1 -
1 file changed, 1 deletion(-)
diff --git a/mkdocs.yml b/mkdocs.yml
index ae9be3b89..0a6aed012 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -149,7 +149,6 @@ nav:
- Allinea Forge (DDT,MAP): software/debuggers/allinea-ddt.md
- Allinea Performance Reports: software/debuggers/allinea-performance-reports.md
- CUBE: software/debuggers/cube.md
- - Intel Performance Counter Monitor: software/debuggers/intel-performance-counter-monitor.md
- Intel VTune Amplifier XE: software/debuggers/intel-vtune-amplifier.md
- Intel VTune Profiler: software/debuggers/intel-vtune-profiler.md
- PAPI: software/debuggers/papi.md
--
GitLab
From b6e703e9fbd05c087c9819e88b2d980b191e5db9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:18:06 +0200
Subject: [PATCH 21/58] Update
docs.it4i/software/debuggers/intel-vtune-amplifier.md
---
.../debuggers/intel-vtune-amplifier.md | 39 ++-----------------
1 file changed, 4 insertions(+), 35 deletions(-)
diff --git a/docs.it4i/software/debuggers/intel-vtune-amplifier.md b/docs.it4i/software/debuggers/intel-vtune-amplifier.md
index e1322d330..629c62806 100644
--- a/docs.it4i/software/debuggers/intel-vtune-amplifier.md
+++ b/docs.it4i/software/debuggers/intel-vtune-amplifier.md
@@ -24,7 +24,7 @@ $ ml av VTune
To profile an application with VTune Amplifier, special kernel modules need to be loaded. The modules are not loaded on the login nodes, thus direct profiling on login nodes is not possible. By default, the kernel modules are not loaded on compute nodes either. In order to have the modules loaded, you need to specify the `vtune=version` PBS resource at job submit. The version is the same as for the environment module. For example, for VTune/2016_update1, use:
```console
-$ qsub -q qexp -A OPEN-0-0 -I -l select=1,vtune=2016_update1
+$ qsub -q qexp -A OPEN-0-0 -I -l select=1,vtune=2020_update3
```
After that, you can verify the modules `sep*`, `pax`, and `vtsspp` are present in the kernel:
@@ -39,13 +39,13 @@ $ lsmod | grep -e sep -e pax -e vtsspp
To launch the GUI, first load the module:
```console
-$ module add VTune/2016_update1
+$ ml VTune/2020_update3
```
and launch the GUI:
```console
-$ amplxe-gui
+$ $ amplxe-gui
```
The GUI will open in a new window. Click on *New Project...* to create a new project. After clicking OK, a new window with project properties will appear. At *Application:*, select the path to the binary you want to profile (the binary should be compiled with the `-g` flag). Some additional options, such as command line arguments, can be selected. At *Managed code profiling mode:* select `Native` (unless you want to profile managed mode .NET/Mono applications). After clicking OK, your project is created.
@@ -59,42 +59,11 @@ VTune Amplifier also allows a form of remote analysis. In this mode, data for an
The command line will look like this:
```console
-/apps/all/VTune/2016_update1/vtune_amplifier_xe_2016.1.1.434111/bin64/amplxe-cl -collect advanced-hotspots -app-working-dir /home/sta545/tmp -- /home/sta545/tmp/sgemm
+amplxe-cl -collect advanced-hotspots -app-working-dir /home/$USER/tmp -- /home/$USER/tmp/sgemm
```
Copy the line to clipboard and then you can paste it in your jobscript or in the command line. After the collection is run, open the GUI once again, click the menu button in the upper right corner, and select *Open > Result...*. The GUI will load the results from the run.
-## Xeon Phi
-
-It is possible to analyze both native and offloaded Xeon Phi applications.
-
-### Native Mode
-
-This mode is useful for native Xeon Phi applications launched directly on the card. In the *Analysis Target* window, select *Intel Xeon Phi coprocessor (native)*, choose the path to the binary and MIC card to run on.
-
-### Offload Mode
-
-This mode is useful for applications that are launched from the host and use offload, OpenCL, or mpirun. In the *Analysis Target* window, select *Intel Xeon Phi coprocessor (native)*, choose the path to the binary and the MIC card to run on.
-
-!!! note
- If the analysis is interrupted or aborted, a further analysis on the card might be impossible and you will get errors like *ERROR connecting to MIC card*. In this case, contact our support to reboot the MIC card.
-
-You may also use remote analysis to collect data from the MIC and then analyze it in the GUI later:
-
-Native launch:
-
-```console
-$ /apps/all/VTune/2016_update1/vtune_amplifier_xe_2016.1.1.434111/bin64/amplxe-cl -target-system mic-native:0 -collect advanced-hotspots -- /home/sta545/tmp/vect-add-mic
-```
-
-Host launch:
-
-```console
-$ /apps/all/VTune/2016_update1/vtune_amplifier_xe_2016.1.1.434111/bin64/amplxe-cl -target-system mic-host-launch:0 -collect advanced-hotspots -- /home/sta545/tmp/sgemm
-```
-
-You can obtain this command line by pressing the *Command line...* button on the Analysis Type screen.
-
## References
1. [Intel® VTune™ Amplifier Support][a]
--
GitLab
From fccb745ae723eac925e947ccb9e37fd8d62bf747 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:20:28 +0200
Subject: [PATCH 22/58] Update
docs.it4i/software/debuggers/intel-vtune-profiler.md
---
docs.it4i/software/debuggers/intel-vtune-profiler.md | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/docs.it4i/software/debuggers/intel-vtune-profiler.md b/docs.it4i/software/debuggers/intel-vtune-profiler.md
index ce496004b..dccd872ad 100644
--- a/docs.it4i/software/debuggers/intel-vtune-profiler.md
+++ b/docs.it4i/software/debuggers/intel-vtune-profiler.md
@@ -19,16 +19,16 @@ $ ml av VTune
## Usage
-To profile an application with VTune Profiler, special kernel modules need to be loaded. The modules are not loaded on the login nodes, thus direct profiling on login nodes is not possible. By default, the kernel modules are not loaded on compute nodes either. In order to have the modules loaded, you need to specify the `vtune=version` PBS resource at job submit. The version is the same as for the environment module. For example, to use VTune/2020_update2-GCC:
+To profile an application with VTune Profiler, special kernel modules need to be loaded. The modules are not loaded on the login nodes, thus direct profiling on login nodes is not possible. By default, the kernel modules are not loaded on compute nodes either. In order to have the modules loaded, you need to specify the `vtune=version` PBS resource at job submit. The version is the same as for the environment module. For example, to use VTune/2020_update3:
```console
-$ qsub -q qexp -A OPEN-0-0 -I -l select=1,vtune=2020_update2-GCC
+$ qsub -q qexp -A OPEN-0-0 -I -l select=1,vtune=2020_update3
```
After that, you can verify that the modules `sep*`, `pax`, and `vtsspp` are present in the kernel:
```console
-login@cn191:~$ lsmod | grep -e sep -e pax -e vtsspp
+$ lsmod | grep -e sep -e pax -e vtsspp
vtsspp 367418 0
sep5 885333 0
socperf3 595104 2 sep5,socwatch2_12
@@ -38,7 +38,7 @@ pax 13820 0
To launch the GUI, first load the module:
```console
-$ ml VTune/2020_update2-GCC
+$ ml VTune/2020_update3
```
and launch the GUI:
@@ -61,7 +61,7 @@ VTune Profiler also allows a form of remote analysis. In this mode, data for ana
The command line will look like this:
```console
-/apps/all/VTune/2020_update2-GCC/vtune_profiler_2020.2.0.610396/bin64/vtune -collect hotspots -app-working-dir /home/$USER/tmp -- /home/$USER/tmp/sgemm
+vtune -collect hotspots -app-working-dir /home/$USER/tmp -- /home/$USER/tmp/sgemm
```
!!! Warning
--
GitLab
From 9a7e5566f6e0dbc06e694ffc319a3dc52a86dbbf Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:25:13 +0200
Subject: [PATCH 23/58] Update docs.it4i/software/debuggers/introduction.md
---
docs.it4i/software/debuggers/introduction.md | 24 ++++----------------
1 file changed, 5 insertions(+), 19 deletions(-)
diff --git a/docs.it4i/software/debuggers/introduction.md b/docs.it4i/software/debuggers/introduction.md
index e74d87bf6..ab3f2f33a 100644
--- a/docs.it4i/software/debuggers/introduction.md
+++ b/docs.it4i/software/debuggers/introduction.md
@@ -4,25 +4,12 @@
We provide state of the art programs and tools to develop, profile, and debug HPC codes at IT4Innovations. In this cestion, we provide an overview of the profiling and debugging tools available on IT4I clusters.
-## Intel Debugger
-
-The Intel debugger is no longer available since Parallel Studio version 2015
-
-The Intel debugger version 13.0 is available, via the intel module. The debugger works for applications compiled with C and C++ compiler and the ifort fortran 77/90/95 compiler. The debugger provides a Java GUI environment.
-
-```console
-$ ml intel
-$ idb
-```
-
-Read more at the [Intel Debugger][1] page.
-
## Allinea Forge (DDT/MAP)
Allinea DDT is a commercial debugger primarily for debugging parallel MPI or OpenMP programs. It also has a support for GPU (CUDA) and Intel Xeon Phi accelerators. DDT provides all the standard debugging features (stack trace, breakpoints, watches, view variables, threads, etc.) for every thread running as part of your program, or for every process - even if these processes are distributed across a cluster using an MPI implementation.
```console
-$ ml Forge
+$ ml Forge/20.0.1
$ forge
```
@@ -33,7 +20,7 @@ Read more at the [Allinea DDT][2] page.
Allinea Performance Reports characterize the performance of HPC application runs. After executing your application through the tool, a synthetic HTML report is generated automatically, containing information about several metrics along with clear behavior statements and hints to help you improve the efficiency of your runs. Our license is limited to 64 MPI processes.
```console
-$ ml PerformanceReports/6.0
+$ ml Forge/20.0.1
$ perf-report mpirun -n 64 ./my_application argument01 argument02
```
@@ -44,7 +31,7 @@ Read more at the [Allinea Performance Reports][3] page.
TotalView is a source- and machine-level debugger for multi-process, multi-threaded programs. Its wide range of tools provides ways to analyze, organize, and test programs, making it easy to isolate and identify problems in individual threads and processes in programs of great complexity.
```console
-$ ml TotalView/8.15.4-6-linux-x86-64
+$ ml TotalView/2021.2.14
$ totalview
```
@@ -55,13 +42,12 @@ Read more at the [Totalview][4] page.
Vampir is a GUI trace analyzer for traces in OTF format.
```console
- $ ml Vampir/8.5.0
- $ vampir
+$ ml Vampir/9.9.0
+$ vampir
```
Read more at the [Vampir][5] page.
-[1]: ../intel/intel-suite/intel-debugger.md
[2]: allinea-ddt.md
[3]: allinea-performance-reports.md
[4]: total-view.md
--
GitLab
From 8190d02a3382696fa4614def4b875272b597a689 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:36:13 +0200
Subject: [PATCH 24/58] Update docs.it4i/software/debuggers/papi.md
---
docs.it4i/software/debuggers/papi.md | 149 ++++++++++++++-------------
1 file changed, 78 insertions(+), 71 deletions(-)
diff --git a/docs.it4i/software/debuggers/papi.md b/docs.it4i/software/debuggers/papi.md
index a69f25dad..ded12014f 100644
--- a/docs.it4i/software/debuggers/papi.md
+++ b/docs.it4i/software/debuggers/papi.md
@@ -24,40 +24,89 @@ The bin directory of PAPI, which is automatically added to $PATH upon loading th
Prints which preset events are available on the current CPU. The third column indicates whether the preset event is available on the current CPU.
+* Barbora cluster
+
```console
$ papi_avail
- Available events and hardware information.
- --------------------------------------------------------------------------------
- PAPI Version : 5.3.2.0
- Vendor string and code : GenuineIntel (1)
- Model string and code : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (45)
- CPU Revision : 7.000000
- CPUID Info : Family: 6 Model: 45 Stepping: 7
- CPU Max Megahertz : 2601
- CPU Min Megahertz : 1200
- Hdw Threads per core : 1
- Cores per Socket : 8
- Sockets : 2
- NUMA Nodes : 2
- CPUs per Node : 8
- Total CPUs : 16
- Running in a VM : no
- Number Hardware Counters : 11
- Max Multiplex Counters : 32
- --------------------------------------------------------------------------------
- Name Code Avail Deriv Description (Note)
- PAPI_L1_DCM 0x80000000 Yes No Level 1 data cache misses
- PAPI_L1_ICM 0x80000001 Yes No Level 1 instruction cache misses
- PAPI_L2_DCM 0x80000002 Yes Yes Level 2 data cache misses
- PAPI_L2_ICM 0x80000003 Yes No Level 2 instruction cache misses
- PAPI_L3_DCM 0x80000004 No No Level 3 data cache misses
- PAPI_L3_ICM 0x80000005 No No Level 3 instruction cache misses
- PAPI_L1_TCM 0x80000006 Yes Yes Level 1 cache misses
- PAPI_L2_TCM 0x80000007 Yes No Level 2 cache misses
- PAPI_L3_TCM 0x80000008 Yes No Level 3 cache misses
+Available PAPI preset and user defined events plus hardware information.
+--------------------------------------------------------------------------------
+PAPI version : 6.0.0.0
+Operating system : Linux 3.10.0-1160.6.1.el7.x86_64
+Vendor string and code : GenuineIntel (1, 0x1)
+Model string and code : Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz (85, 0x55)
+CPU revision : 7.000000
+CPUID : Family/Model/Stepping 6/85/7, 0x06/0x55/0x07
+CPU Max MHz : 3900
+CPU Min MHz : 1000
+Total cores : 72
+SMT threads per core : 2
+Cores per socket : 18
+Sockets : 2
+Cores per NUMA region : 36
+NUMA regions : 2
+Running in a VM : no
+Number Hardware Counters : 10
+Max Multiplex Counters : 384
+Fast counter read (rdpmc): no
+--------------------------------------------------------------------------------
+
+================================================================================
+ PAPI Preset Events
+================================================================================
+ Name Code Avail Deriv Description (Note)
+PAPI_L1_DCM 0x80000000 Yes No Level 1 data cache misses
+PAPI_L1_ICM 0x80000001 Yes No Level 1 instruction cache misses
+PAPI_L2_DCM 0x80000002 Yes Yes Level 2 data cache misses
+PAPI_L2_ICM 0x80000003 Yes No Level 2 instruction cache misses
+PAPI_L3_DCM 0x80000004 No No Level 3 data cache misses
+PAPI_L3_ICM 0x80000005 No No Level 3 instruction cache misses
+PAPI_L1_TCM 0x80000006 Yes Yes Level 1 cache misses
+PAPI_L2_TCM 0x80000007 Yes No Level 2 cache misses
+PAPI_L3_TCM 0x80000008 Yes No Level 3 cache misses
....
```
+* Karolina cluster
+
+```console
+Available PAPI preset and user defined events plus hardware information.
+--------------------------------------------------------------------------------
+PAPI version : 6.0.0.0
+Operating system : Linux 3.10.0-1160.21.1.el7.x86_64
+Vendor string and code : AuthenticAMD (2, 0x2)
+Model string and code : AMD EPYC 7H12 64-Core Processor (49, 0x31)
+CPU revision : 0.000000
+CPUID : Family/Model/Stepping 23/49/0, 0x17/0x31/0x00
+CPU Max MHz : 2594
+CPU Min MHz : 2594
+Total cores : 128
+SMT threads per core : 1
+Cores per socket : 64
+Sockets : 2
+Cores per NUMA region : 16
+NUMA regions : 8
+Running in a VM : no
+Number Hardware Counters : 5
+Max Multiplex Counters : 384
+Fast counter read (rdpmc): no
+--------------------------------------------------------------------------------
+
+================================================================================
+ PAPI Preset Events
+================================================================================
+ Name Code Avail Deriv Description (Note)
+PAPI_L1_DCM 0x80000000 No No Level 1 data cache misses
+PAPI_L1_ICM 0x80000001 No No Level 1 instruction cache misses
+PAPI_L2_DCM 0x80000002 No No Level 2 data cache misses
+PAPI_L2_ICM 0x80000003 No No Level 2 instruction cache misses
+PAPI_L3_DCM 0x80000004 No No Level 3 data cache misses
+PAPI_L3_ICM 0x80000005 No No Level 3 instruction cache misses
+PAPI_L1_TCM 0x80000006 No No Level 1 cache misses
+PAPI_L2_TCM 0x80000007 No No Level 2 cache misses
+PAPI_L3_TCM 0x80000008 No No Level 3 cache misses
+...
+```
+
### Papi_native_avail
Prints which native events are available on the current CPU.
@@ -186,48 +235,6 @@ $ ./matrix
MFLOPS: 2127.529297
```
-### Intel Xeon Phi
-
-!!! note
- PAPI currently supports only a subset of counters on the Intel Xeon Phi processor compared to Intel Xeon, for example the floating point operations counter is missing.
-
-To use PAPI in [Intel Xeon Phi][2] native applications, you need to load the module with the `-mic` suffix, for example `papi/5.3.2-mic`:
-
-```console
-$ ml papi/5.3.2-mic
-```
-
-Then, compile your application in the following way:
-
-```console
-$ ml intel
-$ icc -mmic -Wl,-rpath,/apps/intel/composer_xe_2013.5.192/compiler/lib/mic matrix-mic.c -o matrix-mic -lpapi -lpfm
-```
-
-To execute the application on MIC, you need to manually set `LD_LIBRARY_PATH`:
-
-```console
-$ qsub -q qmic -A NONE-0-0 -I
-$ ssh mic0
-$ export LD_LIBRARY_PATH="/apps/tools/papi/5.4.0-mic/lib/"
-$ ./matrix-mic
-```
-
-Alternatively, you can link PAPI statically (`-static` flag), then `LD_LIBRARY_PATH` does not need to be set.
-
-You can also execute the PAPI tools on MIC :
-
-```console
-$ /apps/tools/papi/5.4.0-mic/bin/papi_native_avail
-```
-
-To use PAPI in offload mode, you need to provide both the host and the MIC versions of PAPI:
-
-```console
-$ ml papi/5.4.0
-$ icc matrix-offload.c -o matrix-offload -offload-option,mic,compiler,"-L$PAPI_HOME-mic/lib -lpapi" -lpapi
-```
-
## References
1. [Main project page](http://icl.cs.utk.edu/papi/)
--
GitLab
From 1999d006490a0297b3dca65dd666eaebb070e524 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:42:32 +0200
Subject: [PATCH 25/58] Update docs.it4i/software/debuggers/scalasca.md
---
docs.it4i/software/debuggers/scalasca.md | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/docs.it4i/software/debuggers/scalasca.md b/docs.it4i/software/debuggers/scalasca.md
index c800343d0..a540d8afc 100644
--- a/docs.it4i/software/debuggers/scalasca.md
+++ b/docs.it4i/software/debuggers/scalasca.md
@@ -35,7 +35,7 @@ After the application is instrumented, runtime measurement can be performed with
An example:
```console
- $ scalasca -analyze mpirun -np 4 ./mympiprogram
+$ scalasca -analyze mpirun -np 4 ./mympiprogram
```
Some notable Scalasca options are:
@@ -44,27 +44,27 @@ Some notable Scalasca options are:
* `-e <directory>` specifies a directory to which the collected data is saved. By default, Scalasca saves the data to a directory with the scorep\_ prefix, followed by the name of the executable and the launch configuration.
!!! note
- Scalasca can generate a huge amount of data, especially if tracing is enabled. Consider saving the data to a [scratch directory][6].
+ Scalasca can generate a huge amount of data, especially if tracing is enabled. Consider saving the data to a scratch directory.
### Analysis of Reports
-For the analysis, you must have the [Score-P][5] and [CUBE][7] modules loaded. The analysis is done in two steps. First, the data is preprocessed and then, the CUBE GUI tool is launched.
+For the analysis, you must have the [Score-P][5] and [CUBE][6] modules loaded. The analysis is done in two steps. First, the data is preprocessed and then, the CUBE GUI tool is launched.
To launch the analysis, run:
```console
-scalasca -examine [options] <experiment_directory>
+$ scalasca -examine [options] <experiment_directory>
```
If you do not wish to launch the GUI tool, use the `-s` option:
```console
-scalasca -examine -s <experiment_directory>
+$ scalasca -examine -s <experiment_directory>
```
Alternatively, you can open CUBE and load the data directly from here. Keep in mind that in this case, the pre-processing is not done and not all metrics will be shown in the viewer.
-Refer to the [CUBE documentation][7] on usage of the GUI viewer.
+Refer to the [CUBE documentation][6] on usage of the GUI viewer.
## References
@@ -75,7 +75,6 @@ Refer to the [CUBE documentation][7] on usage of the GUI viewer.
[3]: ../mpi/running_openmpi.md
[4]: ../mpi/running-mpich2.md
[5]: score-p.md
-[6]: ../../salomon/storage.md
-[7]: cube.md
+[6]: cube.md
[a]: http://www.scalasca.org/
--
GitLab
From 90b52f4f3d6f2b3e5b1b7a7db38bc2756383ba7c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:45:12 +0200
Subject: [PATCH 26/58] Update docs.it4i/software/debuggers/score-p.md
---
docs.it4i/software/debuggers/score-p.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/debuggers/score-p.md b/docs.it4i/software/debuggers/score-p.md
index 658d5ba6f..5c0adff6c 100644
--- a/docs.it4i/software/debuggers/score-p.md
+++ b/docs.it4i/software/debuggers/score-p.md
@@ -62,7 +62,7 @@ void foo()
}
```
- and Fortran:
+and Fortran:
```fortran
#include "scorep/SCOREP_User.inc"
--
GitLab
From 5c57b37ea12a8d1b8a3fcda50aae07799ec26ee0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:51:16 +0200
Subject: [PATCH 27/58] Update docs.it4i/software/debuggers/total-view.md
---
docs.it4i/software/debuggers/total-view.md | 25 +++++++++++-----------
1 file changed, 12 insertions(+), 13 deletions(-)
diff --git a/docs.it4i/software/debuggers/total-view.md b/docs.it4i/software/debuggers/total-view.md
index e62f19ddf..cfaf1a8c7 100644
--- a/docs.it4i/software/debuggers/total-view.md
+++ b/docs.it4i/software/debuggers/total-view.md
@@ -18,7 +18,7 @@ On the cluster, users can debug OpenMP or MPI code that runs up to 64 parallel p
Debugging of GPU accelerated codes is also supported.
-You can check the status of the licenses for [Salomon][a] or [Barbora][b]:
+You can check the status of the licenses for [Karolina][a] or [Barbora][b]:
```console
$ cat /apps/user/licenses/totalview_features_state.txt
@@ -39,20 +39,20 @@ $ cat /apps/user/licenses/totalview_features_state.txt
Load all necessary modules to compile the code. For example:
```console
- ml intel
+$ ml intel/2020b
```
Load the TotalView module:
```console
- ml TotalView
+$ ml TotalView/2021.2.14
```
Compile the code:
```console
- mpicc -g -O0 -o test_debug test.c
- mpif90 -g -O0 -o test_debug test.f
+$ mpicc -g -O0 -o test_debug test.c
+$ mpif90 -g -O0 -o test_debug test.f
```
### Compiler Flags
@@ -69,7 +69,7 @@ Before debugging, you need to compile your code with theses flags:
Be sure to log in with an X window forwarding enabled. This could mean using the `-X` in the `ssh`:
```console
-ssh -X username@salomon.it4i.cz
+local $ssh -X username@login1.karolina.it4i.cz
```
Another option is to access the login node using VNC.
@@ -77,7 +77,7 @@ Another option is to access the login node using VNC.
From the login node an interactive session with X windows forwarding (`-X` option) can be started by the following command:
```console
-$ qsub -I -X -A NONE-0-0 -q qexp -lselect=1:ncpus=24:mpiprocs=24,walltime=01:00:00
+$ qsub -I -X -A NONE-0-0 -q qexp -l select=1:ncpus=128:mpiprocs=128,walltime=01:00:00
```
Then launch the debugger with the totalview command followed by the name of the executable to debug.
@@ -87,7 +87,7 @@ Then launch the debugger with the totalview command followed by the name of the
To debug a serial code, use:
```console
-totalview test_debug
+$ totalview test_debug
```
### Debugging a Parallel Code - Option 1
@@ -120,14 +120,14 @@ dlappend TV::image_load_callbacks mpi_auto_run_starter
The source code of this function can be also found in
```console
-$ /apps/all/OpenMPI/1.10.1-GNU-4.9.3-2.25/etc/openmpi-totalview.tcl #Salomon
+$ $EBROOTOPENMPI/etc/openmpi-totalview.tcl
```
You can also add only following line to your ~/.tvdrc file instead of
the entire function:
```console
-$ source /apps/all/OpenMPI/1.10.1-GNU-4.9.3-2.25/etc/openmpi-totalview.tcl #Salomon
+$ source $EBROOTOPENMPI/etc/openmpi-totalview.tcl
```
You need to do this step only once. See also [OpenMPI FAQ entry][c].
@@ -153,8 +153,7 @@ Another option to start a new parallel debugging session from a command line is
The following example shows how to start a debugging session with the Intel MPI:
```console
-$ ml intel
-$ ml TotalView/8.15.4-6-linux-x86-64
+$ ml intel/2020b TotalView/2021.2.14
$ totalview -mpi "Intel MPI-Hydra" -np 8 ./hello_debug_impi
```
@@ -166,7 +165,7 @@ More information regarding the command line parameters of the TotalView can be f
[1] The [TotalView documentation][d] web page is a good source for learning more about some of the advanced TotalView features.
-[a]: https://extranet.it4i.cz/rsweb/salomon/license/Totalview
+[a]: https://extranet.it4i.cz/rsweb/karolina/license/Totalview
[b]: https://extranet.it4i.cz/rsweb/barbora/license/Totalview
[c]: https://www.open-mpi.org/faq/?category=running#run-with-tv
[d]: http://www.roguewave.com/support/product-documentation/totalview-family.aspx#totalview
--
GitLab
From a314703e8052aa32be47acca1c3360551175203e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:54:53 +0200
Subject: [PATCH 28/58] Update docs.it4i/software/debuggers/valgrind.md
---
docs.it4i/software/debuggers/valgrind.md | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/docs.it4i/software/debuggers/valgrind.md b/docs.it4i/software/debuggers/valgrind.md
index 32fe64bef..de0c85a57 100644
--- a/docs.it4i/software/debuggers/valgrind.md
+++ b/docs.it4i/software/debuggers/valgrind.md
@@ -48,7 +48,7 @@ int main(void)
Now, compile it with the Intel compiler :
```console
-$ module add intel
+$ ml intel/2020b
$ icc -g valgrind-example.c -o valgrind-example
```
@@ -56,6 +56,10 @@ Now, let us run it with Valgrind:
`valgrind [valgrind options] <your program binary> [your program options]`
+```console
+$ ml Valgrind/3.16.1-intel-2020b
+```
+
If no Valgrind options are specified, Valgrind defaults to running the Memcheck tool. For the full description of command line options, refer to the Valgrind documentation.
```console
@@ -155,10 +159,7 @@ The default version without MPI support will however report a large number of fa
==30166== by 0x4008BD: main (valgrind-example-mpi.c:18)
```
-So it is better to use the MPI-enabled Valgrind from the module. The MPI version requires the library:
-
-* Salomon: $EBROOTVALGRIND/lib/valgrind/libmpiwrap-amd64-linux.so
-
+So it is better to use the MPI-enabled Valgrind from the module. The MPI version requires the library `$EBROOTVALGRIND/lib/valgrind/libmpiwrap-amd64-linux.so`
which must be included in the `LD_PRELOAD` environment variable.
Let us look at this MPI example:
@@ -182,10 +183,9 @@ int main(int argc, char *argv[])
There are two errors - use of uninitialized memory and invalid length of the buffer. Let us debug it with Valgrind:
```console
-$ module add intel impi
+$ ml intel/2020b Valgrind/3.16.1-intel-2020b
$ mpicc -g valgrind-example-mpi.c -o valgrind-example-mpi
-$ module add valgrind/3.9.0-impi
-$ mpirun -np 2 -env LD_PRELOAD /apps/tools/valgrind/3.9.0/impi/lib/valgrind/libmpiwrap-amd64-linux.so valgrind ./valgrind-example-mpi
+$ mpirun -np 2 -env LD_PRELOAD $EBROOTVALGRIND/lib/valgrind/libmpiwrap-amd64-linux.so valgrind ./valgrind-example-mpi
```
Prints this output (note that there is an output printed for every launched MPI process):
--
GitLab
From 9b15548c863dd1c118070015238794ea0aff1e84 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 12:56:22 +0200
Subject: [PATCH 29/58] Update mkdocs.yml
---
mkdocs.yml | 2 --
1 file changed, 2 deletions(-)
diff --git a/mkdocs.yml b/mkdocs.yml
index 0a6aed012..fbc4a8360 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -170,8 +170,6 @@ nav:
- Intel MKL: software/intel/intel-suite/intel-mkl.md
- Intel TBB: software/intel/intel-suite/intel-tbb.md
- Intel Trace Analyzer and Collector: software/intel/intel-suite/intel-trace-analyzer-and-collector.md
- - Intel Xeon Phi:
- - Intel Xeon Phi Salomon: software/intel/intel-xeon-phi-salomon.md
- Machine Learning:
- Introduction: software/machine-learning/introduction.md
- TensorFlow: software/machine-learning/tensorflow.md
--
GitLab
From cc2fc1dc76375d2016134422890d392ef806a7b2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 13:05:39 +0200
Subject: [PATCH 30/58] Update
docs.it4i/software/cae/comsol/comsol-multiphysics.md
---
docs.it4i/software/cae/comsol/comsol-multiphysics.md | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/docs.it4i/software/cae/comsol/comsol-multiphysics.md b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
index eacd2054b..9dd9012e4 100644
--- a/docs.it4i/software/cae/comsol/comsol-multiphysics.md
+++ b/docs.it4i/software/cae/comsol/comsol-multiphysics.md
@@ -37,10 +37,10 @@ To prepare COMSOL jobs in the interactive mode, we recommend using COMSOL on the
!!! Note
We recommend using the [Virtual Network Computing (VNC)][2].
-Example for Barbora:
+Example for Karolina:
```console
-$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=36:mpiprocs=36
+$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=128:mpiprocs=128
$ ml av COMSOL
------------------------------ /apps/modules/phys -----------------------------
@@ -69,7 +69,7 @@ To run COMSOL in batch mode without the COMSOL Desktop GUI environment, utilize
```bash
#!/bin/bash
-#PBS -l select=3:ncpus=36:mpiprocs=36
+#PBS -l select=3:ncpus=128:mpiprocs=128
#PBS -q qprod
#PBS -N JOB_NAME
#PBS -A PROJECT_ID
@@ -100,7 +100,7 @@ COMSOL is a software package for the numerical solution of partial differential
LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On the clusters there is 1 commercial (**COM**) and 5 educational (**EDU**) licenses of LiveLink for MATLAB (see the [ISV Licenses][3]). The following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode.
```console
-$ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ncpus=36:mpiprocs=36
+$ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ncpus=128:mpiprocs=128
$ ml MATLAB/R2015b COMSOL/5.2.0-EDU
$ comsol -3drend sw server MATLAB
```
@@ -111,7 +111,7 @@ To run LiveLink for MATLAB in batch mode with (comsol_matlab.pbs) job script, yo
```bash
#!/bin/bash
-#PBS -l select=3:ncpus=36:mpiprocs=36
+#PBS -l select=3:ncpus=128:mpiprocs=128
#PBS -q qprod
#PBS -N JOB_NAME
#PBS -A PROJECT_ID
@@ -135,7 +135,7 @@ cd $EBROOTCOMSOL/mli
matlab -nodesktop -nosplash -r "mphstart; addpath /scratch/project/PROJECT_ID; test_job"
```
-This example shows how to run LiveLink for MATLAB with the following configuration: 3 nodes and 36 cores per node. A working directory has to be created before submitting (comsol_matlab.pbs) job script into the queue. The input file (test_job.m) has to be in the working directory or a full path to the input file has to be specified. The Matlab command option (`-r ”mphstart”`) created a connection with a COMSOL server using the default port number.
+This example shows how to run LiveLink for MATLAB with the following configuration: 3 nodes and 128 cores per node. A working directory has to be created before submitting (comsol_matlab.pbs) job script into the queue. The input file (test_job.m) has to be in the working directory or a full path to the input file has to be specified. The Matlab command option (`-r ”mphstart”`) created a connection with a COMSOL server using the default port number.
[1]: licensing-and-available-versions.md
[2]: ../../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
--
GitLab
From 16abbbaabe07d4748cc749905f50f3ddc8d82e5a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 13:32:23 +0200
Subject: [PATCH 31/58] Update
docs.it4i/software/intel/intel-suite/intel-compilers.md
---
.../intel/intel-suite/intel-compilers.md | 43 ++++++++++++-------
1 file changed, 28 insertions(+), 15 deletions(-)
diff --git a/docs.it4i/software/intel/intel-suite/intel-compilers.md b/docs.it4i/software/intel/intel-suite/intel-compilers.md
index 1d2a64e63..47566dc41 100644
--- a/docs.it4i/software/intel/intel-suite/intel-compilers.md
+++ b/docs.it4i/software/intel/intel-suite/intel-compilers.md
@@ -8,39 +8,52 @@
Intel compilers are available in multiple versions via the `intel` module. The compilers include the icc C and C++ compiler and the ifort Fortran 77/90/95 compiler.
+For the current list of installed versions, use:
+
+```console
+$ ml av intel/
+```
+
```console
-$ ml intel
+$ ml intel/2020b
$ icc -v
+icc version 19.1.3.304 (gcc version 10.2.0 compatibility)
$ ifort -v
+ifort version 19.1.3.304
```
-## AVX2 Vectorization
+## Instructions Vectorization
+
+Intel compilers provide vectorization of the code via the AVX-2/AVX-512 instructions and support threading parallelization via OpenMP.
+
+For maximum performance on the Barbora cluster compute nodes, compile your programs using the AVX-512 instructions, with reporting where the vectorization was used. We recommend the following compilation options for high performance.
+
+``` info
+ Barbora non-accelerated nodes support AVX-512 instructions (cn1-cn192).
-Intel compilers provide vectorization of the code via the AVX2 instructions and support threading parallelization via OpenMP.
+```console
+$ icc -ipo -O3 -xCORE-AVX512 -qopt-report1 -qopt-report-phase=vec myprog.c mysubroutines.c -o myprog.x
+```
-For maximum performance on the Salomon cluster compute nodes, compile your programs using the AVX2 instructions, with reporting where the vectorization was used. We recommend the following compilation options for high performance:
+In this example, we compile the program enabling interprocedural optimizations between source files (`-ipo`), aggressive loop optimizations (`-O3`), and vectorization (`-xCORE-AVX512`).
+
+For maximum performance on the Barbora GPU nodes or Karolina cluster compute nodes, compile your programs using the AVX-2 instructions, with reporting where the vectorization was used. We recommend the following compilation options for high performance.
```console
-$ icc -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec myprog.c mysubroutines.c -o myprog.x
-$ ifort -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec myprog.f mysubroutines.f -o myprog.x
+$ icc -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec myprog.c mysubroutines.c -o myprog.x
```
+``` warn
+ Karolina cluster has AMD cpu, use compiler options `-march=core-avx2`.
+
In this example, we compile the program enabling interprocedural optimizations between source files (`-ipo`), aggressive loop optimizations (`-O3`), and vectorization (`-xCORE-AVX2`).
The compiler recognizes the omp, simd, vector, and ivdep pragmas for OpenMP parallelization and AVX2 vectorization. Enable the OpenMP parallelization by the `-openmp` compiler switch.
```console
$ icc -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec -openmp myprog.c mysubroutines.c -o myprog.x
-$ ifort -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec -openmp myprog.f mysubroutines.f -o myprog.x
```
Read more [here][a].
-## Sandy Bridge/Ivy Bridge/Haswell Binary Compatibility
-
-Salomon compute nodes are equipped with the Haswell-based architecture while the UV1 SMP compute server has Ivy Bridge CPUs, which are equivalent to Sandy Bridge (only smaller manufacturing technology). The new processors are backward compatible with the Sandy Bridge nodes, so all programs that run on the Sandy Bridge processors should also run on the new Haswell nodes. To get the optimal performance out of the Haswell processors, a program should make use of the special AVX2 instructions for this processor. This can be done by recompiling codes with the compiler flags designated to invoke these instructions. For the Intel compiler suite, there are two options:
-
-* Using compiler flag (both for Fortran and C): `-xCORE-AVX2`. This will create a binary with AVX2 instructions, specifically for the Haswell processors. Note that the executable will not run on Sandy Bridge/Ivy Bridge nodes.
-* Using compiler flags (both for Fortran and C): `-xAVX -axCORE-AVX2`. This will generate multiple, feature specific auto-dispatch code paths for Intel® processors, if there is a performance benefit. Therefore, this binary will run both on Sandy Bridge/Ivy Bridge and Haswell processors. During runtime, it will be decided which path to follow, dependent on which processor you are running on. In general, this will result in larger binaries.
-
-[a]: https://software.intel.com/en-us/intel-cplusplus-compiler-16.0-user-and-reference-guide
+[a]: https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top.html
--
GitLab
From 9b64d7684d32ed2bdc4e63eb52491a2a80df6470 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 13:50:47 +0200
Subject: [PATCH 32/58] Update
docs.it4i/software/intel/intel-suite/intel-integrated-performance-primitives.md
Deleted docs.it4i/software/intel/intel-suite/intel-debugger.md
---
.../intel/intel-suite/intel-debugger.md | 80 -------------------
...intel-integrated-performance-primitives.md | 11 +--
2 files changed, 2 insertions(+), 89 deletions(-)
delete mode 100644 docs.it4i/software/intel/intel-suite/intel-debugger.md
diff --git a/docs.it4i/software/intel/intel-suite/intel-debugger.md b/docs.it4i/software/intel/intel-suite/intel-debugger.md
deleted file mode 100644
index 310b118be..000000000
--- a/docs.it4i/software/intel/intel-suite/intel-debugger.md
+++ /dev/null
@@ -1,80 +0,0 @@
-# Intel Debugger
-
-!!! note
- IDB is no longer available since Intel Parallel Studio 2015.
-
-## Introduction
-
-The Intel debugger version is available via the `intel/13.5.192` module. The debugger works for serial and parallel applications compiled with the C and C++ compiler and the ifort Fortran 77/90/95 compiler. The debugger provides a Java GUI environment. Use [X display][1] to run the GUI.
-
-```console
-$ ml intel/13.5.192
-$ ml Java
-$ idb
-```
-
-The debugger may run in text mode. To debug in the text mode, use:
-
-```console
-$ idbc
-```
-
-To debug on compute nodes, the `intel` module must be loaded. The GUI on compute nodes may be accessed using the same way as in the [GUI section][1].
-
-## Debugging Serial Applications
-
-Example:
-
-```console
-$ qsub -q qexp -l select=1:ncpus=24 -X -I
- qsub: waiting for job 19654.srv11 to start
- qsub: job 19654.srv11 ready
-$ ml intel
-$ ml Java
-$ icc -O0 -g myprog.c -o myprog.x
-$ idb ./myprog.x
-```
-
-In this example, we allocate 1 full compute node, compile the myprog.c program with the `-O0 -g` debugging options and run the IDB debugger interactively on the myprog.x executable. The GUI access is via the X11 port forwarding provided by the PBS workload manager.
-
-## Debugging Parallel Applications
-
- The Intel debugger is capable of debugging multithreaded and MPI parallel programs as well.
-
-### Small Number of MPI Ranks
-
-For debugging small number of MPI ranks, you may execute and debug each rank in a separate xterm terminal (do not forget the [X display][1]. Using Intel MPI, this may be done in the following way:
-
-```console
-$ qsub -q qexp -l select=2:ncpus=24 -X -I
- qsub: waiting for job 19654.srv11 to start
- qsub: job 19655.srv11 ready
-$ ml intel
-$ mpirun -ppn 1 -hostfile $PBS_NODEFILE --enable-x xterm -e idbc ./mympiprog.x
-```
-
-In this example, we allocate 2 full compute nodes, run xterm on each node, and start the IDB debugger in the command line mode, debugging two ranks of the mympiprog.x application. The xterm will pop up for each rank with IDB prompt ready. The example is not limited to use of Intel MPI.
-
-### Large Number of MPI Ranks
-
-Run the IDB debugger from within the MPI debug option. This will cause the debugger to bind to all ranks and provide aggregated outputs across the ranks, pausing execution automatically just after startup. You may then set break points and step the execution manually. Using Intel MPI:
-
-```console
-$ qsub -q qexp -l select=2:ncpus=24 -X -I
- qsub: waiting for job 19654.srv11 to start
- qsub: job 19655.srv11 ready
-$ ml intel
-$ mpirun -n 48 -idb ./mympiprog.x
-```
-
-### Debugging Multithreaded Application
-
-Run the IDB debugger in GUI mode. The Parallel menu contains a number of tools for debugging multiple threads. One of the most useful tools is the **Serialize Execution** tool, which serializes execution of concurrent threads for easy orientation and identification of concurrency related bugs.
-
-## Further Information
-
-Exhaustive manual on IDB features and usage is published at the [Intel website][a].
-
-[1]: ../../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
-
-[a]: https://software.intel.com/sites/products/documentation/doclib/iss/2013/compiler/cpp-lin/
diff --git a/docs.it4i/software/intel/intel-suite/intel-integrated-performance-primitives.md b/docs.it4i/software/intel/intel-suite/intel-integrated-performance-primitives.md
index 572d93360..c79bff525 100644
--- a/docs.it4i/software/intel/intel-suite/intel-integrated-performance-primitives.md
+++ b/docs.it4i/software/intel/intel-suite/intel-integrated-performance-primitives.md
@@ -12,11 +12,6 @@ For the current list of installed versions, use:
$ ml av ipp
```
-!!! Important
- Check out IPP before implementing own math functions for data processing, as it is likely already there.
- IPP version 9.0.1 compiled for AVX2 vector instructions is available via the `ipp` module:
- The module sets up environment variables, required for linking and running IPP-enabled applications.
-
## IPP Example
```cpp
@@ -63,16 +58,14 @@ int main(int argc, char* argv[])
Compile the example above, using any compiler and the `ipp` module:
```console
-$ ml intel
-$ ml ipp
+$ ml intel/2020b ipp/2020.3.304
$ icc testipp.c -o testipp.x -lippi -lipps -lippcore
```
You will need the `ipp` module loaded to run an IPP-enabled executable. This may be avoided, by compiling library search paths into the executable:
```console
-$ ml intel
-$ ml ipp
+$ ml intel/2020b ipp/2020.3.304
$ icc testipp.c -o testipp.x -Wl,-rpath=$LIBRARY_PATH -lippi -lipps -lippcore
```
--
GitLab
From 4e3ee322f2b2ab56470e2b25cee5f343badbe55b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 13:56:36 +0200
Subject: [PATCH 33/58] Update
docs.it4i/software/intel/intel-suite/intel-mkl.md
---
.../software/intel/intel-suite/intel-mkl.md | 25 ++-----------------
1 file changed, 2 insertions(+), 23 deletions(-)
diff --git a/docs.it4i/software/intel/intel-suite/intel-mkl.md b/docs.it4i/software/intel/intel-suite/intel-mkl.md
index 997017804..66dce49d5 100644
--- a/docs.it4i/software/intel/intel-suite/intel-mkl.md
+++ b/docs.it4i/software/intel/intel-suite/intel-mkl.md
@@ -66,8 +66,7 @@ A number of examples demonstrating use of the Intel MKL library and its linking
### Working With Examples
```console
-$ ml intel
-$ ml imkl
+$ ml intel/2020b
$ cp -a $MKL_EXAMPLES/cblas /tmp/
$ cd /tmp/cblas
$ make sointel64 function=cblas_dgemm
@@ -78,11 +77,9 @@ In this example, we compile, link, and run the cblas_dgemm example, demonstratin
### Example: MKL and Intel Compiler
```console
-$ ml intel
-$ ml imkl
+$ ml intel/2020b
$ cp -a $MKL_EXAMPLES/cblas /tmp/
$ cd /tmp/cblas
-$
$ icc -w source/cblas_dgemmx.c source/common_func.c -mkl -o cblas_dgemmx.x
$ ./cblas_dgemmx.x data/cblas_dgemmx.d
```
@@ -95,23 +92,6 @@ $ icc -w source/cblas_dgemmx.c source/common_func.c -o cblas_dgemmx.x -I$MKL_INC
In this example, we compile and link the cblas_dgemm example, using LP64 interface to threaded MKL and Intel OMP threads implementation.
-### Example: Intel MKL and GNU Compiler
-
-```console
-$ ml GCC
-$ ml imkl
-$ cp -a $MKL_EXAMPLES/cblas /tmp/
-$ cd /tmp/cblas
-$ gcc -w source/cblas_dgemmx.c source/common_func.c -o cblas_dgemmx.x -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lm
-$ ./cblas_dgemmx.x data/cblas_dgemmx.d
-```
-
-In this example, we compile, link, and run the cblas_dgemm example, using the LP64 interface to threaded MKL and GNU OMP threads implementation.
-
-## MKL and MIC Accelerators
-
-The Intel MKL can automatically offload the computations to the MIC accelerator. For details, see the [Intel Xeon Phi][2] section.
-
## LAPACKE C Interface
MKL includes LAPACKE C Interface to LAPACK. However, note that although Intel is the author of LAPACKE, the LAPACKE header files are not present in MKL. For this reason, we have prepared the LAPACKE module, which includes Intel's LAPACKE headers from official LAPACK, which you can use to compile code using the LAPACKE interface against MKL.
@@ -121,7 +101,6 @@ MKL includes LAPACKE C Interface to LAPACK. However, note that although Intel is
Read more on [Intel website][c], in particular the [MKL user guide][d].
[1]: #examples
-[2]: ../intel-xeon-phi-salomon.md
[a]: http://software.intel.com/sites/products/documentation/doclib/mkl_sa/11/mklman/index.htm
[b]: http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor
--
GitLab
From 46bd3bc0f2e5793076753eae12d7104d6e8b1106 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 13:59:21 +0200
Subject: [PATCH 34/58] Update
docs.it4i/software/intel/intel-suite/intel-parallel-studio-introduction.md
---
.../intel-parallel-studio-introduction.md | 23 +++++--------------
1 file changed, 6 insertions(+), 17 deletions(-)
diff --git a/docs.it4i/software/intel/intel-suite/intel-parallel-studio-introduction.md b/docs.it4i/software/intel/intel-suite/intel-parallel-studio-introduction.md
index f3772f6f9..8b38a9145 100644
--- a/docs.it4i/software/intel/intel-suite/intel-parallel-studio-introduction.md
+++ b/docs.it4i/software/intel/intel-suite/intel-parallel-studio-introduction.md
@@ -18,32 +18,21 @@ Intel Parallel Studio XE
The Intel compilers are available via the intel module. The compilers include the icc C and C++ compiler and the ifort Fortran 77/90/95 compiler.
```console
-$ ml intel
+$ ml intel/2020b
$ icc -v
+icc version 19.1.3.304 (gcc version 10.2.0 compatibility)
$ ifort -v
+ifort version 19.1.3.304
```
Read more at the [Intel Compilers][1] page.
-## Intel Debugger
-
-IDB is no longer available since Parallel Studio 2015.
-
-The Intel debugger version 13.0 is available via the `intel` module. The debugger works for applications compiled with the C and C++ compiler and the ifort Fortran 77/90/95 compiler. The debugger provides a Java GUI environment.
-
-```console
-$ ml intel
-$ idb
-```
-
-Read more at the [Intel Debugger][2] page.
-
## Intel Math Kernel Library
Intel Math Kernel Library (Intel MKL) is a library of math kernel subroutines, extensively threaded and optimized for maximum performance. Intel MKL unites and provides these basic components: BLAS, LAPACK, ScaLapack, PARDISO, FFT, VML, VSL, Data fitting, Feast Eigensolver, and many more.
```console
-$ ml imkl
+$ ml imkl/2020.4.304-iimpi-2020b
```
Read more at the [Intel MKL][3] page.
@@ -53,7 +42,7 @@ Read more at the [Intel MKL][3] page.
Intel Integrated Performance Primitives, version 7.1.1, compiled for AVX is available via the `ipp` module. IPP is a library of highly optimized algorithmic building blocks for media and data applications. This includes signal, image, and frame processing algorithms, such as FFT, FIR, Convolution, Optical Flow, Hough transform, Sum, MinMax, and many more.
```console
-$ ml ipp
+$ ml ipp/2020.3.304
```
Read more at the [Intel IPP][4] page.
@@ -63,7 +52,7 @@ Read more at the [Intel IPP][4] page.
Intel Threading Building Blocks (Intel TBB) is a library that supports scalable parallel programming using standard ISO C++ code. It does not require special languages or compilers. It is designed to promote scalable data parallel programming. Additionally, it fully supports nested parallelism, so you can build larger parallel components from smaller parallel components. To use the library, you specify tasks, not threads, and let the library map tasks onto threads in an efficient manner.
```console
-$ ml tbb
+$ ml tbb/2020.3-GCCcore-10.2.0
```
Read more at the [Intel TBB][5] page.
--
GitLab
From efcc1729d203452108599b73c5a9029492b4710e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 13:59:48 +0200
Subject: [PATCH 35/58] Update
docs.it4i/software/intel/intel-suite/intel-tbb.md
---
docs.it4i/software/intel/intel-suite/intel-tbb.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/docs.it4i/software/intel/intel-suite/intel-tbb.md b/docs.it4i/software/intel/intel-suite/intel-tbb.md
index 2b121cde6..8dd76e535 100644
--- a/docs.it4i/software/intel/intel-suite/intel-tbb.md
+++ b/docs.it4i/software/intel/intel-suite/intel-tbb.md
@@ -21,8 +21,7 @@ Link the TBB library using `-ltbb`.
A number of examples demonstrating use of TBB and its built-in scheduler is available in the $TBB_EXAMPLES directory.
```console
-$ ml intel
-$ ml tbb
+$ ml intel/2020b tbb/2020.3-GCCcore-10.2.0
$ cp -a $TBB_EXAMPLES/common $TBB_EXAMPLES/parallel_reduce /tmp/
$ cd /tmp/parallel_reduce/primes
$ icc -O2 -DNDEBUG -o primes.x main.cpp primes.cpp -ltbb
--
GitLab
From bf93978061c4166aad3c696b1aa126167d853b17 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Mon, 20 Sep 2021 14:17:53 +0200
Subject: [PATCH 36/58] Update
docs.it4i/software/intel/intel-suite/intel-trace-analyzer-and-collector.md
---
.../intel/intel-suite/intel-trace-analyzer-and-collector.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs.it4i/software/intel/intel-suite/intel-trace-analyzer-and-collector.md b/docs.it4i/software/intel/intel-suite/intel-trace-analyzer-and-collector.md
index 1c68d0e93..8c9fc794b 100644
--- a/docs.it4i/software/intel/intel-suite/intel-trace-analyzer-and-collector.md
+++ b/docs.it4i/software/intel/intel-suite/intel-trace-analyzer-and-collector.md
@@ -19,7 +19,7 @@ $ ml av itac
ITAC can collect traces from applications that are using Intel MPI. To generate a trace, simply add the `-trace` option to your `mpirun` command:
```console
-$ ml itac/9.1.2.024
+$ ml itac/2020.3.036
$ mpirun -trace myapp
```
@@ -30,7 +30,7 @@ The trace will be saved in the myapp.stf file in the current directory.
To view and analyze the trace, open the ITAC GUI in a [graphical environment][1]:
```console
-$ ml itac/9.1.2.024
+$ ml itac/2020.3.036
$ traceanalyzer
```
--
GitLab
From adc840b7e23dbe97252606ed0cf38ee733540136 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 11:12:09 +0200
Subject: [PATCH 37/58] Update docs.it4i/software/lang/conda.md
---
docs.it4i/software/lang/conda.md | 101 +++++++------------------------
1 file changed, 22 insertions(+), 79 deletions(-)
diff --git a/docs.it4i/software/lang/conda.md b/docs.it4i/software/lang/conda.md
index efd3711ea..c5c5054a5 100644
--- a/docs.it4i/software/lang/conda.md
+++ b/docs.it4i/software/lang/conda.md
@@ -5,14 +5,11 @@ Conda is an open source package management system and environment management sys
Conda as a package manager helps you find and install packages. If you need a package that requires a different version of Python, you do not need to switch to a different environment manager, because Conda is also an environment manager. With just a few commands, you can set up a completely separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment.
Conda treats Python the same as any other package, so it is easy to manage and update multiple installations.
-Anaconda supports Python 2.7, 3.4, 3.5, and 3.6. Default Python is 2.7 or 3.6, depending on which installer you used:
-
-* For the installers “Anaconda” and “Miniconda,” the default is 2.7.
-* For the installers “Anaconda3” or “Miniconda3,” the default is 3.6.
+Anaconda supports Python 3.X. Default Python is 3.8, depending on which installer you used.
## Conda on the IT4Innovations Clusters
-On the clusters, we have the Anaconda2 and Anaconda3 software installed. How to use these modules is shown below.
+On the clusters, we have the Anaconda3 software installed. How to use these modules is shown below.
!!! note
Use the `ml av conda` command to get up-to-date versions of the modules.
@@ -21,106 +18,52 @@ On the clusters, we have the Anaconda2 and Anaconda3 software installed. How to
$ ml av conda
------------- /apps/modules/lang ---------------------------------
-Anaconda2/4.4.0 Anaconda3/4.4.0
-```
-
-## Anaconda2
-
-Default Python is 2.7.
-
-### First Usage Module Anaconda2
-
-```console
-$ ml Anaconda2/4.4.0
-$ python --version
-Python 2.7.13 :: Anaconda 4.4.0 (64-bit)
-$ conda install numpy
-Fetching package metadata .........
-Solving package specifications: .
-
-Package plan for installation in environment /apps/all/Anaconda2/4.4.0:
-
-The following packages will be UPDATED:
-
-anaconda: 4.4.0-np112py27_0 --> custom-py27_0
-...
-...
-...
-CondaIOError: Missing write permissions in: /apps/all/Anaconda2/4.4.0
-#
-# You don't appear to have the necessary permissions to install packages
-# into the install area '/apps/all/Anaconda2/4.4.0'.
-# However you can clone this environment into your home directory and
-# then make changes to it.
-# This may be done using the command:
-#
-# $ conda create -n my_root --clone="/apps/all/Anaconda2/4.4.0"
-$
-$ conda create -n anaconda2 --clone="/apps/all/Anaconda2/4.4.0"
-Source: /apps/all/Anaconda2/4.4.0
-Destination: /home/svi47/.conda/envs/anaconda2
-The following packages cannot be cloned out of the root environment:
-- conda-4.3.21-py27_0
-- conda-env-2.6.0-0
-Packages: 213
-...
-...
-...
-#
-# To activate this environment, use:
-# > source activate anaconda2
-#
-# To deactivate this en
-```
-
-### Usage Module Anaconda2
-
-```console
-$ ml Anaconda2/4.4.0
-$ source activate anaconda2
-(anaconda2) ~]$
+Anaconda3/2021.05
```
## Anaconda3
-Default Python is 3.6.
+Default Python is 3.8.8.
### First Usage Module Anaconda3
```console
-$ ml Anaconda3/4.4.0
+$ ml Anaconda3/2021.05
$ python --version
-Python 3.6.1 :: Anaconda 4.4.0 (64-bit)
+Python 3.8.8
$ conda install numpy
Fetching package metadata .........
Solving package specifications: .
-Package plan for installation in environment /apps/all/Anaconda3/4.4.0:
+Package plan for installation in environment /apps/all/Anaconda3/2021.05:
The following packages will be UPDATED:
-anaconda: 4.4.0-np112py36_0 --> custom-py36_0
+conda 4.10.1-py38h06a4308_1 --> 4.10.3-py38h06a4308_0
...
...
...
-CondaIOError: Missing write permissions in: /apps/all/Anaconda3/4.4.0
+CondaIOError: Missing write permissions in: /apps/all/Anaconda3/2021.05
#
# You don't appear to have the necessary permissions to install packages
-# into the install area '/apps/all/Anaconda3/4.4.0'.
+# into the install area '/apps/all/Anaconda3/2021.05'.
# However you can clone this environment into your home directory and
# then make changes to it.
# This may be done using the command:
#
-# $ conda create -n my_root --clone="/apps/all/Anaconda3/4.4.0"
+# $ conda create -n my_root --clone="/apps/all/Anaconda3/2021.05"
$
-$ conda create -n anaconda3 --clone="/apps/all/Anaconda3/4.4.0"
-Source: /apps/all/Anaconda3/4.4.0
-Destination: /home/svi47/.conda/envs/anaconda3
+$ conda create -n anaconda3 --clone="/apps/all/Anaconda3/2021.05"
+Source: /apps/all/Anaconda3/2021.05
+Destination: /home/user/.conda/envs/anaconda3
The following packages cannot be cloned out of the root environment:
-- conda-4.3.21-py36_0
-- conda-env-2.6.0-0
-Packages: 200
-Files: 6
+ - defaults/linux-64::conda-env-2.6.0-1
+ - defaults/linux-64::conda-4.10.3-py38h06a4308_0
+ - defaults/noarch::conda-token-0.3.0-pyhd3eb1b0_0
+ - defaults/linux-64::anaconda-navigator-2.0.3-py38_0
+ - defaults/linux-64::conda-build-3.21.4-py38h06a4308_0
+Packages: 339
+Files: 50986
...
...
...
@@ -138,7 +81,7 @@ $ source activate anaconda3
### Usage Module Anaconda3
```console
-$ ml Anaconda3/4.4.0
+$ ml Anaconda3/2021.05
$ source activate anaconda3
(anaconda3) ~]$
```
--
GitLab
From cd3e9c63a6acbe58a78c5489f2d233c051887d44 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 11:22:09 +0200
Subject: [PATCH 38/58] Update docs.it4i/software/lang/csc.md
---
docs.it4i/software/lang/csc.md | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs.it4i/software/lang/csc.md b/docs.it4i/software/lang/csc.md
index 85fb078c1..29cd08a63 100644
--- a/docs.it4i/software/lang/csc.md
+++ b/docs.it4i/software/lang/csc.md
@@ -6,7 +6,7 @@ C# is available on the cluster.
$ ml av mono
-------------------- /apps/modules/lang ---------------
- Mono/5.0.0.100
+ Mono/6.12.0.122
```
!!! note
@@ -15,7 +15,7 @@ $ ml av mono
Activate C# by loading the Mono module:
```console
-$ ml Mono
+$ ml Mono/6.12.0.122
```
## Examples
@@ -106,9 +106,9 @@ class MPIHello
Compile and run the program:
```console
-$ qsub -I -A DD-13-5 -q qexp -l select=2:ncpus=24,walltime=00:30:00
+$ qsub -I -A PROJECT_ID -q qexp -l select=2:ncpus=128,walltime=00:30:00
-$ ml mpi.net
+$ ml n.net
$ mcs -out:csc.exe -reference:/apps/tools/mpi.net/1.0.0-mono-3.12.1/lib/MPI.dll csc.cs
--
GitLab
From d96f4de019a54623186d2b344a0765a92cec3cd4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 11:26:01 +0200
Subject: [PATCH 39/58] Update docs.it4i/software/lang/java.md
---
docs.it4i/software/lang/java.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/docs.it4i/software/lang/java.md b/docs.it4i/software/lang/java.md
index f2be76b9b..3c7a4b85a 100644
--- a/docs.it4i/software/lang/java.md
+++ b/docs.it4i/software/lang/java.md
@@ -51,8 +51,7 @@ class Hello {
```
```console
-$ ml Java/1.8.0_144
-$ ml OpenMPI/1.8.0_144
+$ ml Java/1.8.0_221 OpenMPI/4.1.1-GCC-10.2.0-Java-1.8.0_221
$ mpijavac Hello.java
$ mpirun java Hello
Hello world from rank 23 of 28
--
GitLab
From d3bc3af3fc7409b62358f1ef6014edfdad0be981 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 11:30:29 +0200
Subject: [PATCH 40/58] Update docs.it4i/software/lang/julialang.md,
docs.it4i/software/lang/java.md files
---
docs.it4i/software/lang/java.md | 2 +-
docs.it4i/software/lang/julialang.md | 13 +++++++++++++
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/docs.it4i/software/lang/java.md b/docs.it4i/software/lang/java.md
index 3c7a4b85a..69d7aff93 100644
--- a/docs.it4i/software/lang/java.md
+++ b/docs.it4i/software/lang/java.md
@@ -3,7 +3,7 @@
Java is available on the cluster. Activate Java by loading the Java module:
```console
-$ ml Java
+$ ml Java/1.8.0_221
```
Note that the Java module must be loaded on the compute nodes as well, in order to run Java on compute nodes.
diff --git a/docs.it4i/software/lang/julialang.md b/docs.it4i/software/lang/julialang.md
index 35b3cb4e4..313cb37d9 100644
--- a/docs.it4i/software/lang/julialang.md
+++ b/docs.it4i/software/lang/julialang.md
@@ -2,6 +2,19 @@
A set of unofficial examples of Julia the high-level, high-performance dynamic programming language for technical computing.
+Julia is available on the clusters. Activate Julia by loading the Julia module:
+
+```console
+$ ml Julia/1.5.3-linux-x86_64
+```
+
+Check for Java version and path:
+
+```console
+$ julia -v
+julia version 1.5.3
+```
+
Below are examples of common operations in Julia. They assume you already have Julia installed and working
## Hello World
--
GitLab
From 2a58d314141c56b9fa86871d713a9f861faa9ff9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 11:40:11 +0200
Subject: [PATCH 41/58] Update docs.it4i/software/lang/python.md
---
docs.it4i/software/lang/python.md | 167 ++++++++----------------------
1 file changed, 42 insertions(+), 125 deletions(-)
diff --git a/docs.it4i/software/lang/python.md b/docs.it4i/software/lang/python.md
index 71afced0d..7640de4c2 100644
--- a/docs.it4i/software/lang/python.md
+++ b/docs.it4i/software/lang/python.md
@@ -5,51 +5,24 @@ Python is a widely used high-level programming language for general-purpose prog
Python features a dynamic type system and automatic memory management and supports multiple programming paradigms, including object-oriented, imperative, functional programming, and procedural styles. It has a large and comprehensive standard library.
* [Documentation for Python 3.X][a]
-* [Documentation for Python 2.X][b]
* [PEP 8 -- Style Guide for Python Code][c]
* [Get into Python -- Tutorial][d]
## Python on the IT4Innovations Clusters
-On the clusters, we have the Python 2.X and Python 3.X software installed. How to use these modules is shown below.
+On the clusters, we have the Python 3.X software installed. How to use these modules is shown below.
!!! note
Use the `ml av python/` command to get up-to-date versions of the modules.
-```console
-$ ml av python/
-
--------------------------- /apps/modules/lang --------------------------
- Python/2.7.8-intel-2015b Python/2.7.9-gompi-2015e Python/2.7.10-GCC-4.9.3-2.25-bare Python/2.7.11-intel-2016a Python/3.4.3-intel-2015b Python/3.5.2-intel-2017.00
- Python/2.7.8-intel-2016.01 Python/2.7.9-ictce-7.3.5 Python/2.7.10-GNU-4.9.3-2.25-bare Python/2.7.11-intel-2017a Python/3.5.1-intel-2016.01 Python/3.5.2
- Python/2.7.9-foss-2015b Python/2.7.9-intel-2015b Python/2.7.11-foss-2016a Python/2.7.11-intel-2017.00 Python/3.5.1-intel-2017.00 Python/3.6.1
- Python/2.7.9-foss-2015g Python/2.7.9-intel-2016.01 Python/2.7.11-GCC-4.9.3-2.25-bare Python/2.7.13-base Python/3.5.1 Python/3.6.2-base (D)
- Python/2.7.9-GNU-5.1.0-2.25 Python/2.7.9 Python/2.7.11-intel-2015b Python/2.7.13 Python/3.5.2-foss-2016a
-
--------------------------- /apps/modules/math ---------------------------
- ScientificPython/2.9.4-intel-2015b-Python-2.7.9 ScientificPython/2.9.4-intel-2015b-Python-2.7.11 ScientificPython/2.9.4-intel-2016.01-Python-2.7.9 (D)
-
- Where:
- D: Default Module
-
- If you need software that is not listed, request it at support@it4i.cz.
-```
-
-## Python 2.X
-
-Python 2.7 is scheduled to be the last major version in the 2.x series before it moves into an extended maintenance period. This release contains many of the features that were first released in Python 3.1.
+!!! warn
+ Python 2.7 is not supported - [EOL][b] January 1st, 2020.
```console
-$ ml av python/2
-
------------------------------------------------------------------------------------------------ /apps/modules/lang ------------------------------------------------------------------------------------------------
- Python/2.7.8-intel-2015b Python/2.7.9-GNU-5.1.0-2.25 Python/2.7.9-intel-2016.01 Python/2.7.11-foss-2016a Python/2.7.11-intel-2017a
- Python/2.7.8-intel-2016.01 Python/2.7.9-gompi-2015e Python/2.7.9 Python/2.7.11-GCC-4.9.3-2.25-bare Python/2.7.11-intel-2017.00
- Python/2.7.9-foss-2015b Python/2.7.9-ictce-7.3.5 Python/2.7.10-GCC-4.9.3-2.25-bare Python/2.7.11-intel-2015b Python/2.7.13-base
- Python/2.7.9-foss-2015g Python/2.7.9-intel-2015b Python/2.7.10-GNU-4.9.3-2.25-bare Python/2.7.11-intel-2016a Python/2.7.13
+$ ml av python/
------------------------------------------------------------------------------------------------ /apps/modules/math ------------------------------------------------------------------------------------------------
- ScientificPython/2.9.4-intel-2015b-Python-2.7.9 ScientificPython/2.9.4-intel-2015b-Python-2.7.11 ScientificPython/2.9.4-intel-2016.01-Python-2.7.9 (D)
+------------------------------------------------------ /apps/modules/lang -------------------------------------------------------
+ Python/3.8.2-GCCcore-9.3.0 Python/3.8.6-GCCcore-10.2.0 Python/3.9.5-GCCcore-10.3.0 (D)
Where:
D: Default Module
@@ -57,67 +30,15 @@ $ ml av python/2
If you need software that is not listed, request it at support@it4i.cz.
```
-### Used Module Python/2.x
-
-```console
-$ python --version
-Python 2.6.6
-$ ml Python/2.7.13
-$ python --version
-Python 2.7.1
-```
-
-### Packages in Python/2.x
-
-```console
-$ pip list
-appdirs (1.4.3)
-asn1crypto (0.22.0)
-backports-abc (0.5)
-backports.shutil-get-terminal-size (1.0.0)
-backports.ssl-match-hostname (3.5.0.1)
-BeautifulSoup (3.2.1)
-beautifulsoup4 (4.5.3)
-...
-```
-
-### How to Install New Package to Python/2.x?
-
-```console
-$ ml Python/2.7.13
-$ python --version
-$ pip install wheel --user
-Collecting wheel
- Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
- 100% |████████████████████████████████| 51kB 835kB/s
-Installing collected packages: wheel
-Successfully installed wheel-0.30.0
-```
-
-### How to Update Package in Python/2.x?
-
-```console
-$ ml Python/2.7.13
-$ python --version
-$ pip install scipy --upgrade --user
-Collecting scipy
- Downloading scipy-0.19.1-cp27-cp27mu-manylinux1_x86_64.whl (45.0MB)
- 100% |████████████████████████████████| 45.0MB 5.8kB/s
-Requirement already up-to-date: numpy>=1.8.2 in /apps/all/Python/2.7.13/lib/python2.7/site-packages (from scipy)
-Installing collected packages: scipy
-Successfully installed scipy-0.19.1
-```
-
## Python 3.X
Python 3.0 (a.k.a. "Python 3000" or "Py3k") is a new version of the language that is incompatible with the 2.x line of releases. The language is mostly the same, but many details, especially how built-in objects like dictionaries and strings work, have changed considerably, and many deprecated features have been removed. In addition, the standard library has been reorganized in a few prominent places.
```console
-$ ml av python/3
+$ ml av python3/
----------------------- /apps/modules/lang ----------------------
- Python/3.4.3-intel-2015b Python/3.5.1-intel-2017.00 Python/3.5.2-foss-2016a Python/3.5.2 Python/3.6.2-base (D)
- Python/3.5.1-intel-2016.01 Python/3.5.1 Python/3.5.2-intel-2017.00 Python/3.6.1
+------------------------------------------------------ /apps/modules/lang -------------------------------------------------------
+ Python/3.8.2-GCCcore-9.3.0 Python/3.8.6-GCCcore-10.2.0 Python/3.9.5-GCCcore-10.3.0 (D)
Where:
D: Default Module
@@ -129,65 +50,61 @@ $ ml av python/3
### Used Module Python/3.x
```console
+$ python --version
+Python 2.7.5
+$ ml Python/3.8.6-GCCcore-10.2.0
$ python --version
-Python 2.6.6
-$ ml Python/3.6.2-base
-$ python --version
-Python 3.6.2
+Python 3.8.6
```
### Packages in Python/3.x
```console
$ pip3 list
-nose (1.3.7)
-pip (8.0.2)
-setuptools (20.1.1)
+Package Version
+----------------------------- ----------
+alabaster 0.7.12
+appdirs 1.4.4
+ase 3.22.0
+asn1crypto 1.4.0
+atomicwrites 1.4.0
+attrs 20.2.0
+Babel 2.8.0
+...
+...
+$ pip3 list | wc -l
+138
```
### How to Install New Package to Python/3.x
```console
-$ ml Python/3.6.2-base
+$ ml ml Python/3.8.6-GCCcore-10.2.0
$ python --version
-Python 3.6.2
+Python 3.8.6
$ pip3 install pandas --user
Collecting pandas
- Downloading pandas-0.20.3.tar.gz (10.4MB)
- 100% |████████████████████████████████| 10.4MB 42kB/s
-Collecting python-dateutil>=2 (from pandas)
- Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
- 100% |████████████████████████████████| 196kB 1.3MB/s
-Collecting pytz>=2011k (from pandas)
- Downloading pytz-2017.2-py2.py3-none-any.whl (484kB)
- 100% |████████████████████████████████| 487kB 757kB/s
-Collecting numpy>=1.7.0 (from pandas)
- Using cached numpy-1.13.1.zip
-Collecting six>=1.5 (from python-dateutil>=2->pandas)
- Downloading six-1.11.0-py2.py3-none-any.whl
-Building wheels for collected packages: pandas, numpy
- Running setup.py bdist_wheel for pandas ... done
- Stored in directory: /home/kru0052/.cache/pip/wheels/dd/17/6c/a1c7e8d855f3a700b21256329fd396d105b533c5ed3e20c5e9
- Running setup.py bdist_wheel for numpy ... done
- Stored in directory: /home/kru0052/.cache/pip/wheels/94/44/90/4ce81547e3e5f4398b1601d0051e828b8160f8d3f3dd5a0c8c
-Successfully built pandas numpy
-Installing collected packages: six, python-dateutil, pytz, numpy, pandas
-Successfully installed numpy-1.13.1 pandas-0.20.3 python-dateutil-2.6.1 pytz-2017.2 six-1.11.0
+ Downloading pandas-1.3.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
+ |████████████████████████████████| 11.5 MB 4.8 MB/s
+...
+Installing collected packages: pandas
+Successfully installed pandas-1.3.3
```
### How to Update Package in Python/3.x?
```console
-$ pip3 install scipy --upgrade --user
-Collecting scipy
- Downloading scipy-0.19.1-cp27-cp27mu-manylinux1_x86_64.whl (45.0MB)
- 100% |████████████████████████████████| 45.0MB 5.8kB/s
-Requirement already up-to-date: numpy>=1.8.2 in /apps/all/Python/3.6.2/lib/python3.6/site-packages (from scipy)
-Installing collected packages: scipy
-Successfully installed scipy-0.19.1
+$ pip3 install setuptools --upgrade --user
+Collecting setuptools
+ Downloading setuptools-58.0.4-py3-none-any.whl (816 kB)
+ |███████████████████████████████▎| 798 kB 5.4 MB/s eta 0:00:01
+ |███████████████████████████████▊| 808 kB 5.4 MB/s eta 0:00:01
+ |████████████████████████████████| 816 kB 5.4 MB/s
+Installing collected packages: setuptools
+Successfully installed setuptools-58.0.4
```
[a]: https://docs.python.org/3/
-[b]: https://docs.python.org/2/
+[b]: https://www.python.org/doc/sunset-python-2/
[c]: https://www.python.org/dev/peps/pep-0008/
[d]: https://jobtensor.com/Tutorial/Python/en/Introduction
--
GitLab
From b8c720d8bc331d1d5bc235eb87a704bcb3f2d37c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 11:52:51 +0200
Subject: [PATCH 42/58] Update docs.it4i/software/lang/csc.md
---
docs.it4i/software/lang/csc.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/docs.it4i/software/lang/csc.md b/docs.it4i/software/lang/csc.md
index 29cd08a63..a77bedc71 100644
--- a/docs.it4i/software/lang/csc.md
+++ b/docs.it4i/software/lang/csc.md
@@ -81,6 +81,7 @@ csharp> from f in Directory.GetFiles ("mydirectory")
{ "mydirectory/mynewfile.cs", "mydirectory/script.sh" }
```
+<!--
## MPI.NET
MPI is available for mono:
@@ -118,6 +119,7 @@ Greetings from node 0 of 4 running on cn204
Greetings from node 3 of 4 running on cn199
Greetings from node 1 of 4 running on cn199
```
+-->
For more information, see the [Mono documentation page][a].
--
GitLab
From f4669cd639768e33767ef00039c1e757679a53c3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 14:22:25 +0200
Subject: [PATCH 43/58] Update
docs.it4i/software/machine-learning/tensorflow.md,
docs.it4i/software/mpi/mpi.md files
---
.../software/machine-learning/tensorflow.md | 2 +
docs.it4i/software/mpi/mpi.md | 108 ++++++++++++------
2 files changed, 72 insertions(+), 38 deletions(-)
diff --git a/docs.it4i/software/machine-learning/tensorflow.md b/docs.it4i/software/machine-learning/tensorflow.md
index bb94f7982..44c76851b 100644
--- a/docs.it4i/software/machine-learning/tensorflow.md
+++ b/docs.it4i/software/machine-learning/tensorflow.md
@@ -7,6 +7,7 @@ For searching available modules type:
$ ml av Tensorflow
```
+<!---
## Salomon Modules
Salomon provides (besides other) these TensorFlow modules:
@@ -22,6 +23,7 @@ Salomon provides (besides other) these TensorFlow modules:
* GCC/7.1.0-2.28
* Python/3.6.1
* protobuf/3.2.0-GCC-7.1.0-2.28-Python-3.6.1
+-->
## TensorFlow Application Example
diff --git a/docs.it4i/software/mpi/mpi.md b/docs.it4i/software/mpi/mpi.md
index 428271cb8..7e0dc5761 100644
--- a/docs.it4i/software/mpi/mpi.md
+++ b/docs.it4i/software/mpi/mpi.md
@@ -2,14 +2,11 @@
## Setting Up MPI Environment
-The Salomon cluster provides several implementations of the MPI library:
+The Karolina cluster provides several implementations of the MPI library:
-| MPI Library | Thread support |
-| ----------------- | ------------------------------------------------------------------------------------------------------ |
-| **Intel MPI 4.1** | Full thread support up to `MPI_THREAD_MULTIPLE` |
-| **Intel MPI 5.0** | Full thread support up to `MPI_THREAD_MULTIPLE` |
-| OpenMPI 1.8.6 | Full thread support up to `MPI_THREAD_MULTIPLE`, MPI-3.0 support |
-| SGI MPT 2.12 | |
+* OpenMPI
+* Intel MPI (impi)
+* MPICH
MPI libraries are activated via the environment modules.
@@ -17,56 +14,94 @@ Look up the modulefiles/mpi section in `ml av`:
```console
$ ml av
- ------------------------------ /apps/modules/mpi -------------------------------
- impi/4.1.1.036-iccifort-2013.5.192
- impi/4.1.1.036-iccifort-2013.5.192-GCC-4.8.3
- impi/5.0.3.048-iccifort-2015.3.187
- impi/5.0.3.048-iccifort-2015.3.187-GNU-5.1.0-2.25
- MPT/2.12
- OpenMPI/1.8.6-GNU-5.1.0-2.25
+------------------------------------------------------- /apps/modules/mpi -------------------------------------------------------
+ OpenMPI/3.1.4-GCC-6.3.0-2.27 OpenMPI/4.1.1-GCC-10.2.0
+ OpenMPI/4.0.3-GCC-9.3.0 OpenMPI/4.1.1-GCC-10.3.0 (D)
+ OpenMPI/4.0.5-GCC-10.2.0 impi/2017.4.239-iccifort-2017.8.262-GCC-6.3.0-2.27
+ OpenMPI/4.0.5-gcccuda-2020b impi/2018.4.274-iccifort-2018.5.274-GCC-8.3.0-2.32
+ OpenMPI/4.0.5-iccifort-2020.4.304 impi/2018.4.274-iccifort-2019.1.144-GCC-8.2.0-2.31.1
+ OpenMPI/4.0.5-NVHPC-21.2-CUDA-11.2.2 impi/2019.9.304-iccifort-2020.1.217
+ OpenMPI/4.0.5-NVHPC-21.2-CUDA-11.3.0 impi/2019.9.304-iccifort-2020.4.304
+ OpenMPI/4.1.1-GCC-10.2.0-Java-1.8.0_221 impi/2021.2.0-intel-compilers-2021.2.0 (D)
+ MPICH/3.3.2-GCC-10.2.0
```
-There are default compilers associated with any particular MPI implementation. The defaults may be changed; the MPI libraries may be used in conjunction with any compiler. The defaults are selected via the modules in the following way:
-
-| Module | MPI | Compiler suite |
-| ---------------------------------------- | ---------- | -------------- |
-| impi-5.0.3.048-iccifort- Intel MPI 5.0.3 | 2015.3.187 | |
-| OpenMP-1.8.6-GNU-5.1.0-2 OpenMPI 1.8.6 | .25 | |
+There are default compilers associated with any particular MPI implementation. The defaults may be changed; the MPI libraries may be used in conjunction with any compiler.
Examples:
```console
-$ ml gompi/2015b
+$ ml gompi/2020b
```
-In this example, we activate the latest OpenMPI with the latest GNU compilers (OpenMPI 1.8.6 and GCC 5.1). For more information about toolchains, see the [Environment and Modules][1] section.
+In this example, we activate the OpenMPI with the GNU compilers (OpenMPI 4.0.5 and GCC 10.2.0). For more information about toolchains, see the [Environment and Modules][1] section.
To use OpenMPI with the Intel compiler suite, use:
```console
-$ ml iompi/2015.03
+$ ml iompi/2020b
```
-In this example, the OpenMPI 1.8.6 using the Intel compilers is activated. It uses the `iompi` toolchain.
+In this example, the OpenMPI 4.0.5 using the Intel compilers 2020.4.304 is activated. It uses the `iompi` toolchain.
## Compiling MPI Programs
After setting up your MPI environment, compile your program using one of the MPI wrappers:
+For module `gompi/2020b`
+
```console
$ mpicc -v
+Using built-in specs.
+COLLECT_GCC=/apps/all/GCCcore/10.2.0/bin/gcc
+COLLECT_LTO_WRAPPER=/apps/all/GCCcore/10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
+OFFLOAD_TARGET_NAMES=nvptx-none
+Target: x86_64-pc-linux-gnu
+Configured with: ../configure --enable-languages=c,c++,fortran --without-cuda-driver --enable-offload-targets=nvptx-none --enable-lto --enable-checking=release --disable-multilib --enable-shared=yes --enable-static=yes --enable-threads=posix --enable-plugins --enable-gold=default --enable-ld --with-plugin-ld=ld.gold --prefix=/apps/all/GCCcore/10.2.0 --with-local-prefix=/apps/all/GCCcore/10.2.0 --enable-bootstrap --with-isl=/dev/shm/easybuild/build/GCCcore/10.2.0/system-system/gcc-10.2.0/stage2_stuff
+Thread model: posix
+Supported LTO compression algorithms: zlib
+gcc version 10.2.0 (GCC)
$ mpif77 -v
-$ mpif90 -v
+Using built-in specs.
+COLLECT_GCC=/apps/all/GCCcore/10.2.0/bin/gfortran
+COLLECT_LTO_WRAPPER=/apps/all/GCCcore/10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
+OFFLOAD_TARGET_NAMES=nvptx-none
+Target: x86_64-pc-linux-gnu
+Configured with: ../configure --enable-languages=c,c++,fortran --without-cuda-driver --enable-offload-targets=nvptx-none --enable-lto --enable-checking=release --disable-multilib --enable-shared=yes --enable-static=yes --enable-threads=posix --enable-plugins --enable-gold=default --enable-ld --with-plugin-ld=ld.gold --prefix=/apps/all/GCCcore/10.2.0 --with-local-prefix=/apps/all/GCCcore/10.2.0 --enable-bootstrap --with-isl=/dev/shm/easybuild/build/GCCcore/10.2.0/system-system/gcc-10.2.0/stage2_stuff
+Thread model: posix
+Supported LTO compression algorithms: zlib
+gcc version 10.2.0 (GCC)
+~$ mpif90 -v
+Using built-in specs.
+COLLECT_GCC=/apps/all/GCCcore/10.2.0/bin/gfortran
+COLLECT_LTO_WRAPPER=/apps/all/GCCcore/10.2.0/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
+OFFLOAD_TARGET_NAMES=nvptx-none
+Target: x86_64-pc-linux-gnu
+Configured with: ../configure --enable-languages=c,c++,fortran --without-cuda-driver --enable-offload-targets=nvptx-none --enable-lto --enable-checking=release --disable-multilib --enable-shared=yes --enable-static=yes --enable-threads=posix --enable-plugins --enable-gold=default --enable-ld --with-plugin-ld=ld.gold --prefix=/apps/all/GCCcore/10.2.0 --with-local-prefix=/apps/all/GCCcore/10.2.0 --enable-bootstrap --with-isl=/dev/shm/easybuild/build/GCCcore/10.2.0/system-system/gcc-10.2.0/stage2_stuff
+Thread model: posix
+Supported LTO compression algorithms: zlib
+gcc version 10.2.0 (GCC)
```
When using Intel MPI, use the following MPI wrappers:
+For module `intel/2020b`
+
+
```console
-$ mpicc
-$ mpiifort
+$ mpiicc -v
+mpiicc for the Intel(R) MPI Library 2019 Update 9 for Linux*
+Copyright 2003-2020, Intel Corporation.
+icc version 19.1.3.304 (gcc version 10.2.0 compatibility)
+ld /lib/../lib64/crt1.o /lib/../lib64/crti.o /apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/crtbegin.o --eh-frame-hdr --build-id -dynamic-linker /lib64/ld-linux-x86-64.so.2 -m elf_x86_64 -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib -o a.out -L/apps/all/imkl/2020.4.304-iimpi-2020b/mkl/lib/intel64 -L/apps/all/imkl/2020.4.304-iimpi-2020b/lib/intel64 -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/libfabric/lib -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib -L/apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/tbb/lib/intel64/gcc4.8 -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib -L/apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/compiler/lib/intel64_lin -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib/../lib64 -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib/../lib64 -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib/../lib64 -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib/../lib64 -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/ -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64 -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64/ -L/lib/../lib64 -L/lib/../lib64/ -L/usr/lib/../lib64 -L/usr/lib/../lib64/ -L/apps/all/imkl/2020.4.304-iimpi-2020b/mkl/lib/intel64/ -L/apps/all/imkl/2020.4.304-iimpi-2020b/lib/intel64/ -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/libfabric/lib/ -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release/ -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/ -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib64 -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib/ -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib64 -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib/ -L/apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/tbb/lib/intel64/gcc4.8/ -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib64 -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib/ -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib64 -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib/ -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../ -L/lib64 -L/lib/ -L/usr/lib64 -L/usr/lib --enable-new-dtags -rpath /apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release -rpath /apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib -lmpifort -lmpi -ldl -lrt -lpthread -Bdynamic -Bstatic -limf -lsvml -lirng -Bdynamic -lm -Bstatic -lipgo -ldecimal --as-needed -Bdynamic -lcilkrts -lstdc++ --no-as-needed -lgcc -lgcc_s -Bstatic -lirc -lsvml -Bdynamic -lc -lgcc -lgcc_s -Bstatic -lirc_s -Bdynamic -ldl -lc /apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/crtend.o /lib/../lib64/crtn.o
+$ mpiifort -v
+mpiifort for the Intel(R) MPI Library 2019 Update 9 for Linux*
+Copyright 2003-2020, Intel Corporation.
+ifort version 19.1.3.304
+ld /lib/../lib64/crt1.o /lib/../lib64/crti.o /apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/crtbegin.o --eh-frame-hdr --build-id -dynamic-linker /lib64/ld-linux-x86-64.so.2 -m elf_x86_64 -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib -o a.out /apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/compiler/lib/intel64_lin/for_main.o -L/apps/all/imkl/2020.4.304-iimpi-2020b/mkl/lib/intel64 -L/apps/all/imkl/2020.4.304-iimpi-2020b/lib/intel64 -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/libfabric/lib -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib -L/apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/tbb/lib/intel64/gcc4.8 -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib -L/apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/compiler/lib/intel64_lin -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib/../lib64 -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib/../lib64 -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib/../lib64 -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib/../lib64 -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib/../lib64/ -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/ -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64 -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64/ -L/lib/../lib64 -L/lib/../lib64/ -L/usr/lib/../lib64 -L/usr/lib/../lib64/ -L/apps/all/imkl/2020.4.304-iimpi-2020b/mkl/lib/intel64/ -L/apps/all/imkl/2020.4.304-iimpi-2020b/lib/intel64/ -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/libfabric/lib/ -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release/ -L/apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/ -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib64 -L/apps/all/UCX/1.9.0-GCCcore-10.2.0/lib/ -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib64 -L/apps/all/numactl/2.0.13-GCCcore-10.2.0/lib/ -L/apps/all/iccifort/2020.4.304/compilers_and_libraries_2020.4.304/linux/tbb/lib/intel64/gcc4.8/ -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib64 -L/apps/all/binutils/2.35-GCCcore-10.2.0/lib/ -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib64 -L/apps/all/zlib/1.2.11-GCCcore-10.2.0/lib/ -L/apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../ -L/lib64 -L/lib/ -L/usr/lib64 -L/usr/lib --enable-new-dtags -rpath /apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib/release -rpath /apps/all/impi/2019.9.304-iccifort-2020.4.304/intel64/lib -lmpifort -lmpi -ldl -lrt -lpthread -Bdynamic -Bstatic -lifport -lifcoremt -limf -lsvml -Bdynamic -lm -Bstatic -lipgo -lirc -Bdynamic -lpthread -Bstatic -lsvml -Bdynamic -lc -lgcc -lgcc_s -Bstatic -lirc_s -Bdynamic -ldl -lc /apps/all/GCCcore/10.2.0/lib/gcc/x86_64-pc-linux-gnu/10.2.0/crtend.o /lib/../lib64/crtn.o
```
-Wrappers `mpif90` and `mpif77` provided by Intel MPI are designed for GCC and GFortran. You might be able to compile MPI code by them even with Intel compilers, but you might run into problems (for example, native MIC compilation with `-mmic` does not work with `mpif90`).
+Wrappers `mpif90` and `mpif77` provided by Intel MPI are designed for GCC and GFortran. You might be able to compile MPI code by them even with Intel compilers, but you might run into problems.
Example program:
@@ -119,29 +154,26 @@ The optimal way to run an MPI program depends on its memory requirements, memory
!!! note
Consider these ways to run an MPI program:
- 1. One MPI process per node, 24 threads per process
- 2. Two MPI processes per node, 12 threads per process
- 3. 24 MPI processes per node, 1 thread per process.
+ 1. One MPI process per node, 128 threads per process
+ 2. Two MPI processes per node, 64 threads per process
+ 3. 128 MPI processes per node, 1 thread per process.
-**One MPI** process per node, using 24 threads, is most useful for memory demanding applications that make good use of processor cache memory and are not memory-bound. This is also a preferred way for communication intensive applications as one process per node enjoys full bandwidth access to the network interface.
+**One MPI** process per node, using 128 threads, is most useful for memory demanding applications that make good use of processor cache memory and are not memory-bound. This is also a preferred way for communication intensive applications as one process per node enjoys full bandwidth access to the network interface.
-**Two MPI** processes per node, using 12 threads each, bound to processor socket is most useful for memory bandwidth-bound applications such as BLAS1 or FFT with scalable memory demand. However, note that the two processes will share access to the network interface. The 12 threads and socket binding should ensure maximum memory access bandwidth and minimize communication, migration, and NUMA effect overheads.
+**Two MPI** processes per node, using 64 threads each, bound to processor socket is most useful for memory bandwidth-bound applications such as BLAS1 or FFT with scalable memory demand. However, note that the two processes will share access to the network interface. The 64 threads and socket binding should ensure maximum memory access bandwidth and minimize communication, migration, and NUMA effect overheads.
!!! note
Important! Bind every OpenMP thread to a core!
In the previous two cases with one or two MPI processes per node, the operating system might still migrate OpenMP threads between cores. You want to avoid this by setting the `KMP_AFFINITY` or `GOMP_CPU_AFFINITY` environment variables.
-**24 MPI** processes per node, using 1 thread each bound to a processor core is most suitable for highly scalable applications with low communication demand.
+**128 MPI** processes per node, using 1 thread each bound to a processor core is most suitable for highly scalable applications with low communication demand.
### Running OpenMPI
-The [OpenMPI 1.8.6][a] is based on OpenMPI. Read more on [how to run OpenMPI][2].
-
-The Intel MPI may run on the [Intel Xeon Ph][3] accelerators as well. Read more on [how to run Intel MPI on accelerators][3].
+The [OpenMPI 4.1.1][a] is based on OpenMPI. Read more on [how to run OpenMPI][2].
[1]: ../../modules-matrix.md
[2]: running_openmpi.md
-[3]: ../intel/intel-xeon-phi-salomon.md
[a]: http://www.open-mpi.org/
--
GitLab
From 585ecec042a7400406aab53ad20acf3e5d947482 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 14:49:55 +0200
Subject: [PATCH 44/58] Update docs.it4i/software/mpi/mpi4py-mpi-for-python.md
---
.../software/mpi/mpi4py-mpi-for-python.md | 247 +++++++++++-------
1 file changed, 147 insertions(+), 100 deletions(-)
diff --git a/docs.it4i/software/mpi/mpi4py-mpi-for-python.md b/docs.it4i/software/mpi/mpi4py-mpi-for-python.md
index e1c97170c..bcb78f0b6 100644
--- a/docs.it4i/software/mpi/mpi4py-mpi-for-python.md
+++ b/docs.it4i/software/mpi/mpi4py-mpi-for-python.md
@@ -10,29 +10,18 @@ MPI4Py is available in standard Python modules on the clusters.
## Modules
-MPI4Py is built for OpenMPI. Before you start with MPI4Py, you need to load the Python and OpenMPI modules.
+MPI4Py is built for OpenMPI or Intel MPI. Before you start with MPI4Py, you need to load the mpi4py module.
```console
-$ ml av Python/
---------------------------------------- /apps/modules/lang -------------------------
- Python/2.7.8-intel-2015b Python/2.7.11-intel-2016a Python/3.5.1-intel-2017.00
- Python/2.7.11-intel-2017a Python/2.7.9-foss-2015b Python/2.7.9-intel-2015b
- Python/2.7.11-foss-2016a Python/3.5.2-foss-2016a Python/3.5.1
- Python/2.7.9-foss-2015g Python/3.4.3-intel-2015b Python/2.7.9
- Python/2.7.11-intel-2015b Python/3.5.2
-
-$ ml av OpenMPI/
---------------------------------------- /apps/modules/mpi --------------------------
-OpenMPI/1.8.6-GCC-4.4.7-system OpenMPI/1.8.8-GNU-4.9.3-2.25 OpenMPI/1.10.1-GCC-4.9.3-2.25
-OpenMPI/1.8.6-GNU-5.1.0-2.25 OpenMPI/1.8.8-GNU-5.1.0-2.25 OpenMPI/1.10.1-GNU-4.9.3-2.25
- OpenMPI/1.8.8-iccifort-2015.3.187-GNU-4.9.3-2.25 OpenMPI/2.0.2-GCC-6.3.0-2.27
+$ ml av mpi4py
+------------------------------------------------------- /apps/modules/mpi -------------------------------------------------------
+ mpi4py/3.1.1-gompi-2020b mpi4py/3.1.1-intel-2020b (D)
```
!!! warning "Flavours"
- * modules Python/x.x.x-intel... - intel MPI
- * modules Python/x.x.x-foss... - OpenMPI
- * modules Python/x.x.x - without MPI
+ * modules mpi4py/x.x.x-intel... - intel MPI
+ * modules mpi4py/x.x.x-gompi... - OpenMPI
## Execution
@@ -45,129 +34,187 @@ from mpi4py import MPI
The MPI4Py-enabled Python programs [execute as any other OpenMPI][1] code. The simpliest way is to run:
```console
-$ mpiexec python <script>.py
+$ mpirun python <script>.py
```
For example:
```console
-$ mpiexec python hello_world.py
+$ mpirun python hello_world.py
```
## Examples
-### Hello World!
-
-```python
-from mpi4py import MPI
-
-comm = MPI.COMM_WORLD
-
-print "Hello! I'm rank %d from %d running in total..." % (comm.rank, comm.size)
+Execute the above code as:
-comm.Barrier() # wait for everybody to synchronize
+```console
+$ qsub -q qprod -l select=4:ncpus=128:mpiprocs=128:ompthreads=1 -I -A PROJECT_ID
+$ ml mpi4py/3.1.1-intel-2020b # or mpi4py/3.1.1-gompi-2020b
```
-### Collective Communication With NumPy Arrays
+### Hello World!
```python
-from mpi4py import MPI
-from __future__ import division
-import numpy as np
-
-comm = MPI.COMM_WORLD
-
-print("-"*78)
-print(" Running on %d cores" % comm.size)
-print("-"*78)
+#!/usr/bin/env python
+"""
+Parallel Hello World
+"""
-comm.Barrier()
-
-# Prepare a vector of N=5 elements to be broadcasted...
-N = 5
-if comm.rank == 0:
- A = np.arange(N, dtype=np.float64) # rank 0 has proper data
-else:
- A = np.empty(N, dtype=np.float64) # all other just an empty array
+from mpi4py import MPI
+import sys
-# Broadcast A from rank 0 to everybody
-comm.Bcast( [A, MPI.DOUBLE] )
+size = MPI.COMM_WORLD.Get_size()
+rank = MPI.COMM_WORLD.Get_rank()
+name = MPI.Get_processor_name()
-# Everybody should now have the same...
-print "[%02d] %s" % (comm.rank, A)
+sys.stdout.write(
+ "Hello, World! I am process %d of %d on %s.\n"
+ % (rank, size, name))
```
-Execute the above code as:
-
```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=16:ompthreads=1 -I # Salomon: ncpus=24:mpiprocs=24
-$ ml Python
-$ ml OpenMPI
-$ mpiexec -bycore -bind-to-core python hello_world.py
+mpirun python ./hello_world.py
+...
+Hello, World! I am process 81 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 91 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 15 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 105 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 112 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 11 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 83 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 58 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 103 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 4 of 512 on cn041.karolina.it4i.cz.
+Hello, World! I am process 28 of 512 on cn041.karolina.it4i.cz.
```
-In this example, we run MPI4Py-enabled code on 4 nodes, 16 cores per node (total of 64 processes), each Python process is bound to a different core. More examples and documentation can be found on [MPI for Python webpage][a].
-
-### Adding Numbers
-
-Task: count sum of numbers from 1 to 1 000 000. (There is an easy formula to count the sum of arithmetic sequence, but we are showing the MPI solution with adding numbers one by one).
+### Mandelbrot
```python
-#!/usr/bin/python
-
-import numpy
from mpi4py import MPI
-import time
+import numpy as np
-comm = MPI.COMM_WORLD
-rank = comm.Get_rank()
-size = comm.Get_size()
+tic = MPI.Wtime()
-a = 1
-b = 1000000
+x1 = -2.0
+x2 = 1.0
+y1 = -1.0
+y2 = 1.0
-perrank = b//size
-summ = numpy.zeros(1)
+w = 150
+h = 100
+maxit = 127
-comm.Barrier()
-start_time = time.time()
+def mandelbrot(x, y, maxit):
+ c = x + y*1j
+ z = 0 + 0j
+ it = 0
+ while abs(z) < 2 and it < maxit:
+ z = z**2 + c
+ it += 1
+ return it
-temp = 0
-for i in range(a + rank*perrank, a + (rank+1)*perrank):
- temp = temp + i
+comm = MPI.COMM_WORLD
+size = comm.Get_size()
+rank = comm.Get_rank()
-summ[0] = temp
+rmsg = np.empty(4, dtype='f')
+imsg = np.empty(3, dtype='i')
if rank == 0:
- total = numpy.zeros(1)
-else:
- total = None
-
-comm.Barrier()
-#collect the partial results and add to the total sum
-comm.Reduce(summ, total, op=MPI.SUM, root=0)
-
-stop_time = time.time()
+ rmsg[:] = [x1, x2, y1, y2]
+ imsg[:] = [w, h, maxit]
+
+comm.Bcast([rmsg, MPI.FLOAT], root=0)
+comm.Bcast([imsg, MPI.INT], root=0)
+
+x1, x2, y1, y2 = [float(r) for r in rmsg]
+w, h, maxit = [int(i) for i in imsg]
+dx = (x2 - x1) / w
+dy = (y2 - y1) / h
+
+# number of lines to compute here
+N = h // size + (h % size > rank)
+N = np.array(N, dtype='i')
+# indices of lines to compute here
+I = np.arange(rank, h, size, dtype='i')
+# compute local lines
+C = np.empty([N, w], dtype='i')
+for k in np.arange(N):
+ y = y1 + I[k] * dy
+ for j in np.arange(w):
+ x = x1 + j * dx
+ C[k, j] = mandelbrot(x, y, maxit)
+# gather results at root
+counts = 0
+indices = None
+cdata = None
+if rank == 0:
+ counts = np.empty(size, dtype='i')
+ indices = np.empty(h, dtype='i')
+ cdata = np.empty([h, w], dtype='i')
+comm.Gather(sendbuf=[N, MPI.INT],
+ recvbuf=[counts, MPI.INT],
+ root=0)
+comm.Gatherv(sendbuf=[I, MPI.INT],
+ recvbuf=[indices, (counts, None), MPI.INT],
+ root=0)
+comm.Gatherv(sendbuf=[C, MPI.INT],
+ recvbuf=[cdata, (counts*w, None), MPI.INT],
+ root=0)
+# reconstruct full result at root
+if rank == 0:
+ M = np.zeros([h,w], dtype='i')
+ M[indices, :] = cdata
+toc = MPI.Wtime()
+wct = comm.gather(toc-tic, root=0)
+if rank == 0:
+ for task, time in enumerate(wct):
+ print('wall clock time: %8.2f seconds (task %d)' % (time, task))
+ def mean(seq): return sum(seq)/len(seq)
+ print ('all tasks, mean: %8.2f seconds' % mean(wct))
+ print ('all tasks, min: %8.2f seconds' % min(wct))
+ print ('all tasks, max: %8.2f seconds' % max(wct))
+ print ('all tasks, sum: %8.2f seconds' % sum(wct))
+
+# eye candy (requires matplotlib)
if rank == 0:
- #add the rest numbers to 1 000 000
- for i in range(a + (size)*perrank, b+1):
- total[0] = total[0] + i
- print ("The sum of numbers from 1 to 1 000 000: ", int(total[0]))
- print ("time spent with ", size, " threads in milliseconds")
- print ("-----", int((time.time()-start_time)*1000), "-----")
+ try:
+ from matplotlib import pyplot as plt
+ plt.imshow(M, aspect='equal')
+ plt.spectral()
+ try:
+ import signal
+ def action(*args): raise SystemExit
+ signal.signal(signal.SIGALRM, action)
+ signal.alarm(2)
+ except:
+ pass
+ plt.show()
+ except:
+ pass
+MPI.COMM_WORLD.Barrier()
```
-Execute the code above as:
-
```console
-$ qsub -I -q qexp -l select=4:ncpus=16,walltime=00:30:00
-
-$ ml Python/3.5.2-intel-2017.00
-
-$ mpirun -n 2 python myprogram.py
+mpirun python mandelbrot.py
+...
+wall clock time: 0.26 seconds (task 505)
+wall clock time: 0.25 seconds (task 506)
+wall clock time: 0.24 seconds (task 507)
+wall clock time: 0.25 seconds (task 508)
+wall clock time: 0.25 seconds (task 509)
+wall clock time: 0.26 seconds (task 510)
+wall clock time: 0.25 seconds (task 511)
+all tasks, mean: 0.19 seconds
+all tasks, min: 0.00 seconds
+all tasks, max: 0.73 seconds
+all tasks, sum: 96.82 seconds
```
+In this example, we run MPI4Py-enabled code on 4 nodes, 128 cores per node (total of 512 processes), each Python process is bound to a different core. More examples and documentation can be found on [MPI for Python webpage][a].
+
You can increase `n` and watch the time lowering.
[1]: running_openmpi.md
--
GitLab
From 351d86fcec62b66c08cc9662b462021bb6b1a8e7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 14:55:24 +0200
Subject: [PATCH 45/58] Update docs.it4i/software/mpi/ompi-examples.md
---
docs.it4i/software/mpi/ompi-examples.md | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/docs.it4i/software/mpi/ompi-examples.md b/docs.it4i/software/mpi/ompi-examples.md
index 7b7e8ef7d..352b68409 100644
--- a/docs.it4i/software/mpi/ompi-examples.md
+++ b/docs.it4i/software/mpi/ompi-examples.md
@@ -8,7 +8,7 @@ There are two MPI examples, each using one of six different MPI interfaces:
### Hello World
-```c tab="C"
+```
/*
* Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
* University Research and Technology
@@ -39,7 +39,7 @@ int main(int argc, char* argv[])
}
```
-```c++ tab="C++"
+```
//
// Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
// University Research and Technology
@@ -78,7 +78,7 @@ int main(int argc, char **argv)
}
```
-```fortran tab="F mpi.h"
+```
C
C Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
C University Research and Technology
@@ -108,7 +108,7 @@ C
end
```
-```fortran tab="F use mpi"
+```
!
! Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
! University Research and Technology
@@ -139,7 +139,7 @@ program main
end
```
-```java tab="Java"
+```
/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -219,7 +219,7 @@ The Makefile assumes that the wrapper compilers mpicc, mpic++, and mpifort are i
Although the Makefile is tailored for OpenMPI (e.g., it checks the *mpi_info* command to see if you have support for C++, mpif.h, use mpi, and use mpi_f08 F90), all of the example programs are pure MPI, and therefore not specific to OpenMPI. Hence, you can use a different MPI implementation to compile and run these programs if you wish.
```console
-[login@cn204.anselm ]$ tar xvf ompi.tar.gz
+$ tar xvf ompi.tar.gz
./
./connectivity_c.c
./Hello.java
@@ -228,8 +228,8 @@ Although the Makefile is tailored for OpenMPI (e.g., it checks the *mpi_info* co
...
...
./hello_cxx.cc
-[login@cn204.anselm ]$ ml OpenMPI/2.1.1-GCC-6.3.0-2.27
-[login@cn204.anselm ]$ make
+$ ml OpenMPI/4.1.1-GCC-10.2.0
+$ make
mpicc -g hello_c.c -o hello_c
mpicc -g ring_c.c -o ring_c
mpicc -g connectivity_c.c -o connectivity_c
@@ -255,7 +255,7 @@ shmemcc -g oshmem_strided_puts.c -o oshmem_strided_puts
shmemcc -g oshmem_symmetric_data.c -o oshmem_symmetric_data
shmemfort -g hello_oshmemfh.f90 -o hello_oshmemfh
shmemfort -g ring_oshmemfh.f90 -o ring_oshmemfh
-[login@cn204.anselm ]$ find . -executable -type f
+$ find . -executable -type f
./hello_oshmem
./dtrace/myppriv.sh
./dtrace/partrace.sh
--
GitLab
From 4395561910693c371f70e634c5fe9b5a00ab749b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 14:59:44 +0200
Subject: [PATCH 46/58] Update docs.it4i/software/mpi/mpich.md, mkdocs.yml
files
---
.../mpi/{running-mpich2.md => mpich.md} | 39 ++++---------------
mkdocs.yml | 2 +-
2 files changed, 9 insertions(+), 32 deletions(-)
rename docs.it4i/software/mpi/{running-mpich2.md => mpich.md} (70%)
diff --git a/docs.it4i/software/mpi/running-mpich2.md b/docs.it4i/software/mpi/mpich.md
similarity index 70%
rename from docs.it4i/software/mpi/running-mpich2.md
rename to docs.it4i/software/mpi/mpich.md
index 3f01006dd..4bcee237d 100644
--- a/docs.it4i/software/mpi/running-mpich2.md
+++ b/docs.it4i/software/mpi/mpich.md
@@ -1,8 +1,8 @@
-# Running MPICH2
+# Running MPICH
-## MPICH2 Program Execution
+## MPICH Program Execution
-The MPICH2 programs use MPD daemon or SSH connection to spawn processes, no PBS support is needed. However, the PBS allocation is required to access compute nodes.
+The MPICH programs use MPD daemon or SSH connection to spawn processes, no PBS support is needed. However, the PBS allocation is required to access compute nodes.
### Basic Usage
@@ -12,35 +12,12 @@ The MPICH2 programs use MPD daemon or SSH connection to spawn processes, no PBS
Example:
```console
-$ qsub -q qexp -l select=4:ncpus=16 -I
- qsub: waiting for job 15210.srv11 to start
- qsub: job 15210.srv11 ready
-$ ml impi
-$ mpirun -ppn 1 -hostfile $PBS_NODEFILE ./helloworld_mpi.x
- Hello world! from rank 0 of 4 on host cn17
- Hello world! from rank 1 of 4 on host cn108
- Hello world! from rank 2 of 4 on host cn109
- Hello world! from rank 3 of 4 on host cn110
+$ qsub -q qprod -l select=4:ncpus=128:mpiprocs=128 -I
+$ ml MPICH/3.3.2-GCC-10.2.0
+$ mpirun hostname | wc -l
+512
```
-In this example, we allocate 4 nodes via the express queue interactively. We set up the intel MPI environment and interactively run the helloworld_mpi.x program. We request MPI to spawn 1 process per node.
-Note that the executable helloworld_mpi.x must be available within the same path on all nodes. This is automatically fulfilled on the /home and /scratch filesystem.
-
-You need to preload the executable if running on the local scratch /lscratch filesystem:
-
-```console
-$ pwd
- /lscratch/15210.srv11
-$ mpirun -ppn 1 -hostfile $PBS_NODEFILE cp /home/username/helloworld_mpi.x .
-$ mpirun -ppn 1 -hostfile $PBS_NODEFILE ./helloworld_mpi.x
- Hello world! from rank 0 of 4 on host cn17
- Hello world! from rank 1 of 4 on host cn108
- Hello world! from rank 2 of 4 on host cn109
- Hello world! from rank 3 of 4 on host cn110
-```
-
-In this example, we assume the executable helloworld_mpi.x is present on shared home directory. We run the `cp` command via `mpirun`, copying the executable from shared home to local scratch. Second `mpirun` will execute the binary in the /lscratch/15210.srv11 directory on nodes cn17, cn108, cn109 and cn110, one process per node.
-
!!! note
MPI process mapping may be controlled by PBS parameters.
@@ -51,7 +28,7 @@ The `mpiprocs` and `ompthreads` parameters allow for selection of number of runn
Follow this example to run one MPI process per node, 16 threads per process. Note that no options to `mpirun` are needed
```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=1:ompthreads=16 -I
+$ qsub -q qexp -l select=2:ncpus=128:mpiprocs=1:ompthreads=128 -I
$ ml mvapich2
$ mpirun ./helloworld_mpi.x
```
diff --git a/mkdocs.yml b/mkdocs.yml
index fbc4a8360..6c7b6fbea 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -178,7 +178,7 @@ nav:
- OpenMPI Examples: software/mpi/ompi-examples.md
- MPI4Py (MPI for Python): software/mpi/mpi4py-mpi-for-python.md
- Running OpenMPI: software/mpi/running_openmpi.md
- - Running MPICH2: software/mpi/running-mpich2.md
+ - Running MPICH: software/mpi/mpich.md
- Numerical Languages:
- Introduction: software/numerical-languages/introduction.md
- Clp: salomon/software/numerical-libraries/Clp.md
--
GitLab
From ffc68c10a9e030e45f0f17230f2103c11e07d12c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 15:03:02 +0200
Subject: [PATCH 47/58] Update mkdocs.yml Deleted
docs.it4i/software/mpi/mpich.md
---
docs.it4i/software/mpi/mpich.md | 136 --------------------------------
mkdocs.yml | 1 -
2 files changed, 137 deletions(-)
delete mode 100644 docs.it4i/software/mpi/mpich.md
diff --git a/docs.it4i/software/mpi/mpich.md b/docs.it4i/software/mpi/mpich.md
deleted file mode 100644
index 4bcee237d..000000000
--- a/docs.it4i/software/mpi/mpich.md
+++ /dev/null
@@ -1,136 +0,0 @@
-# Running MPICH
-
-## MPICH Program Execution
-
-The MPICH programs use MPD daemon or SSH connection to spawn processes, no PBS support is needed. However, the PBS allocation is required to access compute nodes.
-
-### Basic Usage
-
-!!! note
- Use the `mpirun` to execute the MPICH2 code.
-
-Example:
-
-```console
-$ qsub -q qprod -l select=4:ncpus=128:mpiprocs=128 -I
-$ ml MPICH/3.3.2-GCC-10.2.0
-$ mpirun hostname | wc -l
-512
-```
-
-!!! note
- MPI process mapping may be controlled by PBS parameters.
-
-The `mpiprocs` and `ompthreads` parameters allow for selection of number of running MPI processes per node as well as number of OpenMP threads per MPI process.
-
-### One MPI Process Per Node
-
-Follow this example to run one MPI process per node, 16 threads per process. Note that no options to `mpirun` are needed
-
-```console
-$ qsub -q qexp -l select=2:ncpus=128:mpiprocs=1:ompthreads=128 -I
-$ ml mvapich2
-$ mpirun ./helloworld_mpi.x
-```
-
-In this example, we demonstrate recommended way to run an MPI application, using 1 MPI processes per node and 16 threads per socket, on 4 nodes.
-
-### Two MPI Processes Per Node
-
-Follow this example to run two MPI processes per node, 8 threads per process. Note the options to `mpirun` for `mvapich2`. No options are needed for `impi`.
-
-```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=2:ompthreads=8 -I
-$ ml mvapich2
-$ mpirun -bind-to numa ./helloworld_mpi.x
-```
-
-In this example, we demonstrate recommended way to run an MPI application, using 2 MPI processes per node and 8 threads per socket, each process and its threads bound to a separate processor socket of the node, on 4 nodes
-
-### 16 MPI Processes Per Node
-
-Follow this example to run 16 MPI processes per node, 1 thread per process. Note the options to `mpirun` for `mvapich2`. No options are needed for `impi`.
-
-```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=16:ompthreads=1 -I
-$ ml mvapich2
-$ mpirun -bind-to core ./helloworld_mpi.x
-```
-
-In this example, we demonstrate recommended way to run an MPI application, using 16 MPI processes per node, single threaded. Each process is bound to separate processor core, on 4 nodes.
-
-### OpenMP Thread Affinity
-
-!!! note
- Important! Bind every OpenMP thread to a core!
-
-In the previous two examples with one or two MPI processes per node, the operating system might still migrate OpenMP threads between cores. You might want to avoid this by setting these environment variable for GCC OpenMP:
-
-```console
-$ export GOMP_CPU_AFFINITY="0-15"
-```
-
-or this one for Intel OpenMP:
-
-```console
-$ export KMP_AFFINITY=granularity=fine,compact,1,0
-```
-
-As of OpenMP 4.0 (supported by GCC 4.9 and later and Intel 14.0 and later) the following variables may be used for Intel or GCC:
-
-```console
-$ export OMP_PROC_BIND=true
-$ export OMP_PLACES=cores
-```
-
-## MPICH2 Process Mapping and Binding
-
-The `mpirun` allows for precise selection of how the MPI processes will be mapped to the computational nodes and how these processes will bind to particular processor sockets and cores.
-
-### Machinefile
-
-Process mapping may be controlled by specifying a machinefile input to the mpirun program. Altough all implementations of MPI provide means for process mapping and binding, following examples are valid for the `impi` and `mvapich2` only.
-
-Example machinefile
-
-```console
- cn110.bullx
- cn109.bullx
- cn108.bullx
- cn17.bullx
- cn108.bullx
-```
-
-Use the machinefile to control process placement
-
-```console
-$ mpirun -machinefile machinefile helloworld_mpi.x
- Hello world! from rank 0 of 5 on host cn110
- Hello world! from rank 1 of 5 on host cn109
- Hello world! from rank 2 of 5 on host cn108
- Hello world! from rank 3 of 5 on host cn17
- Hello world! from rank 4 of 5 on host cn108
-```
-
-In this example, we see that ranks have been mapped on nodes according to the order in which nodes show in the machinefile
-
-### Process Binding
-
-The Intel MPI automatically binds each process and its threads to the corresponding portion of cores on the processor socket of the node, no options needed. The binding is primarily controlled by environment variables. Read more about mpi process binding on [Intel website][a]. The MPICH2 uses the `-bindto` option. Use `-bindto numa` or `-bindto core` to bind the process on single core or entire socket.
-
-### Bindings Verification
-
-In all cases, binding and threading may be verified by executing
-
-```console
-$ mpirun -bindto numa numactl --show
-$ mpirun -bindto numa echo $OMP_NUM_THREADS
-```
-
-## Intel MPI on Xeon Phi
-
-The [MPI section of Intel Xeon Phi chapter][1] provides details on how to run Intel MPI code on Xeon Phi architecture.
-
-[1]: ../intel/intel-xeon-phi-salomon.md
-
-[a]: https://software.intel.com/sites/products/documentation/hpc/ics/impi/41/lin/Reference_Manual/Environment_Variables_Process_Pinning.htm
diff --git a/mkdocs.yml b/mkdocs.yml
index 6c7b6fbea..655604b41 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -178,7 +178,6 @@ nav:
- OpenMPI Examples: software/mpi/ompi-examples.md
- MPI4Py (MPI for Python): software/mpi/mpi4py-mpi-for-python.md
- Running OpenMPI: software/mpi/running_openmpi.md
- - Running MPICH: software/mpi/mpich.md
- Numerical Languages:
- Introduction: software/numerical-languages/introduction.md
- Clp: salomon/software/numerical-libraries/Clp.md
--
GitLab
From f4c2d318ffc268d672ca5d650106734b26fa1083 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Tue, 21 Sep 2021 15:04:08 +0200
Subject: [PATCH 48/58] Update mkdocs.yml Deleted
docs.it4i/software/mpi/running_openmpi.md
---
docs.it4i/software/mpi/running_openmpi.md | 208 ----------------------
mkdocs.yml | 1 -
2 files changed, 209 deletions(-)
delete mode 100644 docs.it4i/software/mpi/running_openmpi.md
diff --git a/docs.it4i/software/mpi/running_openmpi.md b/docs.it4i/software/mpi/running_openmpi.md
deleted file mode 100644
index c8448b66b..000000000
--- a/docs.it4i/software/mpi/running_openmpi.md
+++ /dev/null
@@ -1,208 +0,0 @@
-# Running OpenMPI
-
-## OpenMPI Program Execution
-
-The OpenMPI programs may be executed only via the PBS Workload manager, by entering an appropriate queue. On Salomon, the **OpenMPI 1.8.6** is an OpenMPI-based MPI implementation.
-
-### Basic Usage
-
-!!! note
- Use the `mpiexec` to run the OpenMPI code.
-
-Example (for Anselm):
-
-```console
-$ qsub -q qexp -l select=4:ncpus=16 -I
- qsub: waiting for job 15210.srv11 to start
- qsub: job 15210.srv11 ready
-$ pwd
- /home/username
-$ ml OpenMPI
-$ mpiexec -pernode ./helloworld_mpi.x
- Hello world! from rank 0 of 4 on host cn17
- Hello world! from rank 1 of 4 on host cn108
- Hello world! from rank 2 of 4 on host cn109
- Hello world! from rank 3 of 4 on host cn110
-```
-
-!!! note
- In this example, the directive `-pernode` is used to run only **one task per node**, which is normally an unwanted behavior (unless you want to run hybrid code with just one MPI and 16 OpenMP tasks per node). In normal MPI programs, omit the `-pernode` directive to run up to 16 MPI tasks per each node using `mpiprocs` (see the [examples][1] below).
-
-In this example, we allocate 4 nodes via the express queue interactively. We set up the OpenMPI environment and interactively run the helloworld_mpi.x program. Note that the executable helloworld_mpi.x must be available within the same path on all nodes. This is automatically fulfilled on the /home and /scratch filesystem.
-
-You need to preload the executable, if running on the local scratch /lscratch filesystem:
-
-```console
-$ pwd
- /lscratch/15210.srv11
-$ mpiexec -pernode --preload-binary ./helloworld_mpi.x
- Hello world! from rank 0 of 4 on host cn17
- Hello world! from rank 1 of 4 on host cn108
- Hello world! from rank 2 of 4 on host cn109
- Hello world! from rank 3 of 4 on host cn110
-```
-
-In this example, we assume the executable helloworld_mpi.x is present on compute node cn17 on local scratch. We call the `mpiexec` with the `--preload-binary` argument (valid for OpenMPI). The `mpiexec` will copy the executable from cn17 to the /lscratch/15210.srv11 directory on cn108, cn109, and cn110 and execute the program.
-
-!!! note
- MPI process mapping may be controlled by PBS parameters.
-
-The `mpiprocs` and `ompthreads` parameters allow for selection of number of running MPI processes per node as well as number of OpenMP threads per MPI process.
-
-### One MPI Process Per Node
-
-Follow this example to run one MPI process per node, 16 threads per process (**on Salomon, try 24 threads in following examples**):
-
-```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=1:ompthreads=16 -I
-$ ml OpenMPI
-$ mpiexec --bind-to-none ./helloworld_mpi.x
-```
-
-In this example, we demonstrate the recommended way to run an MPI application, using 1 MPI processes per node and 16 threads per socket, on 4 nodes.
-
-### Two MPI Processes Per Node
-
-Follow this example to run two MPI processes per node, 8 threads per process. Note the options to mpiexec.
-
-```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=2:ompthreads=8 -I
-$ ml openmpi
-$ mpiexec -bysocket -bind-to-socket ./helloworld_mpi.x
-```
-
-In this example, we demonstrate the recommended way to run an MPI application, using 2 MPI processes per node and 8 threads per socket, each process and its threads bound to a separate processor socket of the node, on 4 nodes.
-
-### 16 MPI Processes Per Node
-
-Follow this example to run 16 MPI processes per node, 1 thread per process. Note the options to mpiexec:
-
-```console
-$ qsub -q qexp -l select=4:ncpus=16:mpiprocs=16:ompthreads=1 -I
-$ ml OpenMPI
-$ mpiexec -bycore -bind-to-core ./helloworld_mpi.x
-```
-
-In this example, we demonstrate the recommended way to run an MPI application, using 16 MPI processes per node, single threaded. Each process is bound to separate processor core, on 4 nodes.
-
-### OpenMP Thread Affinity
-
-!!! note
- Important! Bind every OpenMP thread to a core!
-
-In the previous two examples with one or two MPI processes per node, the operating system might still migrate OpenMP threads between cores. You might want to avoid this by setting this environment variable for GCC OpenMP:
-
-```console
-$ export GOMP_CPU_AFFINITY="0-15"
-```
-
-or this one for Intel OpenMP:
-
-```console
-$ export KMP_AFFINITY=granularity=fine,compact,1,0
-```
-
-As of OpenMP 4.0 (supported by GCC 4.9 and later and Intel 14.0 and later), the following variables may be used for Intel or GCC:
-
-```console
-$ export OMP_PROC_BIND=true
-$ export OMP_PLACES=cores
-```
-
-## OpenMPI Process Mapping and Binding
-
-`mpiexec` allows for precise selection of how the MPI processes will be mapped to the computational nodes and how these processes will bind to particular processor sockets and cores.
-
-MPI process mapping may be specified by a hostfile or rankfile input to the `mpiexec` program. Altough all implementations of MPI provide means for process mapping and binding, the following examples are valid for the OpenMPI only.
-
-### Hostfile
-
-Example hostfile:
-
-```console
- cn110.bullx
- cn109.bullx
- cn108.bullx
- cn17.bullx
-```
-
-Use the hostfile to control process placement:
-
-```console
-$ mpiexec -hostfile hostfile ./helloworld_mpi.x
- Hello world! from rank 0 of 4 on host cn110
- Hello world! from rank 1 of 4 on host cn109
- Hello world! from rank 2 of 4 on host cn108
- Hello world! from rank 3 of 4 on host cn17
-```
-
-In this example, we see that ranks have been mapped on nodes according to the order in which nodes show in the hostfile.
-
-### Rankfile
-
-Exact control of MPI process placement and resource binding is provided by specifying a rankfile.
-
-!!! note
- Appropriate binding may boost performance of your application.
-
-Example rankfile:
-
-```console
- rank 0=cn110.bullx slot=1:0,1
- rank 1=cn109.bullx slot=0:*
- rank 2=cn108.bullx slot=1:1-2
- rank 3=cn17.bullx slot=0:1,1:0-2
- rank 4=cn109.bullx slot=0:*,1:*
-```
-
-This rankfile assumes 5 ranks will be running on 4 nodes and provides exact mapping and binding of the processes to the processor sockets and cores.
-
-Explanation:
-rank 0 will be bounded to cn110, socket1 core0 and core1
-rank 1 will be bounded to cn109, socket0 all cores
-rank 2 will be bounded to cn108, socket1 core1 and core2
-rank 3 will be bounded to cn17, socket0 core1, socket1 core0, core1, core2
-rank 4 will be bounded to cn109, all cores on both sockets
-
-```console
-$ mpiexec -n 5 -rf rankfile --report-bindings ./helloworld_mpi.x
- [cn17:11180] MCW rank 3 bound to socket 0[core 1] socket 1[core 0-2]: [. B . . . . . .][B B B . . . . .] (slot list 0:1,1:0-2)
- [cn110:09928] MCW rank 0 bound to socket 1[core 0-1]: [. . . . . . . .][B B . . . . . .] (slot list 1:0,1)
- [cn109:10395] MCW rank 1 bound to socket 0[core 0-7]: [B B B B B B B B][. . . . . . . .] (slot list 0:*)
- [cn108:10406] MCW rank 2 bound to socket 1[core 1-2]: [. . . . . . . .][. B B . . . . .] (slot list 1:1-2)
- [cn109:10406] MCW rank 4 bound to socket 0[core 0-7] socket 1[core 0-7]: [B B B B B B B B][B B B B B B B B] (slot list 0:*,1:*)
- Hello world! from rank 3 of 5 on host cn17
- Hello world! from rank 1 of 5 on host cn109
- Hello world! from rank 0 of 5 on host cn110
- Hello world! from rank 4 of 5 on host cn109
- Hello world! from rank 2 of 5 on host cn108
-```
-
-In this example, we run 5 MPI processes (5 ranks) on four nodes. The rankfile defines how the processes will be mapped on the nodes, sockets and cores. The `--report-bindings` option was used to print out the actual process location and bindings. Note that ranks 1 and 4 run on the same node and their core binding overlaps.
-
-The user must provide the correct number of ranks, sockets, and cores.
-
-### Bindings Verification
-
-In all cases, binding and threading may be verified by executing for example:
-
-```console
-$ mpiexec -bysocket -bind-to-socket --report-bindings echo
-$ mpiexec -bysocket -bind-to-socket numactl --show
-$ mpiexec -bysocket -bind-to-socket echo $OMP_NUM_THREADS
-```
-
-## Changes in OpenMPI 1.8
-
-Some options have changed in OpenMPI version 1.8.
-
-| version 1.6.5 | version 1.8.1 |
-| ---------------- | ------------------- |
-| --bind-to-none | --bind-to none |
-| --bind-to-core | --bind-to core |
-| --bind-to-socket | --bind-to socket |
-| -bysocket | --map-by socket |
-| -bycore | --map-by core |
-| -pernode | --map-by ppr:1:node |
-
-[1]: #one-mpi-process-per-node
\ No newline at end of file
diff --git a/mkdocs.yml b/mkdocs.yml
index 655604b41..b4e27322d 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -177,7 +177,6 @@ nav:
- Introduction: software/mpi/mpi.md
- OpenMPI Examples: software/mpi/ompi-examples.md
- MPI4Py (MPI for Python): software/mpi/mpi4py-mpi-for-python.md
- - Running OpenMPI: software/mpi/running_openmpi.md
- Numerical Languages:
- Introduction: software/numerical-languages/introduction.md
- Clp: salomon/software/numerical-libraries/Clp.md
--
GitLab
From 399cb77fff6ce07aa6d6b448a6e044b05c162ab5 Mon Sep 17 00:00:00 2001
From: Jan Siwiec <jan.siwiec@vsb.cz>
Date: Wed, 22 Sep 2021 07:32:05 +0200
Subject: [PATCH 49/58] trailing space
---
docs.it4i/software/chemistry/gaussian.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs.it4i/software/chemistry/gaussian.md b/docs.it4i/software/chemistry/gaussian.md
index a0c682461..4e5adee1c 100644
--- a/docs.it4i/software/chemistry/gaussian.md
+++ b/docs.it4i/software/chemistry/gaussian.md
@@ -32,7 +32,7 @@ Gaussian is available on Karolina, Salomon, Barbora, and DGX-2 systems in the la
| Module | CPU support | GPU support | Parallelization | Note | Barbora | Salomon | DGX-2 | Karolina |
|--------------------------------------|-------------|--------------|-----------------|---------------------|---------|---------|-------| -------- |
| Gaussian/09-a02 | AVX2 | No | SMP + Linda | IT4I compiled | Yes | No | No | Yes |
-| Gaussian/09-d01 | AVX2 | No | SMP + Linda | IT4I compiled | Yes | No | No | Yes |
+| Gaussian/09-d01 | AVX2 | No | SMP + Linda | IT4I compiled | Yes | No | No | Yes |
| Gaussian/16_rev_c0-binary | AVX2 | Yes | SMP | Binary distribution | Yes | No | Yes | No |
| Gaussian/16_rev_c0-binary-Linda | AVX2 | Yes | SMP + Linda | Binary distribution | Yes | Yes | No | No |
| Gaussian/16_rev_c0-CascadeLake | AVX-512 | No | SMP | IT4I compiled | Yes | No | No | No |
--
GitLab
From eee68ed68d845eae3c16fda1752dd55267e8dada Mon Sep 17 00:00:00 2001
From: Jan Siwiec <jan.siwiec@vsb.cz>
Date: Wed, 22 Sep 2021 07:52:30 +0200
Subject: [PATCH 50/58] removed blank line
---
docs.it4i/software/mpi/mpi.md | 1 -
1 file changed, 1 deletion(-)
diff --git a/docs.it4i/software/mpi/mpi.md b/docs.it4i/software/mpi/mpi.md
index 7e0dc5761..298f7da86 100644
--- a/docs.it4i/software/mpi/mpi.md
+++ b/docs.it4i/software/mpi/mpi.md
@@ -87,7 +87,6 @@ When using Intel MPI, use the following MPI wrappers:
For module `intel/2020b`
-
```console
$ mpiicc -v
mpiicc for the Intel(R) MPI Library 2019 Update 9 for Linux*
--
GitLab
From ee1a9669c9745965df6925ae645c4637012165a6 Mon Sep 17 00:00:00 2001
From: Jan Siwiec <jan.siwiec@vsb.cz>
Date: Wed, 22 Sep 2021 07:55:29 +0200
Subject: [PATCH 51/58] Update tensorflow.md
---
docs.it4i/software/machine-learning/tensorflow.md | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/docs.it4i/software/machine-learning/tensorflow.md b/docs.it4i/software/machine-learning/tensorflow.md
index 44c76851b..2059b8306 100644
--- a/docs.it4i/software/machine-learning/tensorflow.md
+++ b/docs.it4i/software/machine-learning/tensorflow.md
@@ -8,6 +8,7 @@ $ ml av Tensorflow
```
<!---
+
## Salomon Modules
Salomon provides (besides other) these TensorFlow modules:
@@ -23,6 +24,7 @@ Salomon provides (besides other) these TensorFlow modules:
* GCC/7.1.0-2.28
* Python/3.6.1
* protobuf/3.2.0-GCC-7.1.0-2.28-Python-3.6.1
+
-->
## TensorFlow Application Example
@@ -38,6 +40,7 @@ print(sess.run(c))
```
<!---
+
2021-04-08
It's necessary to load the correct NumPy module along with the Tensorflow one.
@@ -46,4 +49,5 @@ It's necessary to load the correct NumPy module along with the Tensorflow one.
As of 2021-03-23, TensorFlow is made available only on the Salomon cluster
Tensorflow-tensorboard/1.5.1-Py-3.6 has not been not tested.
+
-->
--
GitLab
From 53098f81fc741f390582f3fcbfde0bff90e83208 Mon Sep 17 00:00:00 2001
From: Jan Siwiec <jan.siwiec@vsb.cz>
Date: Wed, 22 Sep 2021 07:57:40 +0200
Subject: [PATCH 52/58] fixed blank line
---
docs.it4i/software/lang/csc.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/docs.it4i/software/lang/csc.md b/docs.it4i/software/lang/csc.md
index a77bedc71..40c10f978 100644
--- a/docs.it4i/software/lang/csc.md
+++ b/docs.it4i/software/lang/csc.md
@@ -82,6 +82,7 @@ csharp> from f in Directory.GetFiles ("mydirectory")
```
<!--
+
## MPI.NET
MPI is available for mono:
@@ -119,6 +120,7 @@ Greetings from node 0 of 4 running on cn204
Greetings from node 3 of 4 running on cn199
Greetings from node 1 of 4 running on cn199
```
+
-->
For more information, see the [Mono documentation page][a].
--
GitLab
From 5fa6368393f2f14f686b825c048e79aa9910502a Mon Sep 17 00:00:00 2001
From: Jan Siwiec <jan.siwiec@vsb.cz>
Date: Wed, 22 Sep 2021 07:59:57 +0200
Subject: [PATCH 53/58] fixed code blocks
---
docs.it4i/software/intel/intel-suite/intel-compilers.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs.it4i/software/intel/intel-suite/intel-compilers.md b/docs.it4i/software/intel/intel-suite/intel-compilers.md
index 47566dc41..afbfa6bfe 100644
--- a/docs.it4i/software/intel/intel-suite/intel-compilers.md
+++ b/docs.it4i/software/intel/intel-suite/intel-compilers.md
@@ -28,7 +28,7 @@ Intel compilers provide vectorization of the code via the AVX-2/AVX-512 instruct
For maximum performance on the Barbora cluster compute nodes, compile your programs using the AVX-512 instructions, with reporting where the vectorization was used. We recommend the following compilation options for high performance.
-``` info
+!!! info
Barbora non-accelerated nodes support AVX-512 instructions (cn1-cn192).
```console
@@ -43,7 +43,7 @@ For maximum performance on the Barbora GPU nodes or Karolina cluster compute nod
$ icc -ipo -O3 -xCORE-AVX2 -qopt-report1 -qopt-report-phase=vec myprog.c mysubroutines.c -o myprog.x
```
-``` warn
+!!! warn
Karolina cluster has AMD cpu, use compiler options `-march=core-avx2`.
In this example, we compile the program enabling interprocedural optimizations between source files (`-ipo`), aggressive loop optimizations (`-O3`), and vectorization (`-xCORE-AVX2`).
--
GitLab
From 649c90a2cbf645e36b50de4bef03682151f29419 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Wed, 22 Sep 2021 09:09:40 +0200
Subject: [PATCH 54/58] Update
docs.it4i/software/numerical-languages/introduction.md,
docs.it4i/software/numerical-languages/matlab.md, mkdocs.yml files
---
.../software/numerical-languages/introduction.md | 15 ++++++++++++---
docs.it4i/software/numerical-languages/matlab.md | 3 +--
mkdocs.yml | 1 -
3 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/docs.it4i/software/numerical-languages/introduction.md b/docs.it4i/software/numerical-languages/introduction.md
index d12f02c8b..f4f9f3c2d 100644
--- a/docs.it4i/software/numerical-languages/introduction.md
+++ b/docs.it4i/software/numerical-languages/introduction.md
@@ -11,7 +11,10 @@ This section contains a collection of high-level interpreted languages, primaril
MATLAB® is a high-level language and interactive environment for numerical computation, visualization, and programming.
```console
-$ ml MATLAB
+$ ml av MATLAB
+-------------- /apps/modules/math --------------
+ MATLAB/2021a
+$ ml MATLAB/2021a
$ matlab
```
@@ -22,8 +25,11 @@ Read more at the [MATLAB page][1].
GNU Octave is a high-level interpreted language, primarily intended for numerical computations. The Octave language is quite similar to MATLAB so that most programs are easily portable.
```console
-$ ml Octave
-$ octave
+$ ml av Octave
+-------------- /apps/modules/math --------------
+Octave/6.3.0-intel-2020b-without-X11
+$ ml Octave/6.3.0-intel-2020b-without-X11
+$ octave
```
Read more at the [Octave page][2].
@@ -33,6 +39,9 @@ Read more at the [Octave page][2].
The R is an interpreted language and environment for statistical computing and graphics.
```console
+$ ml av R/
+-------------- /apps/modules/math --------------
+
$ ml R
$ R
```
diff --git a/docs.it4i/software/numerical-languages/matlab.md b/docs.it4i/software/numerical-languages/matlab.md
index 62ab0ebe2..87cb17e01 100644
--- a/docs.it4i/software/numerical-languages/matlab.md
+++ b/docs.it4i/software/numerical-languages/matlab.md
@@ -2,10 +2,9 @@
## Introduction
-!!!notes
+!!! notes
Since 2016, the MATLAB module is not updated anymore and purchase of new licenses is not planned.
However, due to [e-infra integration][b], IT4Innovations may have access to recent MATLAB versions from cooperating organizations in the future.
- More information will be available in May, 2021.
MATLAB is available in versions R2015a and R2015b. There are always two variants of the release:
diff --git a/mkdocs.yml b/mkdocs.yml
index b4e27322d..96fd623f9 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -182,7 +182,6 @@ nav:
- Clp: salomon/software/numerical-libraries/Clp.md
- R: software/numerical-languages/r.md
- Matlab: software/numerical-languages/matlab.md
- - Matlab 2013-2014: software/numerical-languages/matlab_1314.md
- Octave: software/numerical-languages/octave.md
- OpenCoarrays: software/numerical-languages/opencoarrays.md
- Numerical Libraries:
--
GitLab
From ae5d7b33fc82e37469ef8299c06704498f5d0719 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Wed, 22 Sep 2021 11:03:39 +0200
Subject: [PATCH 55/58] Update
docs.it4i/software/numerical-languages/octave.md,
docs.it4i/software/numerical-languages/opencoarrays.md,
docs.it4i/software/numerical-languages/r.md,
docs.it4i/software/numerical-libraries/fftw.md,
docs.it4i/software/numerical-libraries/gsl.md,
docs.it4i/software/numerical-libraries/hdf5.md,
docs.it4i/software/numerical-libraries/intel-numerical-libraries.md,
docs.it4i/software/numerical-libraries/petsc.md,
docs.it4i/software/numerical-languages/matlab.md files Deleted
docs.it4i/software/numerical-languages/matlab_1314.md,
docs.it4i/software/numerical-libraries/magma-for-intel-xeon-phi.md files
---
.../software/numerical-languages/matlab.md | 110 +++------
.../numerical-languages/matlab_1314.md | 211 ------------------
.../software/numerical-languages/octave.md | 63 +-----
.../numerical-languages/opencoarrays.md | 14 +-
docs.it4i/software/numerical-languages/r.md | 27 +--
.../software/numerical-libraries/fftw.md | 31 +--
docs.it4i/software/numerical-libraries/gsl.md | 13 +-
.../software/numerical-libraries/hdf5.md | 14 +-
.../intel-numerical-libraries.md | 32 ++-
.../magma-for-intel-xeon-phi.md | 83 -------
.../software/numerical-libraries/petsc.md | 7 +-
11 files changed, 105 insertions(+), 500 deletions(-)
delete mode 100644 docs.it4i/software/numerical-languages/matlab_1314.md
delete mode 100644 docs.it4i/software/numerical-libraries/magma-for-intel-xeon-phi.md
diff --git a/docs.it4i/software/numerical-languages/matlab.md b/docs.it4i/software/numerical-languages/matlab.md
index 87cb17e01..2f16bc4c2 100644
--- a/docs.it4i/software/numerical-languages/matlab.md
+++ b/docs.it4i/software/numerical-languages/matlab.md
@@ -2,10 +2,6 @@
## Introduction
-!!! notes
- Since 2016, the MATLAB module is not updated anymore and purchase of new licenses is not planned.
- However, due to [e-infra integration][b], IT4Innovations may have access to recent MATLAB versions from cooperating organizations in the future.
-
MATLAB is available in versions R2015a and R2015b. There are always two variants of the release:
* Non-commercial or so-called EDU variant, which can be used for common research and educational purposes.
@@ -21,8 +17,13 @@ The EDU variant is marked as default. If you need other version or variant, load
```console
$ ml av MATLAB
+------------------------------------------------ /apps/modules/math -------------------------------------------------
+ MATLAB/R2015b MATLAB/2021a (D)
```
+!!! info
+ Version 2021a is e-infra licence, without cluster licenses - only basic functionality.
+
If you need to use the MATLAB GUI to prepare your MATLAB programs, you can use MATLAB directly on the login nodes. However, for all computations, use MATLAB on the compute nodes via PBS Pro scheduler.
If you require the MATLAB GUI, follow the general information about [running graphical applications][1].
@@ -53,17 +54,21 @@ Delete previously used file mpiLibConf.m, we have observed crashes when using In
To use Distributed Computing, you first need to setup a parallel profile. We have provided the profile for you, you can either import it in the MATLAB command line:
+* Salomon cluster - SalomonPBSPro.settings
+* Karolina cluster - KarolinaPBSPro.settings
+* Barbora cluster - BarboraPBSPro.settings
+
```console
-> parallel.importProfile('/apps/all/MATLAB/2015b-EDU/SalomonPBSPro.settings')
+> parallel.importProfile('/apps/all/MATLAB/R2015b/KarolinaPBSPro.settings')
ans =
-SalomonPBSPro
+KarolinaPBSPro
```
or in the GUI, go to tab *HOME -> Parallel -> Manage Cluster Profiles...*, click *Import* and navigate to:
-/apps/all/MATLAB/2015b-EDU/SalomonPBSPro.settings
+/apps/all/MATLAB/R2015b/KarolinaPBSPro.settings
With the new mode, MATLAB itself launches the workers via PBS, so you can use either an interactive mode or a batch mode on one node, but the actual parallel processing will be done in a separate job started by MATLAB itself. Alternatively, you can use a "local" mode to run parallel code on just a single node.
@@ -72,8 +77,7 @@ With the new mode, MATLAB itself launches the workers via PBS, so you can use ei
The following example shows how to start the interactive session with support for MATLAB GUI. For more information about GUI based applications, see [this page][1].
```console
-$ xhost +
-$ qsub -I -v DISPLAY=$(uname -n):$(echo $DISPLAY | cut -d ':' -f 2) -A NONE-0-0 -q qexp -l select=1 -l walltime=00:30:00 -l license__matlab-edu__MATLAB=1
+$ qsub -I -X -q qexp -l select=1 -l walltime=00:30:00 -l license__matlab-edu__MATLAB=1
```
This `qsub` command example shows how to run MATLAB on a single node.
@@ -83,7 +87,7 @@ The second part of the command shows how to request all necessary licenses. In t
Once the access to compute nodes is granted by PBS, the user can load following modules and start MATLAB:
```console
-$ ml MATLAB/2015a-EDU
+$ ml MATLAB/R2015b
$ matlab &
```
@@ -95,17 +99,18 @@ To run MATLAB in a batch mode, write a MATLAB script, then write a bash jobscrip
#!/bin/bash
#PBS -A PROJECT ID
#PBS -q qprod
-#PBS -l select=1:ncpus=24:mpiprocs=24:ompthreads=1
+#PBS -l select=1:ncpus=128:mpiprocs=128:ompthreads=1
# change to shared scratch directory
-SCR=/scratch/.../$USER/$PBS_JOBID # change path in according to the cluster
-mkdir -p $SCR ; cd $SCR || exit
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
# copy input file to scratch
cp $PBS_O_WORKDIR/matlabcode.m .
# load modules
-ml MATLAB/2015a-EDU
+ml MATLAB/R2015b
# execute the calculation
matlab -nodisplay -r matlabcode > output.out
@@ -141,10 +146,10 @@ This script creates the scheduler object *cluster* of the type *local* that star
!!! hint
Every MATLAB script that needs to initialize/use `matlabpool` has to contain these three lines prior to calling the `parpool(sched, ...)` function.
-The last step is to start `matlabpool` with the *cluster* object and a correct number of workers. We have 24 cores per node, so we start 24 workers.
+The last step is to start `matlabpool` with the *cluster* object and a correct number of workers. We have 128 cores per node, so we start 128 workers.
```console
-parpool(cluster,24);
+parpool(cluster,128);
... parallel code ...
@@ -159,7 +164,7 @@ The complete example showing how to use Distributed Computing Toolbox in local m
cluster = parcluster('local');
cluster
-parpool(cluster,24);
+parpool(cluster,128);
n=2000;
@@ -184,17 +189,17 @@ You can copy and paste the example in a .m file and execute. Note that the `parp
### Parallel MATLAB Batch Job Using PBS Mode (Workers Spawned in a Separate Job)
-This mode uses the PBS scheduler to launch the parallel pool. It uses the SalomonPBSPro profile that needs to be imported to Cluster Manager, as mentioned before. This method uses MATLAB's PBS Scheduler interface - it spawns the workers in a separate job submitted by MATLAB using qsub.
+This mode uses the PBS scheduler to launch the parallel pool. It uses the KarolinaPBSPro profile that needs to be imported to Cluster Manager, as mentioned before. This method uses MATLAB's PBS Scheduler interface - it spawns the workers in a separate job submitted by MATLAB using qsub.
This is an example of an m-script using the PBS mode:
```matlab
-cluster = parcluster('SalomonPBSPro');
+cluster = parcluster('KarolinaPBSPro');
set(cluster, 'SubmitArguments', '-A OPEN-0-0');
-set(cluster, 'ResourceTemplate', '-q qprod -l select=10:ncpus=24');
-set(cluster, 'NumWorkers', 240);
+set(cluster, 'ResourceTemplate', '-q qprod -l select=10:ncpus=128');
+set(cluster, 'NumWorkers', 1280);
-pool = parpool(cluster,240);
+pool = parpool(cluster,1280);
n=2000;
@@ -219,73 +224,12 @@ Note that we first construct a cluster object using the imported profile, then s
You can start this script using the batch mode the same way as in the Local mode example.
-### Parallel MATLAB Batch With Direct Launch (Workers Spawned Within the Existing Job)
-
-This method is a "hack" invented by us to emulate the `mpiexec` functionality found in previous MATLAB versions. We leverage the MATLAB Generic Scheduler interface, but instead of submitting the workers to PBS, we launch the workers directly within the running job, thus we avoid the issues with master script and workers running in separate jobs (issues with license not available, waiting for the worker's job to spawn, etc.)
-
-!!! warning
- This method is experimental.
-
-For this method, you need to use the SalomonDirect profile, import it using [the same way as SalomonPBSPro][2].
-
-This is an example of an m-script using direct mode:
-
-```matlab
-parallel.importProfile('/apps/all/MATLAB/2015b-EDU/SalomonDirect.settings')
-cluster = parcluster('SalomonDirect');
-set(cluster, 'NumWorkers', 48);
-
-pool = parpool(cluster, 48);
-
-n=2000;
-
-W = rand(n,n);
-W = distributed(W);
-x = (1:n)';
-x = distributed(x);
-spmd
-[~, name] = system('hostname')
-
- T = W*x; % Calculation performed on labs, in parallel.
- % T and W are both codistributed arrays here.
-end
-whos % T and W are both distributed arrays here.
-
-% shut down parallel pool
-delete(gcp('nocreate'))
-```
-
### Non-Interactive Session and Licenses
If you want to run batch jobs with MATLAB, be sure to request appropriate license features with the PBS Pro scheduler, at least the `-l license__matlab-edu__MATLAB=1` for the EDU variant of MATLAB. For more information about how to check the license features states and how to request them with PBS Pro, [look here][3].
In case of non-interactive session, read the [following information][3] on how to modify the `qsub` command to test for available licenses prior getting the resource allocation.
-### MATLAB Distributed Computing Engines Start Up Time
-
-Starting MATLAB workers is an expensive process that requires certain amount of time. For more information, see the following table:
-
-| compute nodes | number of workers | start-up time[s] |
-| ------------- | ----------------- | ---------------- |
-| 16 | 384 | 831 |
-| 8 | 192 | 807 |
-| 4 | 96 | 483 |
-| 2 | 48 | 16 |
-
-## MATLAB on UV2000
-
-The UV2000 machine available in the qfat queue can be used for MATLAB computations. This is an SMP NUMA machine with a large amount of RAM, which can be beneficial for certain types of MATLAB jobs. CPU cores are allocated in chunks of 8 for this machine.
-
-You can use MATLAB on UV2000 in two parallel modes:
-
-### Threaded Mode
-
-Since this is an SMP machine, you can completely avoid using Parallel Toolbox and use only MATLAB's threading. MATLAB will automatically detect the number of cores you have allocated and will set `maxNumCompThreads` accordingly and certain operations, such as fft, eig, svd, etc. will be automatically run in threads. The advantage of this mode is that you do not need to modify your existing sequential codes.
-
-### Local Cluster Mode
-
-You can also use Parallel Toolbox on UV2000. Use [local cluster mode][4], the "SalomonPBSPro" profile will not work.
-
[1]: ../../general/accessing-the-clusters/graphical-user-interface/vnc.md#gui-applications-on-compute-nodes-over-vnc
[2]: #running-parallel-matlab-using-distributed-computing-toolbox---engine
[3]: ../isv_licenses.md
diff --git a/docs.it4i/software/numerical-languages/matlab_1314.md b/docs.it4i/software/numerical-languages/matlab_1314.md
deleted file mode 100644
index c0408ae57..000000000
--- a/docs.it4i/software/numerical-languages/matlab_1314.md
+++ /dev/null
@@ -1,211 +0,0 @@
-# MATLAB 2013-2014
-
-## Introduction
-
-!!! note
- This document relates to the old versions R2013 and R2014. For MATLAB 2015 use [this documentation instead][1].
-
-MATLAB is available in the latest stable version. There are always two variants of the release:
-
-* Non commercial or so called EDU variant, which can be used for common research and educational purposes.
-* Commercial or so called COM variant, which can used also for commercial activities. The licenses for commercial variant are much more expensive, so usually the commercial variant has only subset of features compared to the EDU available.
-
-To load the latest version of MATLAB load the module, use:
-
-```console
-$ ml matlab
-```
-
-The EDU variant is marked as default. If you need other version or variant, load the particular version. To obtain the list of available versions, use:
-
-```console
-$ ml matlab
-```
-
-If you need to use the MATLAB GUI to prepare your MATLAB programs, you can use MATLAB directly on the login nodes. However, for all computations, use MATLAB on the compute nodes via the PBS Pro scheduler.
-
-If you require the MATLAB GUI, follow the general information about [running graphical applications][2].
-
-MATLAB GUI is quite slow using the X forwarding built in the PBS (`qsub -X`), so using X11 display redirection either via SSH or directly by `xauth` (see the [GUI Applications on Compute Nodes over VNC][3] section) is recommended.
-
-To run MATLAB with GUI, use:
-
-```console
-$ matlab
-```
-
-To run MATLAB in text mode, without the MATLAB Desktop GUI environment, use:
-
-```console
-$ matlab -nodesktop -nosplash
-```
-
-plots, images, etc. will be still available.
-
-## Running Parallel MATLAB Using Distributed Computing Toolbox / Engine
-
-A recommended parallel mode for running parallel MATLAB is the MPIEXEC mode. In this mode, the user allocates resources through PBS prior to starting MATLAB. Once resources are granted, the main MATLAB instance is started on the first compute node assigned to job by PBS and workers are started on all remaining nodes. The user can use both interactive and non-interactive PBS sessions. This mode guarantees that the data processing is not performed on login nodes, but all processing is on compute nodes.
-
-
-
-For performance reasons, MATLAB should use system MPI. On our clusters, the supported MPI implementation for MATLAB is Intel MPI. To switch to system MPI, the user has to override default MATLAB setting by creating a new configuration file in its home directory. The path and file name has to be the same as in the following listing:
-
-```matlab
-$ vim ~/matlab/mpiLibConf.m
-
-function [lib, extras] = mpiLibConf
-%MATLAB MPI Library overloading for Infiniband Networks
-
-mpich = '/opt/intel/impi/4.1.1.036/lib64/';
-
-disp('Using Intel MPI 4.1.1.036 over Infiniband')
-
-lib = strcat(mpich, 'libmpich.so');
-mpl = strcat(mpich, 'libmpl.so');
-opa = strcat(mpich, 'libopa.so');
-
-extras = {};
-```
-
-The system MPI library allows MATLAB to communicate through 40 Gbit/s InfiniBand QDR interconnect instead of a slower 1 Gbit Ethernet network.
-
-!!! note
- The path to MPI library in "mpiLibConf.m" has to match with version of the loaded Intel MPI module. In this example, version 4.1.1.036 of Intel MPI is used by MATLAB and therefore the `impi/4.1.1.036` module has to be loaded prior to starting MATLAB.
-
-### Parallel MATLAB Interactive Session
-
-Once this file is in place, the user can request resources from PBS. The following example shows how to start an interactive session with support for MATLAB GUI. For more information about GUI-based applications, see:
-
-```console
-$ xhost +
-$ qsub -I -v DISPLAY=$(uname -n):$(echo $DISPLAY | cut -d ':' -f 2) -A NONE-0-0 -q qexp -l select=2:ncpus=16:mpiprocs=16 -l walltime=00:30:00 -l license__matlab-edu__MATLAB=1
-```
-
-This qsub command example shows how to run MATLAB with 32 workers in the following configuration: 2 nodes (use 16 cores per node) and 16 workers = `mpiprocs` per node (`-l select=2:ncpus=16:mpiprocs=16`). If the user requires to run smaller number of workers per node then the `mpiprocs` parameter has to be changed.
-
-The second part of the command shows how to request all necessary licenses. In this case, 1 MATLAB-EDU license and 32 Distributed Computing Engines licenses.
-
-Once the access to compute nodes is granted by PBS, the user can load the following modules and start MATLAB:
-
-```console
-$ ml matlab/R2013a-EDU
-$ ml impi/4.1.1.036
-$ matlab &
-```
-
-### Parallel MATLAB Batch Job
-
-To run MATLAB in a batch mode, write a MATLAB script, then write a bash jobscript and execute it via the `qsub` command. By default, MATLAB will execute one MATLAB worker instance per allocated core.
-
-```bash
-#!/bin/bash
-#PBS -A PROJECT ID
-#PBS -q qprod
-#PBS -l select=2:ncpus=16:mpiprocs=16:ompthreads=1
-
-# change to shared scratch directory
-SCR=/scratch/$USER/$PBS_JOBID
-mkdir -p $SCR ; cd $SCR || exit
-
-# copy input file to scratch
-cp $PBS_O_WORKDIR/matlabcode.m .
-
-# load modules
-ml matlab/R2013a-EDU
-ml impi/4.1.1.036
-
-# execute the calculation
-matlab -nodisplay -r matlabcode > output.out
-
-# copy output file to home
-cp output.out $PBS_O_WORKDIR/.
-```
-
-This script may be submitted directly to the PBS workload manager via the `qsub` command. The inputs and MATLAB script are in the matlabcode.m file, outputs in the output.out file. Note the missing .m extension in the `matlab -r matlabcodefile` call, **the .m must not be included**. Note that the **shared /scratch must be used**. Further, it is **important to include the `quit`** statement at the end of the matlabcode.m script.
-
-Submit the jobscript using qsub:
-
-```console
-$ qsub ./jobscript
-```
-
-### Parallel MATLAB Program Example
-
-The last part of the configuration is done directly in the user's MATLAB script before Distributed Computing Toolbox is started.
-
-```matlab
-sched = findResource('scheduler', 'type', 'mpiexec');
-set(sched, 'MpiexecFileName', '/apps/intel/impi/4.1.1/bin/mpirun');
-set(sched, 'EnvironmentSetMethod', 'setenv');
-```
-
-This script creates a `sched` scheduler object of the type `mpiexec` that starts workers using the `mpirun` tool. To use a correct version of `mpirun`, the second line specifies the path to correct version of the system Intel MPI library.
-
-!!! note
- Every MATLAB script that needs to initialize/use `matlabpool` has to contain these three lines prior to calling the `matlabpool(sched, ...)` function.
-
-The last step is to start `matlabpool` with the `sched` object and a correct number of workers. In this case, `qsub` asked for the total number of 32 cores, therefore the number of workers is also set to `32`.
-
-```console
-matlabpool(sched,32);
-
-
-... parallel code ...
-
-
-matlabpool close
-```
-
-The complete example showing how to use Distributed Computing Toolbox is show here:
-
-```matlab
-sched = findResource('scheduler', 'type', 'mpiexec');
-set(sched, 'MpiexecFileName', '/apps/intel/impi/4.1.1/bin/mpirun')
-set(sched, 'EnvironmentSetMethod', 'setenv')
-set(sched, 'SubmitArguments', '')
-sched
-
-matlabpool(sched,32);
-
-n=2000;
-
-W = rand(n,n);
-W = distributed(W);
-x = (1:n)';
-x = distributed(x);
-spmd
-[~, name] = system('hostname')
-
- T = W*x; % Calculation performed on labs, in parallel.
- % T and W are both codistributed arrays here.
-end
-T;
-whos % T and W are both distributed arrays here.
-
-matlabpool close
-quit
-```
-
-You can copy and paste the example in a .m file and execute. Note that the `matlabpool` size should correspond to the **total number of cores** available on allocated nodes.
-
-### Non-Interactive Session and Licenses
-
-If you want to run batch jobs with MATLAB, be sure to request appropriate license features with the PBS Pro scheduler, at least the `-l license__matlab-edu__MATLAB=1` for EDU variant of MATLAB. For more information about how to check the license features states and how to request them with PBS Pro, [look here][4].
-
-In the case of non-interactive session, read the [following information][4] on how to modify the qsub command to test for available licenses prior getting the resource allocation.
-
-### MATLAB Distributed Computing Engines Start Up Time
-
-Starting MATLAB workers is an expensive process that requires certain amount of time. For more information, see the following table:
-
-| compute nodes | number of workers | start-up time[s] |
-| ------------- | ----------------- | ---------------- |
-| 16 | 256 | 1008 |
-| 8 | 128 | 534 |
-| 4 | 64 | 333 |
-| 2 | 32 | 210 |
-
-[1]: matlab.md
-[2]: ../../general/accessing-the-clusters/graphical-user-interface/x-window-system.md
-[3]: ../../general/accessing-the-clusters/graphical-user-interface/vnc.md#gui-applications-on-compute-nodes-over-vnc
-[4]: ../isv_licenses.md
diff --git a/docs.it4i/software/numerical-languages/octave.md b/docs.it4i/software/numerical-languages/octave.md
index 02a340bc9..32b1c4d4f 100644
--- a/docs.it4i/software/numerical-languages/octave.md
+++ b/docs.it4i/software/numerical-languages/octave.md
@@ -12,6 +12,8 @@ $ ml av octave
## Modules and Execution
+To load the latest version of Octave load the module:
+
```console
$ ml Octave
```
@@ -30,13 +32,15 @@ To run Octave in batch mode, write an Octave script, then write a bash jobscript
#!/bin/bash
# change to local scratch directory
-cd /lscratch/$PBS_JOBID || exit
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
# copy input file to scratch
cp $PBS_O_WORKDIR/octcode.m .
# load octave module
-ml octave
+ml Octave/6.3.0-intel-2020b-without-X11
# execute the calculation
octave -q --eval octcode > output.out
@@ -50,67 +54,16 @@ exit
This script may be submitted directly to the PBS workload manager via the `qsub` command. The inputs are in the octcode.m file, outputs in the output.out file. See the single node jobscript example in the [Job execution section][1].
-The Octave c compiler `mkoctfile` calls the GNU GCC 4.8.1 for compiling native C code. This is very useful for running native C subroutines in Octave environment.
+The Octave c compiler `mkoctfile` calls the GNU GCC 6.3.0 for compiling native C code. This is very useful for running native C subroutines in Octave environment.
```console
$ mkoctfile -v
+mkoctfile, version 6.3.0
```
Octave may use MPI for interprocess communication This functionality is currently not supported on the clusters. In case you require the Octave interface to MPI, contact [support][b].
-## Xeon Phi Support
-
-<!--- not tested -->
-Octave may take advantage of the Xeon Phi accelerators. This will only work on the [Intel Xeon Phi][2] [accelerated nodes][3].
-
-### Automatic Offload Support
-
-Octave can accelerate BLAS type operations (in particular the Matrix Matrix multiplications] on the Xeon Phi accelerator, via [Automatic Offload using the MKL library][2].
-
-Example
-
-```octave
-$ export OFFLOAD_REPORT=2
-$ export MKL_MIC_ENABLE=1
-$ ml octave
-$ octave -q
- octave:1> A=rand(10000); B=rand(10000);
- octave:2> tic; C=A*B; toc
- [MKL] [MIC --] [AO Function] DGEMM
- [MKL] [MIC --] [AO DGEMM Workdivision] 0.32 0.68
- [MKL] [MIC 00] [AO DGEMM CPU Time] 2.896003 seconds
- [MKL] [MIC 00] [AO DGEMM MIC Time] 1.967384 seconds
- [MKL] [MIC 00] [AO DGEMM CPU->MIC Data] 1347200000 bytes
- [MKL] [MIC 00] [AO DGEMM MIC->CPU Data] 2188800000 bytes
- Elapsed time is 2.93701 seconds.
-```
-
-In this example, the calculation was automatically divided among the CPU cores and the Xeon Phi MIC accelerator, reducing the total runtime from 6.3 secs down to 2.9 secs.
-
-### Native Support
-
-<!--- not tested -->
-A version of [native][2] Octave is compiled for Xeon Phi accelerators. Some limitations apply for this version:
-
-* Only command line support. GUI, graph plotting, etc. is not supported.
-* Command history in interactive mode is not supported.
-
-Octave is linked with parallel Intel MKL, so it is best suited for batch processing of tasks that utilize BLAS, LAPACK, and FFT operations. By default, the number of threads is set to 120, you can control this with the `OMP_NUM_THREADS` environment variable.
-
-!!! note
- Calculations that do not employ parallelism (either by using parallel MKL e.g. via matrix operations, `fork()` function, [parallel package][c], or other mechanism) will actually run slower than on host CPU.
-
-To use Octave on a node with Xeon Phi:
-
-```console
-$ ssh mic0 # login to the MIC card
-$ source /apps/tools/octave/3.8.2-mic/bin/octave-env.sh # set up environment variables
-$ octave -q /apps/tools/octave/3.8.2-mic/example/test0.m # run an example
-```
-
[1]: ../../general/job-submission-and-execution.md
-[2]: ../intel/intel-xeon-phi-salomon.md
-[3]: ../../salomon/compute-nodes.md
[a]: https://www.gnu.org/software/octave/
[b]: https://support.it4i.cz/rt/
diff --git a/docs.it4i/software/numerical-languages/opencoarrays.md b/docs.it4i/software/numerical-languages/opencoarrays.md
index 5ba877258..ab3a4107d 100644
--- a/docs.it4i/software/numerical-languages/opencoarrays.md
+++ b/docs.it4i/software/numerical-languages/opencoarrays.md
@@ -79,10 +79,10 @@ end program synchronization_test
## Compile and Run
-Currently, version 1.8.10 compiled with the OpenMPI 1.10.7 library is installed on the cluster. To load the `OpenCoarrays` module, type:
+Currently, version 2.9.2 compiled with the OpenMPI 4.0.5 library is installed on the cluster. To load the `OpenCoarrays` module, type:
```console
-$ ml OpenCoarrays/1.8.10-GCC-6.3.0-2.27
+$ ml OpenCoarrays/2.9.2-gompi-2020b
```
### Compile CAF Program
@@ -106,7 +106,7 @@ $ mpif90 hello_world.f90 -o hello_world.x -fcoarray=lib -lcaf_mpi
### Run CAF Program
-A CAF program can be run by invoking the `cafrun` wrapper or directly by the `mpiexec`:
+A CAF program can be run by invoking the `cafrun` wrapper or directly by the `mpirun`:
```console
$ cafrun -np 4 ./hello_world.x
@@ -115,17 +115,13 @@ $ cafrun -np 4 ./hello_world.x
Hello world from image 3 of 4
Hello world from image 4 of 4
-$ mpiexec -np 4 ./synchronization_test.x
+$ mpirun -np 4 ./synchronization_test.x
The random number is 242
The random number is 242
The random number is 242
The random number is 242
```
-`-np 4` is the number of images to run. The parameters of `cafrun` and `mpiexec` are the same.
-
-For more information about running CAF program, follow [Running OpenMPI - Salomon][1].
-
-[1]: ../mpi/running_openmpi.md
+`-np 4` is the number of images to run. The parameters of `cafrun` and `mpirun` are the same.
[a]: http://www.opencoarrays.org/
diff --git a/docs.it4i/software/numerical-languages/r.md b/docs.it4i/software/numerical-languages/r.md
index 10f79da41..9e862a47a 100644
--- a/docs.it4i/software/numerical-languages/r.md
+++ b/docs.it4i/software/numerical-languages/r.md
@@ -19,7 +19,6 @@ R version 3.1.1 is available on the cluster, along with GUI interface RStudio
| Application | Version | module |
| ----------- | ----------------- | ------------------- |
| **R** | R 3.1.1 | R/3.1.1-intel-2015b |
-| **RStudio** | RStudio 0.98.1103 | RStudio |
```console
$ ml R
@@ -48,7 +47,9 @@ Example jobscript:
#!/bin/bash
# change to local scratch directory
-cd /lscratch/$PBS_JOBID || exit
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
# copy input file to scratch
cp $PBS_O_WORKDIR/rscript.R .
@@ -370,18 +371,18 @@ An example jobscript for [static Rmpi][4] parallel R execution, running 1 proces
#PBS -l select=100:ncpus=24:mpiprocs=24:ompthreads=1
# change to scratch directory
-SCRDIR=/scratch/work/user/$USER/myjob
-cd $SCRDIR || exit
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
# copy input file to scratch
cp $PBS_O_WORKDIR/rscript.R .
# load R and openmpi module
-ml R
-ml OpenMPI
+ml R OpenMPI
# execute the calculation
-mpiexec -bycore -bind-to-core R --slave --no-save --no-restore -f rscript.R
+mpirun -bycore -bind-to-core R --slave --no-save --no-restore -f rscript.R
# copy output file to home
cp routput.out $PBS_O_WORKDIR/.
@@ -392,22 +393,10 @@ exit
For more information about jobscripts and MPI execution, refer to the [Job submission][1] and general [MPI][5] sections.
-## Xeon Phi Offload
-
-By leveraging MKL, R can accelerate certain computations, most notably linear algebra operations on the Xeon Phi accelerator by using Automated Offload. To use MKL Automated Offload, you need to first set this environment variable before R execution:
-
-```console
-$ export MKL_MIC_ENABLE=1
-```
-
-Read more about automatic offload [here][6].
-
[1]: ../../general/job-submission-and-execution.md
[2]: #interactive-execution
-[3]: ../mpi/running_openmpi.md
[4]: #static-rmpi
[5]: ../mpi/mpi.md
-[6]: ../intel/intel-xeon-phi-salomon.md
[a]: http://www.r-project.org/
[b]: http://cran.r-project.org/doc/manuals/r-release/R-lang.html
diff --git a/docs.it4i/software/numerical-libraries/fftw.md b/docs.it4i/software/numerical-libraries/fftw.md
index 23d33bdb0..4fb7fef9a 100644
--- a/docs.it4i/software/numerical-libraries/fftw.md
+++ b/docs.it4i/software/numerical-libraries/fftw.md
@@ -4,21 +4,18 @@ The discrete Fourier transform in one or more dimensions, MPI parallel
FFTW is a C subroutine library for computing the discrete Fourier transform in one or more dimensions, of arbitrary input size, and of both real and complex data (as well as of even/odd data, e.g. the discrete cosine/sine transforms or DCT/DST). The FFTW library allows for MPI parallel, in-place discrete Fourier transform, with data distributed over number of nodes.
-Two versions, **3.3.x** and **2.1.5** of FFTW are available, each compiled for **Intel MPI** and **OpenMPI** using **Intel** and **gnu** compilers. These are available via modules:
-
-| Version | Parallelization | module | linker options |
-| -------------- | --------------- | ------------------- | ----------------------------------- |
-| FFTW3 gcc3.3.3 | pthread, OpenMP | fftw3/3.3.3-gcc | -lfftw3, -lfftw3_threads-lfftw3_omp |
-| FFTW3 icc3.3.3 | pthread, OpenMP | fftw3 | -lfftw3, -lfftw3_threads-lfftw3_omp |
-| FFTW2 gcc2.1.5 | pthread | fftw2/2.1.5-gcc | -lfftw, -lfftw_threads |
-| FFTW2 icc2.1.5 | pthread | fftw2 | -lfftw, -lfftw_threads |
-| FFTW3 gcc3.3.3 | OpenMPI | fftw-mpi3/3.3.3-gcc | -lfftw3_mpi |
-| FFTW3 icc3.3.3 | Intel MPI | fftw3-mpi | -lfftw3_mpi |
-| FFTW2 gcc2.1.5 | OpenMPI | fftw2-mpi/2.1.5-gcc | -lfftw_mpi |
-| FFTW2 gcc2.1.5 | IntelMPI | fftw2-mpi/2.1.5-gcc | -lfftw_mpi |
+```console
+$ ml av FFTW
+
+---------------------------------------------------- /apps/modules/numlib -----------------------------------------------------
+ FFTW/3.3.7-gompi-2018a FFTW/3.3.8-gompi-2020a FFTW/3.3.8-gompic-2020b FFTW/3.3.8
+ FFTW/3.3.8-gompi-2020a-amd FFTW/3.3.8-gompi-2020b FFTW/3.3.8-iccifort-2020.4.304 FFTW/3.3.9-gompi-2021a (D)
+```
+
+To load the latest version of Octave load the module:
```console
-$ ml fftw3 **or** ml FFTW
+$ ml FFTW
```
The module sets up environment variables, required for linking and running FFTW enabled applications. Make sure that the choice of FFTW module is consistent with your choice of MPI library. Mixing MPI of different implementations may have unpredictable results.
@@ -63,14 +60,10 @@ The module sets up environment variables, required for linking and running FFTW
Load modules and compile:
```console
-$ ml intel
-$ ml fftw3-mpi
+$ ml intel/2020b 3.3.8-iccifort-2020.4.304
$ mpicc testfftw3mpi.c -o testfftw3mpi.x -Wl,-rpath=$LIBRARY_PATH -lfftw3_mpi
```
-Run the example as [Intel MPI program][1].
-
Read more on FFTW usage on the [FFTW website][a].
-[1]: ../mpi/running-mpich2.md
-[a]: http://www.fftw.org/fftw3_doc/
+[a]: http://www.fftw.org/fftw3_doc/
\ No newline at end of file
diff --git a/docs.it4i/software/numerical-libraries/gsl.md b/docs.it4i/software/numerical-libraries/gsl.md
index 2154200ad..07853b436 100644
--- a/docs.it4i/software/numerical-libraries/gsl.md
+++ b/docs.it4i/software/numerical-libraries/gsl.md
@@ -48,6 +48,9 @@ For the list of available gsl modules, use the command:
```console
$ ml av gsl
+---------------- /apps/modules/numlib -------------------
+ GSL/2.5-intel-2017c GSL/2.6-iccifort-2020.1.217 GSL/2.7-GCC-10.3.0 (D)
+ GSL/2.6-GCC-10.2.0 GSL/2.6-iccifort-2020.4.304
```
## Linking
@@ -57,17 +60,14 @@ Load an appropriate `gsl` module. Use the `-lgsl` switch to link your code again
### Compiling and Linking With Intel Compilers
```console
-$ ml intel
-$ ml gsl
+$ ml intel/2020b gsl/2.6-iccifort-2020.4.304
$ icc myprog.c -o myprog.x -Wl,-rpath=$LIBRARY_PATH -mkl -lgsl
```
### Compiling and Linking With GNU Compilers
```console
-$ ml gcc
-$ ml imkl **or** ml mkl
-$ ml gsl/1.16-gcc
+$ ml ml GCC/10.2.0 imkl/2020.4.304-iimpi-2020b GSL/2.6-iccifort-2020.4.304
$ gcc myprog.c -o myprog.x -Wl,-rpath=$LIBRARY_PATH -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lgsl
```
@@ -130,8 +130,7 @@ Following is an example of a discrete wavelet transform implemented by GSL:
Load modules and compile:
```console
-$ ml intel
-$ ml gsl
+$ ml intel/2020b gsl/GSL/2.6-iccifort-2020.4.304
$ icc dwt.c -o dwt.x -Wl,-rpath=$LIBRARY_PATH -mkl -lgsl
```
diff --git a/docs.it4i/software/numerical-libraries/hdf5.md b/docs.it4i/software/numerical-libraries/hdf5.md
index c11c15f42..4f199d1d3 100644
--- a/docs.it4i/software/numerical-libraries/hdf5.md
+++ b/docs.it4i/software/numerical-libraries/hdf5.md
@@ -9,7 +9,12 @@ Hierarchical Data Format library. Serial and MPI parallel version.
For the current list of installed versions, use:
```console
-$ ml av HDF
+$ ml av HDF5
+----------------------------------------------------- /apps/modules/data ------------------------------------------------------
+ HDF5/1.10.6-foss-2020b-parallel HDF5/1.10.6-intel-2020a HDF5/1.10.7-gompi-2021a
+ HDF5/1.10.6-iimpi-2020a HDF5/1.10.6-intel-2020b-parallel HDF5/1.10.7-gompic-2020b
+ HDF5/1.10.6-intel-2020a-parallel HDF5/1.10.7-gompi-2020b HDF5/1.10.7-iimpi-2020b (D)
+
```
To load the module, use the `ml` command.
@@ -67,15 +72,10 @@ The module sets up environment variables required for linking and running HDF5 e
Load modules and compile:
```console
-$ ml intel
-$ ml hdf5-parallel
+$ ml intel/2020b HDF5/1.10.6-intel-2020b-parallel
$ mpicc hdf5test.c -o hdf5test.x -Wl,-rpath=$LIBRARY_PATH $HDF5_INC $HDF5_SHLIB
```
-Run the example as [Intel MPI program][1].
-
For further information, see the [website][a].
-[1]: ../mpi/running-mpich2.md
-
[a]: http://www.hdfgroup.org/HDF5/
diff --git a/docs.it4i/software/numerical-libraries/intel-numerical-libraries.md b/docs.it4i/software/numerical-libraries/intel-numerical-libraries.md
index d4730f8c3..37f5bf743 100644
--- a/docs.it4i/software/numerical-libraries/intel-numerical-libraries.md
+++ b/docs.it4i/software/numerical-libraries/intel-numerical-libraries.md
@@ -7,9 +7,16 @@ Intel libraries for high performance in numerical computing.
Intel Math Kernel Library (Intel MKL) is a library of math kernel subroutines, extensively threaded and optimized for maximum performance. Intel MKL unites and provides these basic components: BLAS, LAPACK, ScaLapack, PARDISO, FFT, VML, VSL, Data fitting, Feast Eigensolver, and many more.
```console
-$ ml mkl **or** ml imkl
+$ ml av mkl
+------------------- /apps/modules/numlib -------------------
+ imkl/2017.4.239-iimpi-2017c imkl/2020.1.217-iimpi-2020a imkl/2021.2.0-iimpi-2021a (D)
+ imkl/2018.4.274-iimpi-2018a imkl/2020.4.304-iimpi-2020b (L) mkl/2020.4.304
+ imkl/2019.1.144-iimpi-2019a imkl/2020.4.304-iompi-2020b
```
+!!! info
+ `imkl` ... with intel toolchain. `mkl` with system toolchain.
+
For more information, see the [Intel MKL][1] section.
## Intel Integrated Performance Primitives
@@ -17,7 +24,9 @@ For more information, see the [Intel MKL][1] section.
Intel Integrated Performance Primitives version 7.1.1, compiled for AVX is available via the `ipp` module. IPP is a library of highly optimized algorithmic building blocks for media and data applications. This includes signal, image, and frame processing algorithms, such as FFT, FIR, Convolution, Optical Flow, Hough transform, Sum, MinMax, and many more.
```console
-$ ml ipp
+$ ml av ipp
+------------------- /apps/modules/perf -------------------
+ ipp/2020.3.304
```
For more information, see the [Intel IPP][2] section.
@@ -27,11 +36,28 @@ For more information, see the [Intel IPP][2] section.
Intel Threading Building Blocks (Intel TBB) is a library that supports scalable parallel programming using standard ISO C++ code. It does not require special languages or compilers. It is designed to promote scalable data parallel programming. Additionally, it fully supports nested parallelism, so you can build larger parallel components from smaller parallel components. To use the library, you specify tasks, not threads, and let the library map tasks onto threads in an efficient manner.
```console
-$ ml tbb
+$ ml av tbb
+------------------- /apps/modules/lib -------------------
+ tbb/2020.3-GCCcore-10.2.0
+
```
Read more at the [Intel TBB][3].
+## Python Hooks for Intel Math Kernel Library
+
+Python hooks for Intel(R) Math Kernel Library runtime control settings.
+
+```console
+$ ml av mkl-service
+------------------- /apps/modules/data -------------------
+ mkl-service/2.3.0-intel-2020b
+```
+
+Read more at the [hooks][a].
+
[1]: ../intel/intel-suite/intel-mkl.md
[2]: ../intel/intel-suite/intel-integrated-performance-primitives.md
[3]: ../intel/intel-suite/intel-tbb.md
+
+[a]: https://github.com/IntelPython/mkl-service
diff --git a/docs.it4i/software/numerical-libraries/magma-for-intel-xeon-phi.md b/docs.it4i/software/numerical-libraries/magma-for-intel-xeon-phi.md
deleted file mode 100644
index 4e4a550c4..000000000
--- a/docs.it4i/software/numerical-libraries/magma-for-intel-xeon-phi.md
+++ /dev/null
@@ -1,83 +0,0 @@
-# MAGMA for Intel Xeon Phi
-
-A next generation dense algebra library for heterogeneous systems with accelerators.
-
-## Compiling and Linking With MAGMA
-
-To compile and link code with the MAGMA library, load the following module:
-
-```console
-$ ml magma/1.3.0-mic
-```
-
-To make compilation more user-friendly, the module also sets these two environment variables:
-
-* `MAGMA_INC` - contains paths to the MAGMA header files (to be used for compilation step).
-* `MAGMA_LIBS` - contains paths to the MAGMA libraries (to be used for linking step).
-
-Compilation example:
-
-```console
-$ icc -mkl -O3 -DHAVE_MIC -DADD_ -Wall $MAGMA_INC -c testing_dgetrf_mic.cpp -o testing_dgetrf_mic.o
-$ icc -mkl -O3 -DHAVE_MIC -DADD_ -Wall -fPIC -Xlinker -zmuldefs -Wall -DNOCHANGE -DHOST testing_dgetrf_mic.o -o testing_dgetrf_mic $MAGMA_LIBS
-```
-
-### Running MAGMA Code
-
-MAGMA implementation for Intel MIC requires a MAGMA server running on accelerator prior to executing the user application. To start or stop the server, use the following scripts:
-
-To start MAGMA server use:
-
-```console
-$MAGMAROOT/start_magma_server
-```
-
-To stop the server use:
-
-```console
-$MAGMAROOT/stop_magma_server
-```
-
-For more information about how the MAGMA server is started, see the following script:
-
-```console
-$MAGMAROOT/launch_anselm_from_mic.sh
-```
-
-To test if the MAGMA server runs properly, we can run one of the examples that are part of the MAGMA installation:
-
-```console
-[user@cn204 ~]$ $MAGMAROOT/testing/testing_dgetrf_mic
-[user@cn204 ~]$ export OMP_NUM_THREADS=16
-[lriha@cn204 ~]$ $MAGMAROOT/testing/testing_dgetrf_mic
- Usage: /apps/libs/magma-mic/magmamic-1.3.0/testing/testing_dgetrf_mic [options] [-h|--help]
-
- M N CPU GFlop/s (sec) MAGMA GFlop/s (sec) ||PA-LU||/(||A||*N)
- =========================================================================
- 1088 1088 --- ( --- ) 13.93 ( 0.06) ---
- 2112 2112 --- ( --- ) 77.85 ( 0.08) ---
- 3136 3136 --- ( --- ) 183.21 ( 0.11) ---
- 4160 4160 --- ( --- ) 227.52 ( 0.21) ---
- 5184 5184 --- ( --- ) 258.61 ( 0.36) ---
- 6208 6208 --- ( --- ) 333.12 ( 0.48) ---
- 7232 7232 --- ( --- ) 416.52 ( 0.61) ---
- 8256 8256 --- ( --- ) 446.97 ( 0.84) ---
- 9280 9280 --- ( --- ) 461.15 ( 1.16) ---
- 10304 10304 --- ( --- ) 500.70 ( 1.46) ---
-```
-
-!!! hint
- MAGMA contains several benchmarks and examples in `$MAGMAROOT/testing/`.
-
-!!! note
- MAGMA relies on the performance of all CPU cores as well as on the performance of the accelerator. Therefore on Anselm, the number of CPU OpenMP threads has to be set to 16 with `export OMP_NUM_THREADS=16`.
-
-See more details at [MAGMA home page][a].
-
-## References
-
-[1] [MAGMA MIC: Linear Algebra Library for Intel Xeon Phi Coprocessors][1], Jack Dongarra et. al
-
-[1]: http://icl.utk.edu/projectsfiles/magma/pubs/24-MAGMA_MIC_03.pdf
-
-[a]: http://icl.cs.utk.edu/magma/
diff --git a/docs.it4i/software/numerical-libraries/petsc.md b/docs.it4i/software/numerical-libraries/petsc.md
index 9352e7b62..53af08d06 100644
--- a/docs.it4i/software/numerical-libraries/petsc.md
+++ b/docs.it4i/software/numerical-libraries/petsc.md
@@ -16,13 +16,12 @@ PETSc (Portable, Extensible Toolkit for Scientific Computation) is a suite of bu
## Modules
-You can start using PETSc by loading the `petsc` module. Module names follox the pattern `petsc/version-compiler-mpi-blas-variant` where `variant` is replaced by one of `{dbg, opt, threads-dbg, threads-opt}`, for example:
+For the current list of installed versions, use:
```console
-$ ml petsc/3.4.4-icc-impi-mkl-opt
-```
+$ ml av petsc
- The `opt` variant is compiled without debugging information (no `-g` option) and with aggressive compiler optimizations (`-O3 -xAVX`). This variant is suitable for performance measurements and production runs. In all other cases, use the debug (`dbg`) variant, because it contains debugging information, performs validations and self-checks, and provides a clear stack trace and message in case of an error. The other two variants `threads-dbg` and `threads-opt` are `dbg` and `opt`, respectively, built with [OpenMP and pthreads threading support][j].
+```
## External Libraries
--
GitLab
From 6f46b91e9127bd5cd6991f07703aeec3de5d7e14 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Wed, 22 Sep 2021 11:10:15 +0200
Subject: [PATCH 56/58] Update
docs.it4i/software/numerical-languages/octave.md, mkdocs.yml files Deleted
docs.it4i/software/numerical-libraries/trilinos.md
---
.../software/numerical-languages/octave.md | 6 +-
.../software/numerical-libraries/trilinos.md | 55 -------------------
mkdocs.yml | 1 -
3 files changed, 4 insertions(+), 58 deletions(-)
delete mode 100644 docs.it4i/software/numerical-libraries/trilinos.md
diff --git a/docs.it4i/software/numerical-languages/octave.md b/docs.it4i/software/numerical-languages/octave.md
index 32b1c4d4f..7e863cdf8 100644
--- a/docs.it4i/software/numerical-languages/octave.md
+++ b/docs.it4i/software/numerical-languages/octave.md
@@ -8,6 +8,8 @@ For a list of available modules, type:
```console
$ ml av octave
+------------------------------- /apps/modules/math -------------------------------
+ Octave/6.3.0-intel-2020b-without-X11
```
## Modules and Execution
@@ -18,7 +20,7 @@ To load the latest version of Octave load the module:
$ ml Octave
```
-Octave on clusters is linked to a highly optimized MKL mathematical library. This provides threaded parallelization to many Octave kernels, notably the linear algebra subroutines. Octave runs these heavy calculation kernels without any penalty. By default, Octave would parallelize to 24 threads on Salomon. You may control the threads by setting the `OMP_NUM_THREADS` environment variable.
+Octave on clusters is linked to a highly optimized MKL mathematical library. This provides threaded parallelization to many Octave kernels, notably the linear algebra subroutines. Octave runs these heavy calculation kernels without any penalty. By default, Octave would parallelize to 128 threads on Karolina. You may control the threads by setting the `OMP_NUM_THREADS` environment variable.
To run Octave interactively, log in with the `ssh -X` parameter for X11 forwarding. Run Octave:
@@ -26,7 +28,7 @@ To run Octave interactively, log in with the `ssh -X` parameter for X11 forwardi
$ octave
```
-To run Octave in batch mode, write an Octave script, then write a bash jobscript and execute via the `qsub` command. By default, Octave will use 24 threads on Salomon when running MKL kernels.
+To run Octave in batch mode, write an Octave script, then write a bash jobscript and execute via the `qsub` command. By default, Octave will use 128 threads on Karolina when running MKL kernels.
```bash
#!/bin/bash
diff --git a/docs.it4i/software/numerical-libraries/trilinos.md b/docs.it4i/software/numerical-libraries/trilinos.md
deleted file mode 100644
index 61ca1759a..000000000
--- a/docs.it4i/software/numerical-libraries/trilinos.md
+++ /dev/null
@@ -1,55 +0,0 @@
-# Trilinos
-
-Packages for large-scale scientific and engineering problems. Provides MPI and hybrid parallelization.
-
-## Introduction
-
-Trilinos is a collection of software packages for the numerical solution of large-scale scientific and engineering problems. It is based on C++ and features modern object-oriented design. Both serial as well as parallel computations based on MPI and hybrid parallelization are supported within Trilinos packages.
-
-## Installed Packages
-
-Current Trilinos installation contains (among others) the following main packages:
-
-* **Epetra** - core linear algebra package containing classes for manipulation with serial and distributed vectors, matrices, and graphs. Dense linear solvers are supported via interface to BLAS and LAPACK (Intel MKL). Its extension **EpetraExt** contains, for example, methods for matrix-matrix multiplication.
-* **Tpetra** - next-generation linear algebra package. Supports 64-bit indexing and arbitrary data type using C++ templates.
-* **Belos** - library of various iterative solvers (CG, block CG, GMRES, block GMRES, etc.).
-* **Amesos** - interface to direct sparse solvers.
-* **Anasazi** - framework for large-scale eigenvalue algorithms.
-* **IFPACK** - distributed algebraic preconditioner (includes, for example, incomplete LU factorization).
-* **Teuchos** - common tools packages. This package contains classes for memory management, output, performance monitoring, BLAS and LAPACK wrappers, etc.
-
-For the full list of Trilinos packages, descriptions of their capabilities, and user manuals, see [the webpage][a].
-
-## Installed Version
-
-For the list of available versions, use the command:
-
-```console
-$ ml av trilinos
-```
-
-## Compiling Against Trilinos
-
-First, load the appropriate module:
-
-```console
-$ ml trilinos
-```
-
-For the compilation of CMake-aware project, Trilinos provides the `FIND_PACKAGE( Trilinos )` capability, which makes it easy to build against Trilinos, including linking against the correct list of libraries.
-
-For compiling using simple Makefiles, Trilinos provides `Makefile.export` system, which allows users to include important Trilinos variables directly into their Makefiles. This can be done simply by inserting the following line into the Makefile:
-
-```cpp
-include Makefile.export.Trilinos
-```
-
-or
-
-```cpp
-include Makefile.export.<package>
-```
-
-if you are interested only in a specific Trilinos package. This will give you access to the variables such as `Trilinos_CXX_COMPILER`, `Trilinos_INCLUDE_DIRS`, `Trilinos_LIBRARY_DIRS`, etc.
-
-[a]: http://www.mcs.anl.gov/petsc/miscellaneous/external.html
diff --git a/mkdocs.yml b/mkdocs.yml
index 96fd623f9..aded0562e 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -190,7 +190,6 @@ nav:
- HDF5: software/numerical-libraries/hdf5.md
- Intel Numerical Libraries: software/numerical-libraries/intel-numerical-libraries.md
- PETSc: software/numerical-libraries/petsc.md
- - Trilinos: software/numerical-libraries/trilinos.md
- Languages:
- Java: software/lang/java.md
- C#: software/lang/csc.md
--
GitLab
From 40c12b02a5d953de2ecb4ef79a18d75ea75cbd1d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Wed, 22 Sep 2021 12:42:19 +0200
Subject: [PATCH 57/58] Update docs.it4i/software/tools/ansys/ansys-cfx.md,
docs.it4i/software/tools/ansys/ansys-fluent.md,
docs.it4i/software/tools/ansys/ansys-ls-dyna.md,
docs.it4i/software/tools/ansys/ansys-mechanical-apdl.md,
docs.it4i/software/tools/ansys/licensing.md, mkdocs.yml,
docs.it4i/software/tools/ansys/setting-license-preferences.md,
docs.it4i/software/tools/ansys/workbench.md,
docs.it4i/software/tools/easybuild-images.md,
docs.it4i/software/tools/spack.md,
docs.it4i/software/tools/virtualization.md,
docs.it4i/software/isv_licenses.md, docs.it4i/software/nvidia-cuda.md files
Deleted docs.it4i/software/tools/ansys/ls-dyna.md
---
docs.it4i/software/isv_licenses.md | 2 +-
docs.it4i/software/nvidia-cuda.md | 12 +-
docs.it4i/software/tools/ansys/ansys-cfx.md | 12 +-
.../software/tools/ansys/ansys-fluent.md | 16 +-
.../software/tools/ansys/ansys-ls-dyna.md | 14 +-
.../tools/ansys/ansys-mechanical-apdl.md | 14 +-
docs.it4i/software/tools/ansys/licensing.md | 6 +-
docs.it4i/software/tools/ansys/ls-dyna.md | 41 ---
.../ansys/setting-license-preferences.md | 2 +-
docs.it4i/software/tools/ansys/workbench.md | 11 +-
docs.it4i/software/tools/easybuild-images.md | 4 -
docs.it4i/software/tools/spack.md | 244 ++++++++++++------
docs.it4i/software/tools/virtualization.md | 5 +
mkdocs.yml | 1 -
14 files changed, 208 insertions(+), 176 deletions(-)
delete mode 100644 docs.it4i/software/tools/ansys/ls-dyna.md
diff --git a/docs.it4i/software/isv_licenses.md b/docs.it4i/software/isv_licenses.md
index 28241d491..2fda53070 100644
--- a/docs.it4i/software/isv_licenses.md
+++ b/docs.it4i/software/isv_licenses.md
@@ -79,4 +79,4 @@ The license is used and accounted only with the real usage of the product. So in
[1]: #Licence
[a]: https://extranet.it4i.cz/rsweb/barbora/licenses
-[b]: http://licelin.it4i.cz/list/
\ No newline at end of file
+[b]: http://licelin.it4i.cz/list/
diff --git a/docs.it4i/software/nvidia-cuda.md b/docs.it4i/software/nvidia-cuda.md
index a55c93487..a78303aac 100644
--- a/docs.it4i/software/nvidia-cuda.md
+++ b/docs.it4i/software/nvidia-cuda.md
@@ -20,13 +20,7 @@ The default programming model for GPU accelerators is NVIDIA CUDA. To set up the
$ ml CUDA
```
-If the user code is hybrid and uses both CUDA and MPI, the MPI environment has to be set up as well. One way to do this is to use the `PrgEnv-gnu` module, which sets up the correct combination of the GNU compiler and MPI library:
-
-```console
-$ ml PrgEnv-gnu
-```
-
-CUDA code can be compiled directly on login1 or login2 nodes. The user does not have to use compute nodes with GPU accelerators for compilation. To compile CUDA source code, use the NVCC compiler:
+CUDA code can be compiled directly on login nodes. The user does not have to use compute nodes with GPU accelerators for compilation. To compile CUDA source code, use the NVCC compiler:
```console
$ nvcc --version
@@ -37,7 +31,7 @@ The CUDA Toolkit comes with a large number of examples, which can be a helpful r
```console
$ cd ~
$ mkdir cuda-samples
-$ cp -R /apps/nvidia/cuda/6.5.14/samples/* ~/cuda-samples/
+$ cp -R /apps/nvidia/cuda/VERSION_CUDA/samples/* ~/cuda-samples/
```
To compile examples, change directory to the particular example (here the example used is deviceQuery) and run `make` to start the compilation;
@@ -51,7 +45,7 @@ To run the code, the user can use a PBS interactive session to get access to a n
```console
$ qsub -I -q qnvidia -A OPEN-0-0
-$ ml cuda
+$ ml CUDA
$ ~/cuda-samples/1_Utilities/deviceQuery/deviceQuery
```
diff --git a/docs.it4i/software/tools/ansys/ansys-cfx.md b/docs.it4i/software/tools/ansys/ansys-cfx.md
index c73546daf..b1411b0a9 100644
--- a/docs.it4i/software/tools/ansys/ansys-cfx.md
+++ b/docs.it4i/software/tools/ansys/ansys-cfx.md
@@ -6,18 +6,16 @@ To run ANSYS CFX in batch mode, you can utilize/modify the default cfx.pbs scrip
```bash
#!/bin/bash
-#PBS -l select=5:ncpus=24:mpiprocs=24
+#PBS -l select=5:ncpus=128:mpiprocs=128
#PBS -q qprod
#PBS -N ANSYS-test
#PBS -A XX-YY-ZZ
-#! Mail to user when job terminate or abort
-#PBS -m ae
-
#!change the working directory (default is home directory)
#cd <working directory> (working directory must exists)
-WORK_DIR="/scratch/$USER/work"
-cd $WORK_DIR
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
echo Running on host `hostname`
echo Time is `date`
@@ -25,7 +23,7 @@ echo Directory is `pwd`
echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
-ml ANSYS/19.1-intel-2017c
+ml ANSYS/21.1-intel-2018a
#### Set number of processors per host listing
#### (set to 1 as $PBS_NODEFILE lists each node twice if :ppn=2)
diff --git a/docs.it4i/software/tools/ansys/ansys-fluent.md b/docs.it4i/software/tools/ansys/ansys-fluent.md
index 0d0b20562..378f748b4 100644
--- a/docs.it4i/software/tools/ansys/ansys-fluent.md
+++ b/docs.it4i/software/tools/ansys/ansys-fluent.md
@@ -9,18 +9,16 @@ To run ANSYS Fluent in a batch mode, you can utilize/modify the default fluent.p
```bash
#!/bin/bash
#PBS -S /bin/bash
-#PBS -l select=5:ncpus=24:mpiprocs=24
+#PBS -l select=5:ncpus=128:mpiprocs=128
#PBS -q qprod
#PBS -N ANSYS-test
#PBS -A XX-YY-ZZ
-#! Mail to user when job terminate or abort
-#PBS -m ae
-
#!change the working directory (default is home directory)
#cd <working directory> (working directory must exists)
-WORK_DIR="/scratch/$USER/work"
-cd $WORK_DIR
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
echo Running on host `hostname`
echo Time is `date`
@@ -29,7 +27,7 @@ echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
#### Load ansys module so that we find the cfx5solve command
-ml ANSYS/19.1-intel-2017c
+ml ANSYS/21.1-intel-2018a
# Use following line to specify MPI for message-passing instead
NCORES=`wc -l $PBS_NODEFILE |awk '{print $1}'`
@@ -91,7 +89,7 @@ To run ANSYS Fluent in batch mode with the user's config file, you can utilize/m
```bash
#!/bin/sh
-#PBS -l nodes=2:ppn=4
+#PBS -l nodes=2:mpiprocs=4:ncpus=128
#PBS -1 qprod
#PBS -N $USE-Fluent-Project
#PBS -A XX-YY-ZZ
@@ -135,7 +133,7 @@ To run ANSYS Fluent in batch mode with the user's config file, you can utilize/m
Fluent arguments: $fluent_args"
#run the solver
- /ansys_inc/v145/fluent/bin/fluent $fluent_args > $outfile
+ fluent $fluent_args > $outfile
```
It runs the jobs out of the directory from which they are submitted (PBS_O_WORKDIR).
diff --git a/docs.it4i/software/tools/ansys/ansys-ls-dyna.md b/docs.it4i/software/tools/ansys/ansys-ls-dyna.md
index 761ec076b..0c68fd650 100644
--- a/docs.it4i/software/tools/ansys/ansys-ls-dyna.md
+++ b/docs.it4i/software/tools/ansys/ansys-ls-dyna.md
@@ -6,18 +6,16 @@ To run ANSYS LS-DYNA in batch mode, you can utilize/modify the default ansysdyna
```bash
#!/bin/bash
-#PBS -l select=5:ncpus=24:mpiprocs=24
+#PBS -l select=5:ncpus=128:mpiprocs=128
#PBS -q qprod
#PBS -N ANSYS-test
#PBS -A XX-YY-ZZ
-#! Mail to user when job terminate or abort
-#PBS -m ae
-
#!change the working directory (default is home directory)
#cd <working directory>
-WORK_DIR="/scratch/$USER/work"
-cd $WORK_DIR
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
echo Running on host `hostname`
echo Time is `date`
@@ -30,7 +28,7 @@ NPROCS=`wc -l < $PBS_NODEFILE`
echo This job has allocated $NPROCS nodes
-ml ANSYS/19.1-intel-2017c
+ml ANSYS/21.1-intel-2018a
#### Set number of processors per host listing
#### (set to 1 as $PBS_NODEFILE lists each node twice if :ppn=2)
@@ -47,7 +45,7 @@ done
echo Machines: $hl
-ansys191 -dis -lsdynampp i=input.k -machines $hl
+ansys211 -dis -lsdynampp i=input.k -machines $hl
```
The header of the PBS file (above) is common and the description can be found on [this site][1]. [SVS FEM][b] recommends to utilize sources by keywords: nodes, ppn. These keywords allows addressing directly the number of nodes (computers) and cores (ppn) utilized in the job. In addition, the rest of the code assumes such structure of allocated resources.
diff --git a/docs.it4i/software/tools/ansys/ansys-mechanical-apdl.md b/docs.it4i/software/tools/ansys/ansys-mechanical-apdl.md
index 1b0e647de..d027f175f 100644
--- a/docs.it4i/software/tools/ansys/ansys-mechanical-apdl.md
+++ b/docs.it4i/software/tools/ansys/ansys-mechanical-apdl.md
@@ -6,18 +6,16 @@ To run ANSYS MAPDL in batch mode you can utilize/modify the default mapdl.pbs sc
```bash
#!/bin/bash
-#PBS -l select=5:ncpus=24:mpiprocs=24
+#PBS -l select=5:ncpus=128:mpiprocs=128
#PBS -q qprod
#PBS -N ANSYS-test
#PBS -A XX-YY-ZZ
-#! Mail to user when job terminate or abort
-#PBS -m ae
-
#!change the working directory (default is home directory)
#cd <working directory> (working directory must exists)
-WORK_DIR="/scratch/$USER/work"
-cd $WORK_DIR
+DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+mkdir -p "$DIR"
+cd "$DIR" || exit
echo Running on host `hostname`
echo Time is `date`
@@ -25,7 +23,7 @@ echo Directory is `pwd`
echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
-ml ANSYS/19.1-intel-2017c
+ml ANSYS/21.1-intel-2018a
#### Set number of processors per host listing
#### (set to 1 as $PBS_NODEFILE lists each node twice if :ppn=2)
@@ -45,7 +43,7 @@ echo Machines: $hl
#-i input.dat includes the input of analysis in APDL format
#-o file.out is output file from ansys where all text outputs will be redirected
#-p the name of license feature (aa_r=ANSYS Academic Research, ane3fl=Multiphysics(commercial), aa_r_dy=Academic AUTODYN)
-ansys191 -b -dis -p aa_r -i input.dat -o file.out -machines $hl -dir $WORK_DIR
+ansys211 -b -dis -p aa_r -i input.dat -o file.out -machines $hl -dir $WORK_DIR
```
The header of the PBS file (above) is common and the description can be found on [this site][1]. [SVS FEM][b] recommends utilizing sources by keywords: nodes, ppn. These keywords allow addressing directly the number of nodes (computers) and cores (ppn) utilized in the job. In, addition the rest of the code assumes such structure of allocated resources.
diff --git a/docs.it4i/software/tools/ansys/licensing.md b/docs.it4i/software/tools/ansys/licensing.md
index 7e2c1dc3e..30b0bf4b9 100644
--- a/docs.it4i/software/tools/ansys/licensing.md
+++ b/docs.it4i/software/tools/ansys/licensing.md
@@ -22,15 +22,13 @@ lic-ansys.vsb.cz / 1055 (2325)
## Available Versions
-* 19.1
-* 19.3
+* 21.1
``` console
$ ml av ANSYS
---------------- /apps/modules/tools -----------------------
- ANSYS/19.1-intel-2017c (C6) ANSYS/19.3-intel-2017c (D)
+ ANSYS/21.1-intel-2018a (D)
Where:
- C6: Old CentOS6 module
D: Default Module
```
diff --git a/docs.it4i/software/tools/ansys/ls-dyna.md b/docs.it4i/software/tools/ansys/ls-dyna.md
deleted file mode 100644
index 5ed660201..000000000
--- a/docs.it4i/software/tools/ansys/ls-dyna.md
+++ /dev/null
@@ -1,41 +0,0 @@
-# LS-DYNA
-
-[LS-DYNA][a] is a multi-purpose, explicit and implicit finite element program used to analyze the nonlinear dynamic response of structures. Its fully automated contact analysis capability, a wide range of constitutive models to simulate a whole range of engineering materials (steels, composites, foams, concrete, etc.), error-checking features, and the high scalability have enabled users worldwide to solve successfully many complex problems. Additionally LS-DYNA is extensively used to simulate impacts on structures from drop tests, underwater shock, explosions or high-velocity impacts. Explosive forming, process engineering, accident reconstruction, vehicle dynamics, thermal brake disc analysis, or nuclear safety are further areas in the broad range of possible applications. In leading-edge research, LS-DYNA is used to investigate the behavior of materials like composites, ceramics, concrete, or wood. Moreover, it is used in biomechanics, human modeling, molecular structures, casting, forging, or virtual testing.
-
-!!! Info
- We provide **1 commercial license of LS-DYNA without HPC** support, now.
-
-To run LS-DYNA in batch mode, you can utilize/modify the default lsdyna.pbs script and execute it via the qsub command:
-
-```bash
-#!/bin/bash
-#PBS -l select=5:ncpus=16:mpiprocs=16
-#PBS -q qprod
-#PBS -N ANSYS-test
-#PBS -A XX-YY-ZZ
-
-#! Mail to user when job terminate or abort
-#PBS -m ae
-
-#!change the working directory (default is home directory)
-#cd <working directory> (working directory must exists)
-WORK_DIR="/scratch/$USER/work"
-cd $WORK_DIR
-
-echo Running on host `hostname`
-echo Time is `date`
-echo Directory is `pwd`
-
-ml lsdyna
-
-/apps/engineering/lsdyna/lsdyna700s i=input.k
-```
-
-The header of the PBS file (above) is common and the description can be found on [this site][1]. [SVS FEM][b] recommends utilizing sources by keywords: nodes, ppn. These keywords allow addressing directly the number of nodes (computers) and cores (ppn) utilized in the job. In addition, the rest of the code assumes such structure of allocated resources.
-
-A working directory has to be created before sending the PBS job into the queue. The input file should be in the working directory or a full path to input file has to be specified. The input file has to be defined by a common LS-DYNA **.k** file which is attached to the LS-DYNA solver via the `i=` parameter.
-
-[1]: ../../../general/job-submission-and-execution.md
-
-[a]: http://www.lstc.com/
-[b]: http://www.svsfem.cz
diff --git a/docs.it4i/software/tools/ansys/setting-license-preferences.md b/docs.it4i/software/tools/ansys/setting-license-preferences.md
index adafabb19..3c13b571f 100644
--- a/docs.it4i/software/tools/ansys/setting-license-preferences.md
+++ b/docs.it4i/software/tools/ansys/setting-license-preferences.md
@@ -1,6 +1,6 @@
# Setting License Preferences
-Some ANSYS tools allow you to explicitly specify usage of academic or commercial licenses in the command line (e.g. ansys161 -p aa_r to select the Academic Research license). However, we have observed that not all tools obey this option and choose the commercial license.
+Some ANSYS tools allow you to explicitly specify usage of academic or commercial licenses in the command line (e.g. ansys211 -p aa_r to select the Academic Research license). However, we have observed that not all tools obey this option and choose the commercial license.
Thus you need to configure preferred license order with ANSLIC_ADMIN. Follow these steps and move the Academic Research license to the top or bottom of the list accordingly.
diff --git a/docs.it4i/software/tools/ansys/workbench.md b/docs.it4i/software/tools/ansys/workbench.md
index b62a4c1d2..e7ef53d93 100644
--- a/docs.it4i/software/tools/ansys/workbench.md
+++ b/docs.it4i/software/tools/ansys/workbench.md
@@ -18,17 +18,16 @@ Now, save the project and close Workbench. We will use this script to launch the
```bash
#!/bin/bash
- #PBS -l select=2:ncpus=24
+ #PBS -l select=2:ncpus=128
#PBS -q qprod
#PBS -N test9_mpi_2
#PBS -A OPEN-0-0
- # Mail to user when job terminate or abort
- #PBS -m a
# change the working directory
- WORK_DIR="$PBS_O_WORKDIR"
- cd $WORK_DIR
+ DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
+ mkdir -p "$DIR"
+ cd "$DIR" || exit
echo Running on host `hostname`
echo Time is `date`
@@ -36,7 +35,7 @@ Now, save the project and close Workbench. We will use this script to launch the
echo This jobs runs on the following nodes:
echo `cat $PBS_NODEFILE`
- ml ANSYS
+ ml ANSYS/21.1-intel-2018a
#### Set number of processors per host listing
procs_per_host=24
diff --git a/docs.it4i/software/tools/easybuild-images.md b/docs.it4i/software/tools/easybuild-images.md
index cb86b95bf..1efc6d857 100644
--- a/docs.it4i/software/tools/easybuild-images.md
+++ b/docs.it4i/software/tools/easybuild-images.md
@@ -2,10 +2,6 @@
EasyBuild has support for generating container recipes that will use EasyBuild to build and install a specified software stack. In addition, EasyBuild can (optionally) leverage the build tool provided by the container software of choice to create container images.
-!!! info
- The features documented here have been available since **EasyBuild v3.6.0** but are still experimental, which implies they are subject to change in upcoming versions of EasyBuild.
- To use the features, enable the `--experimental` configuration option.
-
## Generating Container Recipes
To generate container recipes, use `eb --containerize`, or `eb -C` for short.
diff --git a/docs.it4i/software/tools/spack.md b/docs.it4i/software/tools/spack.md
index 8084dbe95..b318202b6 100644
--- a/docs.it4i/software/tools/spack.md
+++ b/docs.it4i/software/tools/spack.md
@@ -11,8 +11,7 @@ For more information, see Spack's [documentation][a].
$ ml av Spack
---------------------- /apps/modules/devel ------------------------------
- Spack/default
-
+ Spack/0.16.2 (D)
```
!!! note
@@ -27,8 +26,8 @@ $ ml Spack
== Settings for first use
Couldn't import dot_parser, loading of dot files will not be possible.
== temporary log file in case of crash /tmp/eb-wLh1RT/easybuild-54vEn3.log
-== processing EasyBuild easyconfig /apps/easybuild/easyconfigs-it4i/s/Spack/Spack-0.10.0.eb
-== building and installing Spack/0.10.0...
+== processing EasyBuild easyconfig /apps/easybuild/easyconfigs-master/easybuild/easyconfigs/s/Spack/Spack-0.16.2.eb
+== building and installing Spack/0.16.2...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
@@ -46,29 +45,27 @@ Couldn't import dot_parser, loading of dot files will not be possible.
== permissions...
== packaging...
== COMPLETED: Installation ended successfully
-== Results of the build can be found in the log file(s) ~/.local/easybuild/software/Spack/0.10.0/easybuild/easybuild-Spack-0.10.0-20170707.122650.log
+== Results of the build can be found in the log file(s) /home/user/.local/easybuild/software/Spack/0.16.2/easybuild/easybuild-Spack-0.16.2-20210922.123022.log
== Build succeeded for 1 out of 1
== Temporary log file(s) /tmp/eb-wLh1RT/easybuild-54vEn3.log* have been removed.
== Temporary directory /tmp/eb-wLh1RT has been removed.
== Create folder ~/Spack
The following have been reloaded with a version change:
- 1) Spack/default => Spack/0.10.0
+ 1) Spack/default => Spack/0.16.2
$ spack --version
-0.10.0
+0.16.2
```
## Usage Module Spack/Default
```console
$ ml Spack
+$ ml
-The following have been reloaded with a version change:
- 1) Spack/default => Spack/0.10.0
-
-$ spack --version
-0.10.0
+Currently Loaded Modules:
+ 1) Spack/0.16.2
```
## Build Software Package
@@ -107,9 +104,22 @@ To see more available versions of a package, run `spack versions`.
```console
$ spack versions git
==> Safe versions (already checksummed):
- 2.11.0 2.9.3 2.9.2 2.9.1 2.9.0 2.8.4 2.8.3 2.8.2 2.8.1 2.8.0 2.7.3 2.7.1
+ 2.29.0 2.27.0 2.25.0 2.20.1 2.19.1 2.17.1 2.15.1 2.13.0 2.12.1 2.11.1 2.9.3 2.9.1 2.8.4 2.8.2 2.8.0 2.7.1
+ 2.28.0 2.26.0 2.21.0 2.19.2 2.18.0 2.17.0 2.14.1 2.12.2 2.12.0 2.11.0 2.9.2 2.9.0 2.8.3 2.8.1 2.7.3
==> Remote versions (not yet checksummed):
- Found no versions for git
+ 2.33.0 2.26.2 2.23.3 2.21.1 2.18.3 2.16.1 2.13.6 2.10.4 2.7.0 2.5.2 2.4.2 2.3.0 2.0.2 1.8.5.2 1.8.3.1
+ 2.32.0 2.26.1 2.23.2 2.20.5 2.18.2 2.16.0 2.13.5 2.10.3 2.6.7 2.5.1 2.4.1 2.2.3 2.0.1 1.8.5.1 1.8.3
+ 2.31.1 2.25.5 2.23.1 2.20.4 2.18.1 2.15.4 2.13.4 2.10.2 2.6.6 2.5.0 2.4.0 2.2.2 2.0.0 1.8.5 1.8.2.3
+ 2.31.0 2.25.4 2.23.0 2.20.3 2.17.6 2.15.3 2.13.3 2.10.1 2.6.5 2.4.12 2.3.10 2.2.1 1.9.5 1.8.4.5 0.7
+ 2.30.2 2.25.3 2.22.5 2.20.2 2.17.5 2.15.2 2.13.2 2.10.0 2.6.4 2.4.11 2.3.9 2.2.0 1.9.4 1.8.4.4 0.6
+ 2.30.1 2.25.2 2.22.4 2.20.0 2.17.4 2.15.0 2.13.1 2.9.5 2.6.3 2.4.10 2.3.8 2.1.4 1.9.3 1.8.4.3 0.5
+ 2.30.0 2.25.1 2.22.3 2.19.6 2.17.3 2.14.6 2.12.5 2.9.4 2.6.2 2.4.9 2.3.7 2.1.3 1.9.2 1.8.4.2 0.04
+ 2.29.3 2.24.4 2.22.2 2.19.5 2.17.2 2.14.5 2.12.4 2.8.6 2.6.1 2.4.8 2.3.6 2.1.2 1.9.1 1.8.4.1 0.03
+ 2.29.2 2.24.3 2.22.1 2.19.4 2.16.6 2.14.4 2.12.3 2.8.5 2.6.0 2.4.7 2.3.5 2.1.1 1.9.0 1.8.4.rc0 0.02
+ 2.29.1 2.24.2 2.22.0 2.19.3 2.16.5 2.14.3 2.11.4 2.7.6 2.5.6 2.4.6 2.3.4 2.1.0 1.8.5.6 1.8.4 0.01
+ 2.28.1 2.24.1 2.21.4 2.19.0 2.16.4 2.14.2 2.11.3 2.7.5 2.5.5 2.4.5 2.3.3 2.0.5 1.8.5.5 1.8.3.4
+ 2.27.1 2.24.0 2.21.3 2.18.5 2.16.3 2.14.0 2.11.2 2.7.4 2.5.4 2.4.4 2.3.2 2.0.4 1.8.5.4 1.8.3.3
+ 2.26.3 2.23.4 2.21.2 2.18.4 2.16.2 2.13.7 2.10.5 2.7.2 2.5.3 2.4.3 2.3.1 2.0.3 1.8.5.3 1.8.3.2
```
## Graph for Software Package
@@ -118,6 +128,7 @@ Spack provides the `spack graph` command to display the dependency graph. By def
```console
$ spack graph git
+==> Warning: gcc@4.8.5 cannot build optimized binaries for "zen2". Using best target possible: "x86_64"
o git
|\
| |\
@@ -127,45 +138,82 @@ o git
| | | | | |\
| | | | | | |\
| | | | | | | |\
-| | | | | | | o | curl
-| |_|_|_|_|_|/| |
-|/| | | |_|_|/ /
-| | | |/| | | |
-| | | o | | | | openssl
-| |_|/ / / / /
-|/| | | | | |
-| | | | o | | gettext
-| | | | |\ \ \
-| | | | | |\ \ \
-| | | | | | |\ \ \
-| | | | | | | |\ \ \
-| | | | | | | o | | | libxml2
-| |_|_|_|_|_|/| | | |
-|/| | | | |_|/| | | |
+| | | | | | | | |\
+| | | | | | | | | |\
+| | | | | | | | | | |\
+| | | | | | | | | | | |\
+| | | | | | | | | | | | |\
+| | | | o | | | | | | | | | openssh
+| |_|_|/| | | | | | | | | |
+|/| | |/| | | | | | | | | |
+| | | | |\ \ \ \ \ \ \ \ \ \
+| | | | | | | | | | | | o | | curl
+| |_|_|_|_|_|_|_|_|_|_|/| | |
+|/| | | |_|_|_|_|_|_|_|/| | |
+| | | |/| | | | | |_|_|/ / /
+| | | | | | | | |/| | | | |
+| | | o | | | | | | | | | | openssl
+| |_|/| | | | | | | | | | |
+|/| |/ / / / / / / / / / /
+| |/| | | | | | | | | | |
+| | | | | | | | | o | | | gettext
+| | | | |_|_|_|_|/| | | |
+| | | |/| | | | |/| | | |
+| | | | | | | | | |\ \ \ \
+| | | | | | | | | | |\ \ \ \
+| | | | | | | | | | | |\ \ \ \
+| | | | | | | | | | | o | | | | libxml2
+| |_|_|_|_|_|_|_|_|_|/| | | | |
+|/| | | | | | | | |_|/| | | | |
+| | | | | | | | |/| |/| | | | |
+| | | | | | | | | |/| | | | | |
+o | | | | | | | | | | | | | | | zlib
+ / / / / / / / / / / / / / / /
+| | | | | | | | o | | | | | | xz
+| | | | | | | | / / / / / /
+| | | | | | | | o | | | | | tar
+| | | | | | | |/ / / / / /
+| | | | | | | | | | | o | automake
+| |_|_|_|_|_|_|_|_|_|/| |
+|/| | | | | | | | | | | |
+| | | | | | | | | | | |/
+| | | | | | | | | | | o autoconf
+| |_|_|_|_|_|_|_|_|_|/|
+|/| | | | |_|_|_|_|_|/
| | | | |/| | | | | |
-o | | | | | | | | | | zlib
- / / / / / / / / / /
-| | | o | | | | | | xz
-| | | / / / / / /
-| | | o | | | | | tar
-| | | / / / / /
-| | | | o | | | pkg-config
-| | | | / / /
-o | | | | | | perl
- / / / / / /
-o | | | | | pcre
- / / / / /
-| o | | | ncurses
+o | | | | | | | | | | perl
+|\ \ \ \ \ \ \ \ \ \ \
+o | | | | | | | | | | | gdbm
+o | | | | | | | | | | | readline
+| |_|/ / / / / / / / /
+|/| | | | | | | | | |
+| | | o | | | | | | | libedit
+| |_|/ / / / / / / /
+|/| | | | | | | | |
+o | | | | | | | | | ncurses
+| |_|_|_|_|_|/ / /
+|/| | | | | | | |
+o | | | | | | | | pkgconf
+ / / / / / / / /
+| o | | | | | | pcre2
+| / / / / / /
+| | o | | | | libtool
+| |/ / / / /
+| o | | | | m4
+| | o | | | libidn2
+| | o | | | libunistring
+| | |/ / /
+| o | | | libsigsegv
| / / /
-| | | o autoconf
-| | | o m4
-| | | o libsigsegv
-| | |
-o | | libiconv
- / /
+| | o | bzip2
+| | o | diffutils
+| |/ /
+| o | libiconv
+| /
| o expat
+| o libbsd
|
-o bzip2
+o berkeley-db
```
### Information for Software Package
@@ -174,45 +222,74 @@ To get more information on a particular package from `spack list`, use `spack in
```console
$ spack info git
-Package: git
-Homepage: http://git-scm.com
+AutotoolsPackage: git
+
+Description:
+ Git is a free and open source distributed version control system
+ designed to handle everything from small to very large projects with
+ speed and efficiency.
+
+Homepage: http://git-scm.com
+
+Tags:
+ None
+
+Preferred version:
+ 2.29.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.29.0.tar.gz
Safe versions:
- 2.11.0 https://github.com/git/git/tarball/v2.11.0
- 2.9.3 https://github.com/git/git/tarball/v2.9.3
- 2.9.2 https://github.com/git/git/tarball/v2.9.2
- 2.9.1 https://github.com/git/git/tarball/v2.9.1
- 2.9.0 https://github.com/git/git/tarball/v2.9.0
- 2.8.4 https://github.com/git/git/tarball/v2.8.4
- 2.8.3 https://github.com/git/git/tarball/v2.8.3
- 2.8.2 https://github.com/git/git/tarball/v2.8.2
- 2.8.1 https://github.com/git/git/tarball/v2.8.1
- 2.8.0 https://github.com/git/git/tarball/v2.8.0
- 2.7.3 https://github.com/git/git/tarball/v2.7.3
- 2.7.1 https://github.com/git/git/tarball/v2.7.1
+ 2.29.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.29.0.tar.gz
+ 2.28.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.28.0.tar.gz
+ 2.27.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.27.0.tar.gz
+ 2.26.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.26.0.tar.gz
+ 2.25.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.25.0.tar.gz
+ 2.21.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.21.0.tar.gz
+ 2.20.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.20.1.tar.gz
+ 2.19.2 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.19.2.tar.gz
+ 2.19.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.19.1.tar.gz
+ 2.18.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.18.0.tar.gz
+ 2.17.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.17.1.tar.gz
+ 2.17.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.17.0.tar.gz
+ 2.15.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.15.1.tar.gz
+ 2.14.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.14.1.tar.gz
+ 2.13.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.13.0.tar.gz
+ 2.12.2 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.12.2.tar.gz
+ 2.12.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.12.1.tar.gz
+ 2.12.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.12.0.tar.gz
+ 2.11.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.11.1.tar.gz
+ 2.11.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.11.0.tar.gz
+ 2.9.3 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.9.3.tar.gz
+ 2.9.2 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.9.2.tar.gz
+ 2.9.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.9.1.tar.gz
+ 2.9.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.9.0.tar.gz
+ 2.8.4 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.8.4.tar.gz
+ 2.8.3 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.8.3.tar.gz
+ 2.8.2 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.8.2.tar.gz
+ 2.8.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.8.1.tar.gz
+ 2.8.0 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.8.0.tar.gz
+ 2.7.3 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.7.3.tar.gz
+ 2.7.1 https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.7.1.tar.gz
Variants:
- None
+ Name [Default] Allowed values Description
+ ============== ============== ===========================================
+
+ tcltk [off] on, off Gitk: provide Tcl/Tk in the run environment
Installation Phases:
- install
+ autoreconf configure build install
Build Dependencies:
- autoconf curl expat gettext libiconv openssl pcre perl zlib
+ autoconf automake curl expat gettext iconv libidn2 libtool m4 openssl pcre pcre2 perl tk zlib
Link Dependencies:
- curl expat gettext libiconv openssl pcre perl zlib
+ curl expat gettext iconv libidn2 openssl pcre pcre2 perl tk zlib
Run Dependencies:
- None
+ openssh
Virtual Packages:
None
-
-Description:
- Git is a free and open source distributed version control system
- designed to handle everything from small to very large projects with
- speed and efficiency.
```
### Install Software Package
@@ -220,10 +297,23 @@ Description:
`spack install` will install any package shown by `spack list`. For example, to install the latest version of the `git` package, you might type `spack install git` for default version or `spack install git@version` to chose a particular one.
```console
-$ spack install git@2.11.0
-==> Installing git
-==> Installing pcre
-==> Fetching http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.bz2
+$ spack install git@2.29.0
+==> Warning: specifying a "dotkit" module root has no effect [support for "dotkit" has been dropped in v0.13.0]
+==> Warning: gcc@4.8.5 cannot build optimized binaries for "zen2". Using best target possible: "x86_64"
+==> Installing libsigsegv-2.12-lctnabj6w4bmnyxo7q6ct4wewke2bqin
+==> No binary for libsigsegv-2.12-lctnabj6w4bmnyxo7q6ct4wewke2bqin found: installing from source
+==> Fetching https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/_source-cache/archive/3a/3ae1af359eebaa4ffc5896a1aee3568c052c99879316a1ab57f8fe1789c390b6.tar.gz
+######################################################################## 100.0%
+==> libsigsegv: Executing phase: 'autoreconf'
+==> libsigsegv: Executing phase: 'configure'
+==> libsigsegv: Executing phase: 'build'
+==> libsigsegv: Executing phase: 'install'
+[+] /home/kru0052/Spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/libsigsegv-2.12-lctnabj6w4bmnyxo7q6ct4wewke2bqin
+==> Installing berkeley-db-18.1.40-bwuaqjex546zw3bimt23bgokfctnt46y
+==> No binary for berkeley-db-18.1.40-bwuaqjex546zw3bimt23bgokfctnt46y found: installing from source
+==> Fetching https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/_source-cache/archive/0c/0cecb2ef0c67b166de93732769abdeba0555086d51de1090df325e18ee8da9c8.tar.gz
+######################################################################## 100.0%
+...
...
```
diff --git a/docs.it4i/software/tools/virtualization.md b/docs.it4i/software/tools/virtualization.md
index 3ce3432c2..e4559e6b7 100644
--- a/docs.it4i/software/tools/virtualization.md
+++ b/docs.it4i/software/tools/virtualization.md
@@ -1,5 +1,10 @@
# Virtualization
+<!--
+musime proverit
+-->
+
+
Running virtual machines on compute nodes.
## Introduction
diff --git a/mkdocs.yml b/mkdocs.yml
index aded0562e..8c8dd110b 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -206,7 +206,6 @@ nav:
- ANSYS Fluent: software/tools/ansys/ansys-fluent.md
- ANSYS LS-DYNA: software/tools/ansys/ansys-ls-dyna.md
- ANSYS MAPDL: software/tools/ansys/ansys-mechanical-apdl.md
- - LS-DYNA: software/tools/ansys/ls-dyna.md
- Workbench: software/tools/ansys/workbench.md
- Setting License Preferences: software/tools/ansys/licensing.md
- Licensing and Available Versions: software/tools/ansys/setting-license-preferences.md
--
GitLab
From a0397f1cf1192890ca331db67cf78d8f08c36876 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Wed, 22 Sep 2021 13:32:53 +0200
Subject: [PATCH 58/58] Update docs.it4i/software/tools/spack.md
---
docs.it4i/software/tools/spack.md | 94 ++++++++++++++++---------------
1 file changed, 48 insertions(+), 46 deletions(-)
diff --git a/docs.it4i/software/tools/spack.md b/docs.it4i/software/tools/spack.md
index b318202b6..cfb932219 100644
--- a/docs.it4i/software/tools/spack.md
+++ b/docs.it4i/software/tools/spack.md
@@ -315,6 +315,16 @@ $ spack install git@2.29.0
######################################################################## 100.0%
...
...
+==> Fetching https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/_source-cache/archive/8f/8f3bf70ddb515674ce2e19572920a39b1be96af12032b77f1dd57898981fb151.tar.gz
+################################################################################################################################################################################################################################ 100.0%
+==> Moving resource stage
+ source : /tmp/kru0052/resource-git-manpages-cabbbb7qozeijgspy2wl3hf6on6f4b4c/spack-src/
+ destination : /tmp/kru0052/spack-stage-git-2.29.0-cabbbb7qozeijgspy2wl3hf6on6f4b4c/spack-src/git-manpages
+==> git: Executing phase: 'autoreconf'
+==> git: Executing phase: 'configure'
+==> git: Executing phase: 'build'
+==> git: Executing phase: 'install'
+[+] /home/kru0052/Spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/git-2.29.0-cabbbb7qozeijgspy2wl3hf6on6f4b4c
```
!!! warning
@@ -329,44 +339,6 @@ $ spack edit git
!!! note
To change the source link (`ftp://` to `http://`), use `spack create URL -f` to regenerate rules.
-#### **Example**
-
-```console
-$ spack install git
-==> Installing git
-==> Installing pcre
-==> Fetching ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.bz2
-curl: (7) couldn't connect to host
-==> Fetching from ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.bz2 failed.
-==> Error: FetchError: All fetchers failed for pcre-8.39-bm3lumpbghly2l7bkjsi4n2l3jyam6ax
-...
-
-$ spack create http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.bz2 -f
-==> This looks like a URL for pcre
-==> Found 2 versions of pcre:
-
- 8.41 http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.41.tar.bz2
- 8.40 http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.40.tar.bz2
-
-How many would you like to checksum? (default is 1, q to abort) 1
-==> Downloading...
-==> Fetching http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.41.tar.bz2
-######################################################################## 100,0%
-==> Checksummed 1 version of pcre
-==> This package looks like it uses the cmake build system
-==> Created template for pcre package
-==> Created package file: ~/.local/easybuild/software/Spack/0.10.0/var/spack/repos/builtin/packages/pcre/package.py
-$
-$ spack install git
-==> Installing git
-==> Installing pcre
-==> Installing cmake
-==> Installing ncurses
-==> Fetching http://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.0.tar.gz
-######################################################################## 100,0%
-...
-```
-
## Available Spack Module
We know that `spack list` shows you the names of available packages, but how do you figure out which are already installed?
@@ -382,6 +354,36 @@ Spack colorizes output.
```console
$ spack find | less -R
+-- linux-centos7-x86_64 / gcc@4.8.5 -----------------------------
+autoconf@2.69
+automake@1.16.2
+berkeley-db@18.1.40
+bzip2@1.0.8
+curl@7.72.0
+diffutils@3.7
+expat@2.2.10
+gdbm@1.18.1
+gettext@0.21
+git@2.29.0
+libbsd@0.10.0
+libedit@3.1-20191231
+libiconv@1.16
+libidn2@2.3.0
+libsigsegv@2.12
+libtool@2.4.6
+libunistring@0.9.10
+libxml2@2.9.10
+m4@1.4.18
+ncurses@6.2
+openssh@8.4p1
+openssl@1.1.1h
+pcre2@10.35
+perl@5.32.0
+pkgconf@1.7.3
+readline@8.0
+tar@1.32
+xz@5.2.5
+zlib@1.2.11
```
`spack find` shows the specs of installed packages. A spec is like a name, but it has a version, compiler, architecture, and build options associated with it. In Spack, you can have many installations of the same package with different specs.
@@ -398,24 +400,24 @@ $ spack load git
the commands below. Choose the right command for your shell.
For bash and zsh:
- . ~/.local/easybuild/software/Spack/0.10.0/share/spack/setup-env.sh
+ . ~/.local/easybuild/software/Spack/0.16.2/share/spack/setup-env.sh
For csh and tcsh:
- setenv SPACK_ROOT ~/.local/easybuild/software/Spack/0.10.0
- source ~/.local/easybuild/software/Spack/0.10.0/share/spack/setup-env.csh
+ setenv SPACK_ROOT ~/.local/easybuild/software/Spack/0.16.2
+ source ~/.local/easybuild/software/Spack/0.16.2/share/spack/setup-env.csh
```
### First Usage
```console
-$ . ~/.local/easybuild/software/Spack/0.10.0/share/spack/setup-env.sh
+$ . ~/.local/easybuild/software/Spack/0.16.2/share/spack/setup-env.sh
```
```console
$ git version 1.7.1
$ spack load git
$ git --version
-git version 2.11.0
+git version 2.29.0
$ spack unload git
$ git --version
git version 1.7.1
@@ -431,13 +433,13 @@ You may force uninstall a package with the `--force` option.
$ spack uninstall git
==> The following packages will be uninstalled :
--- linux-centos6-x86_64 / gcc@4.4.7 -----------------------------
-xmh3hmb git@2.11.0%gcc
+-- linux-centos7-x86_64 / gcc@4.4.7 -----------------------------
+xmh3hmb git@2.29.0%gcc
==> Do you want to proceed ? [y/n]
y
-==> Successfully uninstalled git@2.11.0%gcc@4.4.7 arch=linux-centos6-x86_64 -xmh3hmb
+==> Successfully uninstalled git@2.29.00%gcc@4.8.5 arch=linux-centos6-x86_64 -xmh3hmb
```
--
GitLab