Skip to content
Snippets Groups Projects
Commit 6686a364 authored by Lukáš Krupčík's avatar Lukáš Krupčík
Browse files

Update singularity-it4i.md

parent b745b2b7
Branches
Tags
4 merge requests!368Update prace.md to document the change from qprace to qprod as the default...,!367Update prace.md to document the change from qprace to qprod as the default...,!366Update prace.md to document the change from qprace to qprod as the default...,!323extended-acls-storage-section
Hide whitespace changes
Inline Side-by-side
docs.it4i/software/tools/singularity-it4i.md
+ 26
− 116
View file @ 6686a364
...@@ -3,18 +3,16 @@ ...@@ -3,18 +3,16 @@
On our clusters, the Singularity images of main linux distributions are prepared. On our clusters, the Singularity images of main linux distributions are prepared.
```console ```console
Salomon Anselm Salomon Anselm Barbora
├── CentOS ├── CentOS ├── CentOS ├── CentOS ├── CentOS
│ ├── 6.9 │ ├── 6.9 │ ├── 6 │ ├── 6 │ ├── 6
│ ├── 6.9-MIC │ ├── 6.9-GPU │ └── 7 │ └── 7 │ └── 7
│ ├── 7.5 │ ├── 7.4 ├── Debian ├── Debian ├── Debian
│ └── 7.5-intel │ └── 7.4-GPU │ └── latest │ └── latest │ └── latest
├── Debian ├── Debian ├── Fedora ├── Fedora ├── Fedora
│ └── 8.0 │ ├── 8.0 │ └── latest │ └── latest │ └── latest
├── Fedora │ └── 8.0-GPU └── Ubuntu └── Ubuntu └── Ubuntu
│ └── 29 └── Ubuntu └── latest └── latest └── latest
└── Ubuntu ├── 16.04
└── 16.04 └── 16.04-GPU
``` ```
!!! info !!! info
...@@ -22,9 +20,6 @@ On our clusters, the Singularity images of main linux distributions are prepared ...@@ -22,9 +20,6 @@ On our clusters, the Singularity images of main linux distributions are prepared
The bootstrap scripts, wrappers, features, etc. are located [here][a]. The bootstrap scripts, wrappers, features, etc. are located [here][a].
!!! note
The images with graphic card support are marked as **-GPU** and images with Intel Xeon Phi support are marked as **-MIC**
## IT4Innovations Singularity Wrappers ## IT4Innovations Singularity Wrappers
For better user experience with Singularity containers we prepared several wrappers: For better user experience with Singularity containers we prepared several wrappers:
...@@ -56,8 +51,8 @@ In the examples, we will use the prepared Singularity images. ...@@ -56,8 +51,8 @@ In the examples, we will use the prepared Singularity images.
### Load Image ### Load Image
```console ```console
$ ml CentOS/6.9 $ ml CentOS/6
Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS-6.9_20180220133305.img Your image of CentOS/6 is at location: /home/login/.singularity/images/CentOS-6_20180220133305.img
``` ```
!!! tip !!! tip
...@@ -70,8 +65,8 @@ Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS- ...@@ -70,8 +65,8 @@ Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS-
Executes the given command inside the Singularity image. The container is in this case started, then the command is executed and the container is stopped. Executes the given command inside the Singularity image. The container is in this case started, then the command is executed and the container is stopped.
```console ```console
$ ml CentOS/7.3 $ ml CentOS/7
Your image of CentOS/7.3 is at location: /home/login/.singularity/images/CentOS-7.3_20180220104046.img Your image of CentOS/7 is at location: /home/login/.singularity/images/CentOS-7_20180220104046.img
$ image-exec cat /etc/centos-release $ image-exec cat /etc/centos-release
CentOS Linux release 7.3.1708 (Core) CentOS Linux release 7.3.1708 (Core)
``` ```
...@@ -89,11 +84,11 @@ This command runs the runscript inside the Singularity image. Note, that the pre ...@@ -89,11 +84,11 @@ This command runs the runscript inside the Singularity image. Note, that the pre
Invokes an interactive shell inside the Singularity image. Invokes an interactive shell inside the Singularity image.
```console ```console
$ ml CentOS/7.3 $ ml CentOS/7
$ image-shell $ image-shell
Singularity: Invoking an interactive shell within container... Singularity: Invoking an interactive shell within container...
Singularity CentOS-7.3_20180220104046.img:~> Singularity CentOS-7_20180220104046.img:~>
``` ```
### Update Image ### Update Image
...@@ -101,104 +96,19 @@ Singularity CentOS-7.3_20180220104046.img:~> ...@@ -101,104 +96,19 @@ Singularity CentOS-7.3_20180220104046.img:~>
This command is for updating your local copy of the Singularity image. The local copy is overridden in this case. This command is for updating your local copy of the Singularity image. The local copy is overridden in this case.
```console ```console
$ ml CentOS/6.9 $ ml CentOS/6
New version of CentOS image was found. (New: CentOS-6.9_20180220092823.img Old: CentOS-6.9_20170220092823.img) New version of CentOS image was found. (New: CentOS-6_20180220092823.img Old: CentOS-6_20170220092823.img)
For updating image use: image-update For updating image use: image-update
Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS-6.9_20170220092823.img Your image of CentOS/6 is at location: /home/login/.singularity/images/CentOS-6_20170220092823.img
$ image-update $ image-update
New version of CentOS image was found. (New: CentOS-6.9_20180220092823.img Old: CentOS-6.9_20170220092823.img) New version of CentOS image was found. (New: CentOS-6_20180220092823.img Old: CentOS-6_20170220092823.img)
Do you want to update local copy? (WARNING all user modification will be deleted) [y/N]: y Do you want to update local copy? (WARNING all user modification will be deleted) [y/N]: y
Updating image CentOS-6.9_20180220092823.img Updating image CentOS-6_20180220092823.img
2.71G 100% 199.49MB/s 0:00:12 (xfer#1, to-check=0/1) 2.71G 100% 199.49MB/s 0:00:12 (xfer#1, to-check=0/1)
sent 2.71G bytes received 31 bytes 163.98M bytes/sec sent 2.71G bytes received 31 bytes 163.98M bytes/sec
total size is 2.71G speedup is 1.00 total size is 2.71G speedup is 1.00
New version is ready. (/home/login/.singularity/images/CentOS-6.9_20180220092823.img) New version is ready. (/home/login/.singularity/images/CentOS-6_20180220092823.img)
```
### Intel Xeon Phi Cards - MIC
In the following example, we are using a job submitted by the command: `qsub -A PROJECT -q qprod -l select=1:mpiprocs=24:accelerator=true -I`
!!! info
The MIC image was prepared only for the Salomon cluster.
**Code for the Offload Test**
```c
#include <stdio.h>
#include <thread>
#include <stdlib.h>
#include <unistd.h>
int main() {
char hostname[1024];
gethostname(hostname, 1024);
unsigned int nthreads = std::thread::hardware_concurrency();
printf("Hello world, #of cores: %d\n",nthreads);
#pragma offload target(mic)
{
nthreads = std::thread::hardware_concurrency();
printf("Hello world from MIC, #of cores: %d\n",nthreads);
}
}
```
**Compile and Run**
```console
[login@r38u03n975 ~]$ ml CentOS/6.9-MIC
Your image of CentOS/6.9-MIC is at location: /home/login/.singularity/images/CentOS-6.9-MIC_20180220112004.img
[login@r38u03n975 ~]$ image-shell
Singularity: Invoking an interactive shell within container...
Singularity CentOS-6.9-MIC_20180220112004.img:~> ml intel/2017b
Singularity CentOS-6.9-MIC_20180220112004.img:~> ml
Currently Loaded Modules:
1) GCCcore/6.3.0 3) icc/2017.1.132-GCC-6.3.0-2.27 5) iccifort/2017.1.132-GCC-6.3.0-2.27 7) iimpi/2017a 9) intel/2017a
2) binutils/2.27-GCCcore-6.3.0 4) ifort/2017.1.132-GCC-6.3.0-2.27 6) impi/2017.1.132-iccifort-2017.1.132-GCC-6.3.0-2.27 8) imkl/2017.1.132-iimpi-2017a
Singularity CentOS-6.9-MIC_20180220112004.img:~> icpc -std=gnu++11 -qoffload=optional hello.c -o hello-host
Singularity CentOS-6.9-MIC_20180220112004.img:~> ./hello-host
Hello world, #of cores: 24
Hello world from MIC, #of cores: 244
```
### GPU Image
In the following example, we are using a job submitted by the command: `qsub -A PROJECT -q qnvidia -l select=1:ncpus=16:mpiprocs=16 -l walltime=01:00:00 -I`
!!! note
The GPU image was prepared only for the Anselm cluster.
**Checking NVIDIA Driver Inside Image**
```console
[login@cn199.anselm ~]$ image-shell
Singularity: Invoking an interactive shell within container...
Singularity CentOS-6.9-GPU_20180309130604.img:~> ml
No modules loaded
Singularity CentOS-6.9-GPU_20180309130604.img:~> nvidia-smi
Mon Mar 12 07:07:53 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.30 Driver Version: 390.30 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20m Off | 00000000:02:00.0 Off | 0 |
| N/A 28C P0 51W / 225W | 0MiB / 4743MiB | 89% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
``` ```
### MPI ### MPI
...@@ -222,11 +132,11 @@ performance benefits (e.g. if a container wasn’t built with the proper OFED as ...@@ -222,11 +132,11 @@ performance benefits (e.g. if a container wasn’t built with the proper OFED as
#### MPI Inside Singularity Image #### MPI Inside Singularity Image
```console ```console
$ ml CentOS/6.9 $ ml CentOS/7
$ image-shell $ image-shell
Singularity: Invoking an interactive shell within container... Singularity: Invoking an interactive shell within container...
Singularity CentOS-6.9_20180220092823.img:~> mpirun hostname | wc -l Singularity CentOS-7_20180220092823.img:~> mpirun hostname | wc -l
24 24
``` ```
...@@ -235,8 +145,8 @@ As you can see in this example, we allocated two nodes, but MPI can use only one ...@@ -235,8 +145,8 @@ As you can see in this example, we allocated two nodes, but MPI can use only one
#### MPI Outside Singularity Image #### MPI Outside Singularity Image
```console ```console
$ ml CentOS/6.9 $ ml CentOS/7
Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS-6.9_20180220092823.img Your image of CentOS/7 is at location: /home/login/.singularity/images/CentOS-7_20180220092823.img
$ image-mpi hostname | wc -l $ image-mpi hostname | wc -l
48 48
``` ```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment