From 91c3144b3bd2ced3ba118d31362055c97f3c1ee3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Krup=C4=8D=C3=ADk?= <lukas.krupcik@vsb.cz>
Date: Wed, 7 Mar 2018 14:41:23 +0100
Subject: [PATCH] fix
---
docs.it4i/software/tools/singularity-it4i.md | 95 +++++++++++++++++++-
1 file changed, 92 insertions(+), 3 deletions(-)
diff --git a/docs.it4i/software/tools/singularity-it4i.md b/docs.it4i/software/tools/singularity-it4i.md
index fbb02b2ac..63ab62bae 100644
--- a/docs.it4i/software/tools/singularity-it4i.md
+++ b/docs.it4i/software/tools/singularity-it4i.md
@@ -1,6 +1,7 @@
# Singularity on IT4Innovations
On clusters we have different versions of operating systems singularity images. Below you see the available operating systems singularity images.
+Our images bootstrap, wrappers, features are [here](https://code.it4i.cz/sccs/it4i-singularity).
```console
Salomon Anselm
@@ -44,7 +45,7 @@ Open image and run command into image.
```console
$ ml CentOS/7.3
-Your image of CentOS/7.3 is at location: /home/kru0052/.singularity/images/CentOS-7.3_20180220104046.img
+Your image of CentOS/7.3 is at location: /home/login/.singularity/images/CentOS-7.3_20180220104046.img
image-exec cat /etc/centos-release
CentOS Linux release 7.3.1708 (Core)
```
@@ -103,7 +104,7 @@ Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS-
!!! note
First usage image copy image from /apps/all/OS/... to your /home (.singularity/images)
-For GPU and MIC image
+For GPU and MIC images
```console
$ ml CentOS/6.9-GPU
@@ -116,6 +117,94 @@ $ ml CentOS/6.9-MIC
!!! note
For the GPU image, you must allocate node with GPU card and for the MIC image, you must allocate node with the Intel Xeon Phi cards.
+### Intel Xeon Phi Cards - MIC
+
+For example submit job `qsub -A PROJECT -q qprod -l select=1:mpiprocs=24:accelerator=true -I`
+
+!!! info
+ MIC image was prepared for only Salomon cluster
+
+**Code for testing offload**
+
+```c
+#include <stdio.h>
+#include <thread>
+#include <stdlib.h>
+#include <unistd.h>
+
+int main() {
+
+ char hostname[1024];
+ gethostname(hostname, 1024);
+
+ unsigned int nthreads = std::thread::hardware_concurrency();
+ printf("Hello world, #of cores: %d\n",nthreads);
+ #pragma offload target(mic)
+ {
+ nthreads = std::thread::hardware_concurrency();
+ printf("Hello world from MIC, #of cores: %d\n",nthreads);
+ }
+
+
+
+}
+```
+
+**Compile and run**
+
+```console
+[login@r38u03n975 ~]$ ml CentOS/6.9-MIC
+Your image of CentOS/6.9-MIC is at location: /home/login/.singularity/images/CentOS-6.9-MIC_20180220112004.img
+[login@r38u03n975 ~]$ image-shell
+Singularity: Invoking an interactive shell within container...
+
+Singularity CentOS-6.9-MIC_20180220112004.img:~> ml intel/2017b
+Singularity CentOS-6.9-MIC_20180220112004.img:~> ml
+
+Currently Loaded Modules:
+ 1) GCCcore/6.3.0 3) icc/2017.1.132-GCC-6.3.0-2.27 5) iccifort/2017.1.132-GCC-6.3.0-2.27 7) iimpi/2017a 9) intel/2017a
+ 2) binutils/2.27-GCCcore-6.3.0 4) ifort/2017.1.132-GCC-6.3.0-2.27 6) impi/2017.1.132-iccifort-2017.1.132-GCC-6.3.0-2.27 8) imkl/2017.1.132-iimpi-2017a
+Singularity CentOS-6.9-MIC_20180220112004.img:~> icpc -std=gnu++11 -qoffload=optional hello.c -o hello-host
+Singularity CentOS-6.9-MIC_20180220112004.img:~> ./hello-host
+Hello world, #of cores: 24
+Hello world from MIC, #of cores: 244
+```
+
+### GPU image
+
+For example submit job `qsub -A PROJECT -q qnvidia -l select=1:ncpus=16:mpiprocs=16 -l walltime=01:00:00 -I`
+
+!!! info
+ GPU image was prepared for only Anselm cluster
+
+**Checking nvidia driver inside image**
+
+```console
+[login@cn187.anselm ~]$ ml CentOS/6.9-GPU
+Your image of CentOS/6.9-GPU is at location: /home/login/.singularity/images/CentOS-6.9-GPU_20171024164243.img
+[login@cn187.anselm ~]$ image-shell
+Singularity: Invoking an interactive shell within container...
+
+Singularity CentOS-6.9-GPU_20171024164243.img:~> nvidia-smi
+Wed Mar 7 14:35:40 2018
++-----------------------------------------------------------------------------+
+| NVIDIA-SMI 384.111 Driver Version: 384.111 |
+|-------------------------------+----------------------+----------------------+
+| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
+| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
+|===============================+======================+======================|
+| 0 Tesla K20m Off | 00000000:02:00.0 Off | 0 |
+| N/A 37C P0 54W / 225W | 0MiB / 4742MiB | 87% Default |
++-------------------------------+----------------------+----------------------+
+
++-----------------------------------------------------------------------------+
+| Processes: GPU Memory |
+| GPU PID Type Process name Usage |
+|=============================================================================|
+| No running processes found |
++-----------------------------------------------------------------------------+
+```
+
### MPI
For example submit job `qsub -A PROJECT -q qprod -l select=2:mpiprocs=24 -l walltime=00:30:00 -I`
@@ -151,7 +240,7 @@ Singularity CentOS-6.9_20180220092823.img:~> mpirun hostname | wc -l
```console
$ ml CentOS/6.9
-Your image of CentOS/6.9 is at location: /home/kru0052/.singularity/images/CentOS-6.9_20180220092823.img
+Your image of CentOS/6.9 is at location: /home/login/.singularity/images/CentOS-6.9_20180220092823.img
$ image-mpi hostname | wc -l
48
```
--
GitLab