Skip to content
Snippets Groups Projects
Commit bc50e254 authored by Josef Hrabal's avatar Josef Hrabal
Browse files

Singularity documentation reworked for version 2.4

parent 0e534013
No related branches found
No related tags found
6 merge requests!368Update prace.md to document the change from qprace to qprod as the default...,!367Update prace.md to document the change from qprace to qprod as the default...,!366Update prace.md to document the change from qprace to qprod as the default...,!323extended-acls-storage-section,!196Master,!180Singularity
Hide whitespace changes
Inline Side-by-side
docs.it4i/software/tools/singularity.md
+ 80
− 93
View file @ bc50e254
...@@ -9,122 +9,109 @@ Singularity also allows you to leverage the resources of whatever host you are o ...@@ -9,122 +9,109 @@ Singularity also allows you to leverage the resources of whatever host you are o
* No user contextual changes or root escalation allowed * No user contextual changes or root escalation allowed
* No root owned daemon processes * No root owned daemon processes
These documentation are for Singularity Version 2.4 and newer.
## Using Docker Images ## Using Docker Images
Singularity can import, bootstrap, and even run Docker images directly from [Docker Hub](https://hub.docker.com/). You can easily run RHEL7 like this: Singularity can import, bootstrap, and even run Docker images directly from [Docker Hub](https://hub.docker.com/). You can easily run RHEL7 like this:
```console ```console
[hrb33@r33u01n865 ~]$ cat /etc/redhat-release hra0031@login4:~$ cat /etc/redhat-release
CentOS release 6.7 (Final) CentOS release 6.9 (Final)
[hrb33@r33u01n865 ~]$ ml Singularity hra0031@login4:~$ ml Singularity
[hrb33@r33u01n865 ~]$ singularity shell docker://centos:latest hra0031@login4:~$ singularity shell docker://centos:latest
library/centos:latest Docker image path: index.docker.io/library/centos:latest
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 Cache folder set to /home/hra0031/.singularity/docker
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 [1/1] |===================================| 100.0%
Downloading layer: sha256:45a2e645736c4c66ef34acce2407ded21f7a9b231199d3b92d6c9776df264729 Creating container runtime...
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Singularity: Invoking an interactive shell within container... Singularity: Invoking an interactive shell within container...
Singularity.centos:latest> cat /etc/redhat-release Singularity centos:latest:~> cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core) CentOS Linux release 7.4.1708 (Core)
``` ```
In this case, Image is downloaded from Docker Hub, extracted to a temporary directory and singularity interactive shell is invoked. This procedure can take a lot of time, especially for larger images.
## Creating Own Image From Docker Image ## Creating Own Image From Docker Image
Singularity containers can be in three different formats:
* read-only **squashfs** (default) - best for production
* writable **ext3** (--writable option)
* writable **(ch)root directory** (--sandbox option) - best for development
Squashfs and (ch)root directory image can be built from docker source directly on the cluster, no root privileges are needed. Creating native singularity image is strongly recommended due to the speed of launching the container.
```console
hra0031@login4:~$ ml Singularity
hra0031@login4:~$ singularity build ubuntu.img docker://ubuntu:latest
Docker image path: index.docker.io/library/ubuntu:latest
Cache folder set to /home/hra0031/.singularity/docker
Importing: base Singularity environment
Importing: /home/hra0031/.singularity/docker/sha256:50aff78429b146489e8a6cb9334d93a6d81d5de2edc4fbf5e2d4d9253625753e.tar.gz
Importing: /home/hra0031/.singularity/docker/sha256:f6d82e297bce031a3de1fa8c1587535e34579abce09a61e37f5a225a8667422f.tar.gz
Importing: /home/hra0031/.singularity/docker/sha256:275abb2c8a6f1ce8e67a388a11f3cc014e98b36ff993a6ed1cc7cd6ecb4dd61b.tar.gz
Importing: /home/hra0031/.singularity/docker/sha256:9f15a39356d6fc1df0a77012bf1aa2150b683e46be39d1c51bc7a320f913e322.tar.gz
Importing: /home/hra0031/.singularity/docker/sha256:fc0342a94c89e477c821328ccb542e6fb86ce4ef4ebbf1098e85669e051ef0dd.tar.gz
Importing: /home/hra0031/.singularity/metadata/sha256:c6a9ef4b9995d615851d7786fbc2fe72f72321bee1a87d66919b881a0336525a.tar.gz
WARNING: Building container as an unprivileged user. If you run this container as root
WARNING: it may be missing some functionality.
Building Singularity image...
Singularity container built: ubuntu.img
Cleaning up...
```
## Launching the Container
The interactive shell can be invoked by the **shell** command. This is useful for development purposes. To make changes inside the container permanent use **-w | --writable** option.
```console ```console
hrb33@hrb33-toshiba:/$ cd /tmp/ hra0031@login4:~$ singularity shell -w ubuntu.img
hrb33@hrb33-toshiba:/tmp$ sudo singularity create /tmp/c7.img
[sudo] password for hrb33:
Creating a new image with a maximum size of 768MiB...
Executing image create helper
Formatting image with ext3 file system
Done.
hrb33@hrb33-toshiba:/tmp$ sudo singularity import c7.img docker://centos:latest
library/centos:latest
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Downloading layer: sha256:45a2e645736c4c66ef34acce2407ded21f7a9b231199d3b92d6c9776df264729
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Adding Docker CMD as Singularity runscript...
Bootstrap initialization
No bootstrap definition passed, updating container
Executing Prebootstrap module
Executing Postbootstrap module
Done.
hrb33@hrb33-toshiba:/tmp$ sudo singularity shell --writable c7.img
Singularity: Invoking an interactive shell within container... Singularity: Invoking an interactive shell within container...
Singularity.c7.img> mkdir /apps /scratch Singularity ubuntu.img:~> cat /etc/lsb-release
Singularity.c7.img> exit DISTRIB_ID=Ubuntu
hrb33@hrb33-toshiba:/tmp$ rsync -av c7.img hrb33@login4.salomon:/home/hrb33/c7.img DISTRIB_RELEASE=16.04
sending incremental file list DISTRIB_CODENAME=xenial
c7.img DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
```
sent 805,503,090 bytes received 34 bytes 9,205,749.99 bytes/sec To execute command inside container (without interactive shell) use **exec** command.
total size is 805,306,399 speedup is 1.00
```
hra0031@login4:~$ singularity exec ubuntu.img cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
``` ```
Accessing /HOME and /SCRATCH Within Container Singularity image can contain a runscript. This script is executed inside container after the **run** command is used. The runscript is mostly used to run an application for which is the container build, in the following example it is **fortune | cowsay** command.
```console ```
hrb33@hrb33-toshiba:/tmp$ ssh hrb33@login4.salomon hra0031@login4:~$ singularity run ubuntu.img
___________________
< Are you a turtle? >
-------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
```
_____ _ ## Accessing /HOME and /SCRATCH Within Container
/ ____| | |
| (___ __ _| | ___ _ __ ___ ___ _ __
\___ \ / _` | |/ _ \| '_ ` _ \ / _ \| '_ \
____) | (_| | | (_) | | | | | | (_) | | | |
|_____/ \__,_|_|\___/|_| |_| |_|\___/|_| |_|
http://www.it4i.cz/?lang=en By default, user home directory is mounted inside the container, therefore you can access your files directly. If you need access to **/SCRATCH** storage for your computation, this must by mounted be **-B | --bind** option.
!!!Warning
Be aware, that the mounted folder has to exist inside the container or the container image has to be writable!
Last login: Fri Feb 10 14:38:36 2017 from 10.0.131.12 ```console
[hrb33@login4.salomon ~]$ ml Singularity hra0031@login4:~$ singularity shell -B /scratch -w ubuntu.img
[hrb33@login4.salomon ~]$ singularity shell --bind /scratch --bind /apps --writable c7.img
Singularity: Invoking an interactive shell within container... Singularity: Invoking an interactive shell within container...
Singularity.c7.img> ls /apps/ -l Singularity ubuntu.img:~> ls /scratch
total 68 ddn sys temp work
drwx------ 4 root root 29 Sep 29 10:28 SCS
drwxrwxr-x 301 2757 2796 8192 Feb 16 10:58 all
drwxrwxr-x 3 2757 2796 19 Jul 9 2015 base
drwxrwxr-x 16 2757 2796 4096 Nov 24 21:47 bio
drwxrwxr-x 10 2757 2796 116 Apr 8 2016 cae
drwxrwxr-x 18 2757 2796 4096 Jan 17 09:49 chem
drwxrwxr-x 11 2757 2796 122 Dec 7 09:25 compiler
drwxrwxr-x 7 2757 2796 73 Jun 29 2016 data
drwxr-xr-x 7 2757 2796 88 Jan 8 2016 debugger
drwxrwxr-x 38 2757 2796 4096 Feb 16 13:37 devel
drwxrwxr-x 9 2757 2796 130 Jan 9 08:40 easybuild
drwxr-xr-x 11 3900 4011 4096 Feb 15 09:50 gentoo
drwxr-xr-x 10 3900 4011 4096 Feb 10 17:01 gentoo_uv
drwxrwxr-x 5 2757 2796 39 Jan 18 2016 geo
drwxr-xr-x 18 2757 2796 4096 Sep 6 16:03 intel2017
drwxrwxr-x 20 2757 2796 4096 Nov 28 08:50 lang
drwxrwxr-x 31 2757 2796 4096 Dec 7 07:48 lib
drwxrwxr-x 4 2757 2796 32 Nov 9 09:19 licenses
drwxrwxr-x 17 2757 2796 4096 Nov 15 09:24 math
drwxr-xr-x 22 2757 2796 4096 Jan 19 13:15 modules
drwxrwxr-x 8 2757 2796 82 Apr 18 2016 mpi
drwxrwxr-x 13 2757 2796 4096 Oct 24 09:08 numlib
drwxrwxr-x 10 2757 2796 108 Feb 3 11:01 perf
drwxrwxr-x 5 2757 2796 41 Jan 17 09:49 phys
drwxrwxr-x 2 2757 2796 6 Feb 3 11:01 prace
drwxr-xr-x 4 root root 36 Jun 18 2015 sw
drwxrwxr-x 5 2757 2796 49 Feb 15 2016 system
drwxr-xr-x 3 root root 19 Dec 4 2015 test
drwxrwxr-x 13 2757 2796 138 May 31 2016 toolchain
drwxrwxr-x 39 2757 2796 4096 Feb 3 11:27 tools
drwxr-xr-x 4 root root 31 Aug 11 2015 user
drwxrwxr-x 21 2757 2796 4096 Jan 5 18:56 uv
drwxrwxr-x 40 2757 2796 4096 Feb 3 11:01 vis
Singularity.c7.img> ls /scratch/ -l
total 32
drwx------ 3 root root 4096 Aug 15 2016 backup
drwxr-x--- 2 root root 4096 Dec 5 10:34 sys
drwxrwxrwt 154 root root 20480 Feb 14 14:03 temp
drwxr-xr-x 4 root root 4096 Jan 25 10:48 work
Singularity.c7.img>
``` ```
Comprehensive documentation can be found at the [Singularity](http://singularity.lbl.gov/quickstart) website.
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment