diff --git a/docs.it4i/software/tools/singularity.md b/docs.it4i/software/tools/singularity.md index 03ecdb3b5ab17bedd66148c14f09bed21377da9d..3578dea7c2fb8bd39b9a4eca5228402405de50b4 100644 --- a/docs.it4i/software/tools/singularity.md +++ b/docs.it4i/software/tools/singularity.md @@ -1,74 +1,61 @@ -# Singularity Container +# Apptainer Container -[Singularity][a] enables users to have full control of their environment. A non-privileged user can "swap out" the operating system on the host for one they control. So if the host system is running RHEL6 but your application runs in Ubuntu/RHEL7, you can create an Ubuntu/RHEL7 image, install your applications into that image, copy the image to another host, and run your application on that host in its native Ubuntu/RHEL7 environment. - -Singularity also allows you to leverage the resources of whatever host you are on. This includes HPC interconnects, resource managers, file systems, GPUs and/or accelerators, etc. Singularity does this by enabling several key facets: - -* Encapsulation of the environment -* Containers are image based -* No user contextual changes or root escalation allowed -* No root owned daemon processes - -This documentation is for Singularity version 2.4 and newer. - -<div align="center"> - <iframe width="250" height="50%" src="https://www.youtube.com/embed/m8llDjFuXlc" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> - <iframe width="250" height="50%" src="https://www.youtube.com/embed/SJHizTjwyFk" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> - <iframe width="250" height="50%" src="https://www.youtube.com/embed/97VuBVnfcwg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> - <iframe width="250" height="50%" src="https://www.youtube.com/embed/wGJnkRmW5iU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> -</div> +[Apptainer][a] is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. You can build a container using Apptainer on your laptop, and then run it on many of the largest HPC clusters in the world, local university or company clusters, a single server, in the cloud, or on a workstation down the hall. Your container is a single file, and you don’t have to worry about how to install all the software you need on each different operating system. ## Using Docker Images -Singularity can import, bootstrap, and even run Docker images directly from [Docker Hub][b]. You can easily run an RHEL7 container like this: +Apptainer can import, bootstrap, and even run Docker images directly from [Docker Hub][b]. You can easily run an CentOS container like this: ```console -hra0031@login4:~$ cat /etc/redhat-release -CentOS release 6.9 (Final) -hra0031@login4:~$ ml Singularity -hra0031@login4:~$ singularity shell docker://centos:latest -Docker image path: index.docker.io/library/centos:latest -Cache folder set to /home/hra0031/.singularity/docker -[1/1] |===================================| 100.0% -Creating container runtime... -Singularity: Invoking an interactive shell within container... - -Singularity centos:latest:~> cat /etc/redhat-release -CentOS Linux release 7.4.1708 (Core) +$ cat /etc/redhat-release +CentOS Linux release 7.9.2009 (Core) +$ ml apptainer +$ apptainer shell docker://centos:latest +INFO: Converting OCI blobs to SIF format +INFO: Starting build... +Getting image source signatures +Copying blob a1d0c7532777 done +Copying config 8c1402b22a done +Writing manifest to image destination +Storing signatures +2023/01/17 12:55:08 info unpack layer: sha256:a1d0c75327776413fa0db9ed3adcdbadedc95a662eb1d360dad82bb913f8a1d1 +2023/01/17 12:55:09 warn rootless{usr/bin/newgidmap} ignoring (usually) harmless EPERM on setxattr "security.capability" +2023/01/17 12:55:09 warn rootless{usr/bin/newuidmap} ignoring (usually) harmless EPERM on setxattr "security.capability" +2023/01/17 12:55:09 warn rootless{usr/bin/ping} ignoring (usually) harmless EPERM on setxattr "security.capability" +2023/01/17 12:55:10 warn rootless{usr/sbin/arping} ignoring (usually) harmless EPERM on setxattr "security.capability" +2023/01/17 12:55:10 warn rootless{usr/sbin/clockdiff} ignoring (usually) harmless EPERM on setxattr "security.capability" +INFO: Creating SIF file... +Apptainer> cat /etc/redhat-release +CentOS Linux release 8.4.2105 ``` -In this case, the image is downloaded from Docker Hub, extracted to a temporary directory, and Singularity interactive shell is invoked. This procedure can take a lot of time, especially with large images. +In this case, the image is downloaded from Docker Hub, extracted to a temporary directory, and Apptainer interactive shell is invoked. This procedure can take a lot of time, especially with large images. ## Importing Docker Image -Singularity containers can be in three different formats: +Apptainer containers can be in three different formats: * read-only **squashfs** (default) - best for production * writable **ext3** (--writable option) * writable **(ch)root directory** (--sandbox option) - best for development -Squashfs and (ch)root directory images can be built from Docker source directly on the cluster, no root privileges are needed. It is strongly recommended to create a native Singularity image to speed up the launch of the container. +Squashfs and (ch)root directory images can be built from Docker source directly on the cluster, no root privileges are needed. It is strongly recommended to create a native Apptainer image to speed up the launch of the container. ```console -hra0031@login4:~$ ml Singularity -hra0031@login4:~$ singularity build ubuntu.img docker://ubuntu:latest -Docker image path: index.docker.io/library/ubuntu:latest -Cache folder set to /home/hra0031/.singularity/docker -Importing: base Singularity environment -Importing: /home/hra0031/.singularity/docker/sha256:50aff78429b146489e8a6cb9334d93a6d81d5de2edc4fbf5e2d4d9253625753e.tar.gz -Importing: /home/hra0031/.singularity/docker/sha256:f6d82e297bce031a3de1fa8c1587535e34579abce09a61e37f5a225a8667422f.tar.gz -Importing: /home/hra0031/.singularity/docker/sha256:275abb2c8a6f1ce8e67a388a11f3cc014e98b36ff993a6ed1cc7cd6ecb4dd61b.tar.gz -Importing: /home/hra0031/.singularity/docker/sha256:9f15a39356d6fc1df0a77012bf1aa2150b683e46be39d1c51bc7a320f913e322.tar.gz -Importing: /home/hra0031/.singularity/docker/sha256:fc0342a94c89e477c821328ccb542e6fb86ce4ef4ebbf1098e85669e051ef0dd.tar.gz -Importing: /home/hra0031/.singularity/metadata/sha256:c6a9ef4b9995d615851d7786fbc2fe72f72321bee1a87d66919b881a0336525a.tar.gz -WARNING: Building container as an unprivileged user. If you run this container as root -WARNING: it may be missing some functionality. -Building Singularity image... -Singularity container built: ubuntu.img -Cleaning up... +$ ml apptainer +$ apptainer build ubuntu.sif docker://ubuntu:latest +INFO: Starting build... +Getting image source signatures +Copying blob 6e3729cf69e0 done +Copying config 415250ec06 done +Writing manifest to image destination +Storing signatures +2023/01/17 12:58:04 info unpack layer: sha256:6e3729cf69e0ce2de9e779575a1fec8b7fb5efdfa822829290ab6d5d1bc3e797 +INFO: Creating SIF file... +INFO: Build complete: ubuntu.sif ``` -alternatively, you can create your own docker image and import it to singularity. +alternatively, you can create your own docker image and import it to Apptainer. For example, we show how to create and run ubuntu docker image with gvim installed: ```console @@ -83,48 +70,59 @@ your_local_machine $ docker tag 4dd97cefde62 ubuntu_gvim your_local_machine $ docker save -o ubuntu_gvim.tar ubuntu_gvim ``` -copy the `ubuntu_gvim.tar` archive to IT4I supercomputers, convert to Singularity image and run: +copy the `ubuntu_gvim.tar` archive to IT4I supercomputers, convert to Apptainer image and run: ```console -$ ml Singularity -$ singularity build ubuntu_givm.img docker-archive://ubuntu_gvim.tar -$ singularity shell -B /usr/user/$ID ubuntu_gvim.img +$ ml Apptainer +$ apptainer build ubuntu_givm.sif docker-archive://ubuntu_gvim.tar +$ apptainer shell -B /usr/user/$ID ubuntu_gvim.sif ``` Note the bind to `/usr/user/$ID` directory. ## Launching the Container -The interactive shell can be invoked by the `singularity shell` command. This is useful for development purposes. Use the `-w | --writable` option to make changes inside the container permanent. +The interactive shell can be invoked by the `apptainer shell` command. This is useful for development purposes. Use the `-w | --writable` option to make changes inside the container permanent. ```console -hra0031@login4:~$ singularity shell -w ubuntu.img -Singularity: Invoking an interactive shell within container... +$ apptainer shell ubuntu.sif -Singularity ubuntu.img:~> cat /etc/lsb-release +Apptainer> cat /etc/lsb-release DISTRIB_ID=Ubuntu -DISTRIB_RELEASE=16.04 -DISTRIB_CODENAME=xenial -DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS" +DISTRIB_RELEASE=22.04 +DISTRIB_CODENAME=jammy +DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS" ``` -A command can be run inside the container (without an interactive shell) by invoking the `singularity exec` command. +A command can be run inside the container (without an interactive shell) by invoking the `apptainer exec` command. ``` -hra0031@login4:~$ singularity exec ubuntu.img cat /etc/lsb-release +$ apptainer exec ubuntu.sif cat /etc/lsb-release DISTRIB_ID=Ubuntu -DISTRIB_RELEASE=16.04 -DISTRIB_CODENAME=xenial -DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS" +DISTRIB_RELEASE=22.04 +DISTRIB_CODENAME=jammy +DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"" ``` -A singularity image can contain a runscript. This script is executed inside the container after the `singularity run` command is used. The runscript is mostly used to run an application for which the container is built. In the following example, it is the `fortune | cowsay` command: +A Apptainer image can contain a runscript. This script is executed inside the container after the `apptainer run` command is used. The runscript is mostly used to run an application for which the container is built. In the following example, it is the `fortune | cowsay` command: ``` -hra0031@login4:~$ singularity run ubuntu.img - ___________________ -< Are you a turtle? > - ------------------- +$ apptainer build lolcow.sif docker://ghcr.io/apptainer/lolcow +INFO: Starting build... +Getting image source signatures +Copying blob 5ca731fc36c2 skipped: already exists +Copying blob 16ec32c2132b skipped: already exists +Copying config fd0daa4d89 done +Writing manifest to image destination +Storing signatures +2023/01/17 13:06:01 info unpack layer: sha256:16ec32c2132b43494832a05f2b02f7a822479f8250c173d0ab27b3de78b2f058 +2023/01/17 13:06:01 info unpack layer: sha256:5ca731fc36c28789c5ddc3216563e8bfca2ab3ea10347e07554ebba1c953242e +INFO: Creating SIF file... +INFO: Build complete: lolcow.sif +$ apptainer exec lolcow.sif cowsay moo + _____ +< moo > + ----- \ ^__^ \ (oo)\_______ (__)\ )\/\ @@ -140,15 +138,13 @@ A user home directory is mounted inside the container automatically. If you need The mounted folder has to exist inside the container or the container image has to be writable! ```console -hra0031@login4:~$ singularity shell -B /scratch -w ubuntu.img -Singularity: Invoking an interactive shell within container... - -Singularity ubuntu.img:~> ls /scratch +$ apptainer shell -B /scratch ubuntu.sif +Apptainer> ls /scratch ddn sys temp work ``` -A comprehensive documentation can be found at the [Singularity][c] website. +A comprehensive documentation can be found at the [Apptainer Quick Start][c] website. -[a]: http://singularity.lbl.gov/ +[a]: https://apptainer.org/docs/user/latest/introduction.html [b]: https://hub.docker.com/ -[c]: http://singularity.lbl.gov/quickstart +[c]: https://apptainer.org/docs/user/latest/quick_start.html