Commit f356318d authored by Jan Siwiec's avatar Jan Siwiec

Update singularity-it4i.md

parent 8fbc5424
Pipeline #11304 passed with stages
in 23 minutes and 15 seconds
# Singularity on IT4Innovations
On our clusters, the Singularity images of main linux distributions are prepared.
On our clusters, the Singularity images of main Linux distributions are prepared.
```console
Salomon Anselm Barbora
......@@ -16,13 +16,13 @@ On our clusters, the Singularity images of main linux distributions are prepared
```
!!! info
Current information about available Singularity images can be obtained by the `ml av` command. The Images are listed in the `OS` section.
Current information about available Singularity images can be obtained by the `ml av` command. The images are listed in the `OS` section.
The bootstrap scripts, wrappers, features, etc. are located [here][a].
## IT4Innovations Singularity Wrappers
For better user experience with Singularity containers we prepared several wrappers:
For better user experience with Singularity containers, we prepared several wrappers:
* image-exec
* image-mpi
......@@ -30,23 +30,23 @@ For better user experience with Singularity containers we prepared several wrapp
* image-shell
* image-update
Listed wrappers help you to use prepared Singularity images loaded as modules. You can easily load Singularity image like any other module on the cluster by `ml OS/version` command. After the module is loaded for the first time, the prepared image is copied into your home folder and is ready for use. When you load the module next time, the version of image is checked and image update (if exists) is offered. Then you can update your copy of image by the `image-update` command.
Listed wrappers help you to use prepared Singularity images loaded as modules. You can easily load a Singularity image like any other module on the cluster by the `ml OS/version` command. After the module is loaded for the first time, the prepared image is copied into your home folder and is ready for use. When you load the module next time, the version of the image is checked and an image update (if exists) is offered. Then you can update your copy of the image by the `image-update` command.
!!! warning
With image update, all user changes to the image will be overridden.
With an image update, all user changes to the image will be overridden.
The runscript inside the Singularity image can be run by the `image-run` command. This command automatically mounts the `/scratch` and `/apps` storage and invokes the image as writable, so user changes can be made.
Very similar to `image-run` is the `image-exec` command. The only difference is that `image-exec` runs user-defined command instead of the runscript. In this case, the command to be run is specified as a parameter.
Very similar to `image-run` is the `image-exec` command. The only difference is that `image-exec` runs a user-defined command instead of a runscript. In this case, the command to be run is specified as a parameter.
For development is very useful to use interactive shell inside the Singularity container. In this interactive shell you can make any changes to the image you want, but be aware that you can not use the `sudo` privileged commands directly on the cluster. To invoke interactive shell easily just use the `image-shell` command.
Using the interactive shell inside the Singularity container is very useful for development. In this interactive shell, you can make any changes to the image you want, but be aware that you can not use the `sudo` privileged commands directly on the cluster. To simply invoke interactive shell, use the `image-shell` command.
Another useful feature of the Singularity is direct support of OpenMPI. For proper MPI function, you have to install the same version of OpenMPI inside the image as you use on cluster. OpenMPI/3.1.4 is installed in prepared images. The MPI must be started outside the container. The easiest way to start the MPI is to use the `image-mpi` command.
This command has the same parameters as the `mpirun`. Thanks to that, there is no difference between running normal MPI application and MPI application in Singularity container.
Another useful feature of the Singularity is the direct support of OpenMPI. For proper MPI function, you have to install the same version of OpenMPI inside the image as you use on the cluster. OpenMPI/3.1.4 is installed in prepared images. The MPI must be started outside the container. The easiest way to start the MPI is to use the `image-mpi` command.
This command has the same parameters as `mpirun`, so there is no difference between running normal MPI application and MPI application in a Singularity container.
## Examples
In the examples, we will use the prepared Singularity images.
In the examples, we will use prepared Singularity images.
### Load Image
......@@ -73,11 +73,11 @@ CentOS Linux release 7.3.1708 (Core)
**image-mpi**
MPI wrapper - see more in the chapter [Examples MPI][1].
MPI wrapper - see more in the [Examples MPI][1] section.
**image-run**
This command runs the runscript inside the Singularity image. Note, that the prepared images don't contain a runscript.
This command runs the runscript inside the Singularity image. Note, that the prepared images do not contain a runscript.
**image-shell**
......@@ -93,7 +93,7 @@ Singularity CentOS-7_20180220104046.img:~>
### Update Image
This command is for updating your local copy of the Singularity image. The local copy is overridden in this case.
This command is for updating your local Singularity image copy. The local copy is overridden in this case.
```console
$ ml CentOS/6
......@@ -124,10 +124,10 @@ With Singularity, the MPI usage model is to call `mpirun` from outside the conta
$ mpirun -np 24 singularity exec container.img /path/to/contained_mpi_prog
```
By calling `mpirun` outside of the container, we solve several very complicated work-flow aspects. For example, if `mpirun` is called from within the container it must have a method for spawning processes on remote nodes. Historically the SSH is used for this which means that there must be an `sshd` running within the container on the remote nodes, and this `sshd` process must not conflict with the `sshd` running on that host! It is also possible for the resource manager to launch the job and (in OpenMPI’s case) the Orted (Open RTE User-Level Daemon) processes on the remote system, but that then requires resource manager modification and container awareness.
By calling `mpirun` outside of the container, we solve several very complicated work-flow aspects. For example, if `mpirun` is called from within the container, it must have a method for spawning processes on remote nodes. Historically the SSH is used for this, which means that there must be an `sshd` running within the container on the remote nodes and this `sshd` process must not conflict with the `sshd` running on that host. It is also possible for the resource manager to launch the job and (in OpenMPI’s case) the Orted (Open RTE User-Level Daemon) processes on the remote system, but that then requires resource manager modification and container awareness.
In the end, we do not gain anything by calling `mpirun` from within the container except for increasing the complexity levels and possibly losing out on some added
performance benefits (e.g. if a container wasn’t built with the proper OFED as the host).
performance benefits (e.g. if a container was not built with the proper OFED as the host).
#### MPI Inside Singularity Image
......@@ -151,7 +151,7 @@ $ image-mpi hostname | wc -l
48
```
In this case, the MPI wrapper behaves like `mpirun` command. The `mpirun` is called outside the container and the communication between nodes are propagated
In this case, the MPI wrapper behaves like the `mpirun` command. The `mpirun` command is called outside the container and the communication between nodes are propagated
into the container automatically.
## How to Use Own Image on Cluster?
......@@ -167,7 +167,7 @@ local:$ scp container.img login@login4.salomon.it4i.cz:~/.singularity/image/cont
* Use your image
!!! note
If you want to use the Singularity wrappers with your own images, then load module `Singularity-wrappers/master` and set the environment variable `IMAGE_PATH_LOCAL=/path/to/container.img`.
If you want to use the Singularity wrappers with your own images, load the `Singularity-wrappers/master` module and set the environment variable `IMAGE_PATH_LOCAL=/path/to/container.img`.
## How to Edit IT4Innovations Image?
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment