Creating Singularity containers
CSC's supercomputers Puhti and Mahti support running Singularity containers. If you wish to run a container-based application, first check the application pages to see if a pre-installed container is already available. Also see our documentation on how to run Singularity containers.
If you cannot find a pre-built container, one option is to build your own. If a Docker container image already exists, you can often simply convert that to a Singularity container. Another option is to build your own container from scratch. Both approaches will be discussed below. As always, if you have any problems or questions, don't hesitate to contact CSC's Service Desk.
Converting a Docker container
If you already have an existing Docker container, in many cases it can easily be converted to a Singularity image. Docker container images can be found in public repositories such as Docker Hub, but please take care to only use images uploaded from reputable sources as these images can easily be a source of security vulnerabilities or even contain malicious code.
GPU-optimized containers can also be found in NVIDIA's GPU cloud (NGC). These containers have been prepared by NVIDIA, and should thus be safe.
Here is an example of how to build a Singularity image from NVIDIA's PyTorch Docker image. We'll use
sinteractive as heavy processing should not be done in the login nodes.
# Let's start a 1 hour interactive job sinteractive --account <project> --time 1:00:00 # Let's use the fast local drive for temporary storage export SINGULARITY_TMPDIR=$LOCAL_SCRATCH export SINGULARITY_CACHEDIR=$LOCAL_SCRATCH # This is just to avoid some annoying warnings unset XDG_RUNTIME_DIR # Change directory to where you wish to store the image cd /projappl/<project> # Do the actual conversion # NOTE: the Docker image is downloaded straight from NGC singularity build pytorch_21.03-py3.sif docker://nvcr.io/nvidia/pytorch:21.03-py3
Note that the Singularity image
.sif files can easily be several GB in size, so they should not be stored in your home directory, but for example in the project application directory projappl.
Also see our documentation on how to run Singularity containers.
Build a container from scratch
You can also build your own Singularity container from scratch. This is an option for more experienced users, and your main source of information is the official Singularity documentation on building containers.
You can find some help also by looking at some of the Singularity recipes used by CSC's staff to build the containers provided on our supercomputers.
Building a container requires root (superuser) access and thus cannot be done on CSC's supercomputers. Instead, you can use your own computer or for example a Pouta virtual machine to build the Singularity container image and then copy the resulting file to Puhti or Mahti.
Last edited Thu Apr 1 2021