LUMI

LUMI uses SingularityCE instead of Apptainer. We recommend to read the instructions for Apptainer containers which apply to SingularityCE on LUMI with certain modifications explained in this page. We use the command singularity instead of apptainer.

Running containers

We can run commands from a container as follows:

singularity exec container.sif mycommand

On LUMI, we can bind mount common disk areas to the container as follows:

singularity exec --bind="/pfs,/users,/projappl,/scratch,/project,/flash" container.sif mycommand

We can also enable ROCm support for AMD GPUs on LUMI as follows:

singularity exec --rocm container.sif mycommand

Building containers

Temporary directory

On LUMI, we use /tmp as the temporary directory. SingularityCE bind mounts it by default to the build environment. Therefore, manually bind mounting the temporary directory is not required.

Cache directory

The SingularityCE cache directory can be changed if needed:

export SINGULARITY_CACHEDIR=/scratch/project_id/$USER/.singularity

Build location

On the login node, we can build container images that are small enough that they do not run into memory limits. Virtual memory in LUMI is quite large (64 GB) and preset to the hard limit, thus it is not required to adjust it.

We must build large container images on the compute node via a slurm job. For example, we can reserve an interactive slurm job as follows, just replace myproject with your project:

srun --account myproject --partition small --time 0:15:00 --mem 8000 --cpus-per-task 1 --pty bash

On the compute node /tmp is a tmpfs which is limited by memory. We must request memory that is at least twice the size of the uncompressed size of your container image (SIF file) to avoid running out of memory.

Building an SIF image from definition file

We can write SingularityCE definition files using the .def file extension. Here is a simple example of container definition:

container.def

Bootstrap: docker
From: docker.io/opensuse/leap:15.5

%post
    # Replace the failing commands with always succeeding dummies.
    cp /usr/bin/true /usr/sbin/useradd
    cp /usr/bin/true /usr/sbin/groupadd

    # Continue to install software into the container normally.
    zypper --non-interactive update

On LUMI, we need to use proot to build SIF images with SingularityCE. We can load proot as follows:

module load LUMI systools

Then, we can build container images in the standard way as follows:

singularity build container.sif container.def

Do not use the --fakeroot flag with SingularityCE on LUMI, as it does not behave in the same way as with Apptainer.