Building Apptainer containers from scratch
On CSC platforms, there are two ways to use Apptainer containers (formerly known as Singularity) that you have built yourself. You may:
- Convert Docker containers to Apptainer image files, or
- Build an Apptainer image entirely from scratch.
This tutorial focuses on the latter scenario, with further information available on the Apptainer website.
Containers as distinct environments
One can think of containers as distinct environments of their own. For example, the Linux distribution used in the container may differ from that of the host environment where the container is executed. Apptainer images also have an internal file system that is not visible outside of the container. Likewise, most of the host environment will not be visible to a container unless explicitly specified. Selected folders can be exposed to the container by using bind mounts.
When planning the contents and configuration of an Apptainer container, a decision needs to be made on whether specific installations should be done within the container or added as bind mounts. For example, certain dependencies may already be present on the host. The container Linux distribution will naturally affect this decision.
Frequently the environment variables used within a container versus the host environment will also differ (more on that below).
Preparing a definition file
A definition file (with the extension .def) is a file containing the commands required to build the container. Examples of definition files used on CSC computing environments can be found in our singularity-recipes repository.
A definition file is split into sections serving different purposes (see here for details). At the very top of the file, one must provide a header specifying a bootstrap keyword and a Linux distribution. For example, to create a header for a CentOS 7.7 container, one would type:
Bootstrap: library From: centos:7.7
The choice of Linux distribution defines the types of commands used in the definition file. For example, a CentOS container would use commands such as
yum -y install, whereas an Ubuntu container would use commands such as
apt-get -y install.
Other sections one can use include the following. Note that different sections can be added or left out, depending on your needs (e.g. if you have no external files to include in the container, there is no need for a
%labels(e.g. name and contact of the image maintainer)
%files(commands for copying external files into a specific location in the container)
%post(section for software installation commands)
%environment(environment variables defined upon launching the container)
A few words on environment variables
Environment variables can be defined in both the
%environment sections. Those specified in the former may be useful during software installations, but unlike variables specified under the
%environment section, will not be visible upon launching the container.
If you wish to specify environment variables on the host environment in a way that is visible to the container, these must be preceded by
APPTAINERENV_. Variables without this prefix will not be passed onto the container.
Building a container
Building an Apptainer container requires root (
sudo) access. As such, this cannot be directly done on CSC's supercomputers. Instead you must use your own computer, or a virtual computer where you have root access, to build the container image.
One option is to use a Pouta virtual machine. This enables one to specify the type of environment used to build the image. Most of the time, the Linux distribution of the environment used to build the container will matter relatively little. However, certain installations (e.g. NVIDIA CUDA drivers) have strict version requirements with regard to kernel headers, meaning that the distribution must effectively be matched between the environment for container building, the container and the host.
To build a container using a definition file, one can simply run:
sudo apptainer build container.sif recipe.def
Note: in older versions of Linux the command may be
singularity instead of
If everything is successful, this results in a container image (.sif file) that can be moved onto your computing environment of choice. The image file is immutable, meaning that it can not be modified after the build step. It is also possible to create writable sandbox directories that can be used for testing (and can be subsequently converted into Apptainer images).
Running a container
There are a couple of ways to run Apptainer containers on CSC computing environments. For detailed instructions, visit our documentation on running containers.
The options include using:
apptainer_wrapper exec(wrapper script binding local folders, e.g.
Many of the containers available on CSC environments come with wrappers for selected commands, enabling them to be executed without a need for the above. It is also possible to prepare wrappers for containers built from scratch, although this requires some familiarity with writing bash scripts.
Moving containers onto CSC computing environments
Puhti and Mahti
Apptainer image files can be transferred onto Puhti or Mahti using
rsync or a graphical file transfer tool.