Skip to content

Utilizing Apptainer containers in SD Desktop

Warning-label

Skill level - Advanced

As the SD Desktop is not directly connected internet, you can't use tools like Git, Conda or Pip to install new software there. Instead, you can use Apptainer software container tool to add new software to your SD Desktop environment. However, you have to first build or download an Apptainer container elsewhere, and then use Allas/SD Connect to import the container to SD Desktop.

Note: Apptainer is a fork on Singularity container system, so in many occasions instructions may refer to Singularity. In most cases you can just replace "Singularity" with "Apptainer".

If you have root access to a machine with Apptainer, you can build your own container that contains exactly the software and datasets you need. Many software are also available as ready-made Apptainer containers or as Docker containers that can be converted into Appainer containers. In this document we show how to import a ready-made Apptaner container from a public repository to SD Desktop.

Step by step tutorial

Before you start, please activate services Puhti, SD Desktop and Allas/SD Connect for your project. This happens in MyCSC.

How to import a ready-made Apptainer container from a public repository to SD Desktop:

  1. Find a suitable container
  2. Download the container
  3. Upload the container to SD Connect
  4. Download the container to SD Desktop

Importing ready-made container through Puhti

Find a suitable container

In the example below we import BETA Binding and Expression Target Analysis software to SD Desktop. This tool is available as a ready-made Apptainer container in Biocontainers repository. You can find the tool by searching for Binding and Expression Target Analysis in the repository. When you open the detailed information of the resulting cistrome_beta container, you can see that the Singularity module can be downloaded from URL: https://depot.galaxyproject.org/singularity/cistrome_beta:1.0.7--py27heb79e2c_4

Download the container

As we don't need to build the container from scratch, we can use puhti.csc.fi server to download the container image and push it to Allas.

First login to puhti.csc.fi. Then, start an interactive batch job session with command:

sinteractive

In the interactive session move to LOCAL_SCRATCH directory and set some Singularity related environment variables:

export SINGULARITY_TMPDIR=$LOCAL_SCRATCH
export SINGULARITY_CACHEDIR=$LOCAL_SCRATCH
unset XDG_RUNTIME_DIR

Then download a local copy of the Beta container with command

apptainer pull beta.sif https://depot.galaxyproject.org/singularity/cistrome_beta:1.0.7--py27heb79e2c_4

This creates a new singularity container file, beta.sif. From the home page of BETA software we download also a test data set for confirming that the container works.

wget http://cistrome.org/BETA/src/BETA_test_data.zip

Upload the container to Allas / SD Connect

Then we upload these two files to Allas/SD Connect. In this example we use project 2012345.

module load allas
allas-conf project_2012345
a-put --sdx beta.sif -b 2012345_beta
a-put --sdx BETA_test_data.zip -b 2012345_beta

The commands above store the files into bucket 2012345_beta in Allas. a-put is used with option --sdx in order to encrypt the uploaded data with SD Desktop compatible encryption.

Using a container in SD Desktop

First login to sd-desktop.csc.fi and open your virtual desktop session.

Once the .sif formatted Apptainer container file and the sample data has been uploaded to Allas, we can copy them to SD Desktop. To do this open DataGateway, in your session SD Desktop. After that copy the data to local disk in SD Desktop.

You can do that using the graphical DataGateway tool on the Desktop (see the video).

Or you can use Linux command line: Open a Linux terminal in the SD-Desktop. In the terminal, move the Apptainer file and test data to your current locations:

cp Projects/SD-connect/project_201234/2012345_beta/beta.sif ./
cp Projects/SD-connect/project_201234/2012345_beta/BETA_test_data.zip ./

Unzip the test dataset:

unzip BETA_test_data.zip

Now you can run BETA through apptainser command. For example the help of command BETA minus is shown with command:

apptainer exec beta.sif BETA minus -h

And the analysis with sample data in directory BETA_test_data can be executed with commands like:

apptainer exec beta.sif BETA minus -p BETA_test_data/3656_peaks.bed --bl -g hg19

In this example the results will be written to directory BETA_OUTPUT.

Last update: March 28, 2024