Skip to content

Using Allas from LUMI supercomputer

At the moment the LUMI object storage service, LUMI-O, is not yet in use and tools for using object storage services are not by default installed in LUMI-C. In this document we describe how you can install commonly used object storage clients to LUMI-C and how to configure connection to Allas object storage services.

Installing Allas tools

In this example we do the installation to the /project area of a project so that all project members can utilize these tools. In this example we use example project 462000007. Please use your own project number when installing these tools for your own project.

Once you have logged in to Lumi, move to the /project disk area of the project and make there a directory called allas.

cd /project/project_462000007
mkdir allas
cd allas
Then we clone allas-cli-utils repository to this location and continue to the cloned directory:

git clone https://github.com/CSCfi/allas-cli-utils  
cd allas-cli-utils

Allas-cli-utils has a list of software dependencies that needs to be be installed. We use here the lumi-container-wrapper tool to install most of them (s3cmd, Python OpenStack clinet, Python swift client, zstdmt and crypt4gh). The installation is done with commands:

module load LUMI lumi-container-wrapper
conda-containerize new --prefix /project/project_462000007/allas allas-dependencies.yaml
The command above will install the dependencies to a singularity container and make command-like execution scripts to a bin directory that locates in the directory defined with the --prefix option. (in this case /project/project_462000007/allas/bin).

Finally we need to modify a_env_conf file a bit so that a-tools works smoothly. Open the file

nano allas-cli-utils/a_env_conf

In this file do following modifications (remember to use your own project number in stead of 462000007)

local_host=”lumi”
allas_conf_path="/project/project_462000007/allas/allas-cli-utils/allas_conf"
tmp_root="/scratch/project_462000007"

Now the installation is ready. In the future you and your group members need to only run the connection configuration commands below, to enable object storage tools and to open connection to Allas.

Opening connection from Lumi to Allas

When you want to use these object storage tools, add the installation directories to your command path (remember here too to replace the project number with your own project).

export PATH=/project/project_462000007/allas/allas-cli-utils:/project/project_462000007/allas/bin:$PATH

Then open Allas connection with command:

source /project/project_462000007/allas/allas-cli-utils/allas_conf

Now commands like rclone, swift or a-put should work in the same way as in Puhti and Mahti.

To simplify the configuration process you can write the two command above into one file and run source command just for that file. For example if you would have the commands in a file called: open-allas in your home directory you could open the connection with command:

source  $HOME/open_allas

Last edited Fri Apr 1 2022