Using Allas object storage service from CSC Supercomputing environment
The disk environments of CSC supercomputers, Puhti and Mahti, are not intended for storing research data that is not actively used for computing. The data that needs to be stored for a longer time than just few weeks, should be copied to Allas object storage service.
Allas provides a platform that you can use to store your data as long as you CSC project is active. In addition to storing, Allas can be used for transporting data between different servers and sharing data with other users.
Getting access to Allas
The default storage quota in Allas is 10 TB. As this space is shared with all project members, it is possible that the space is not sufficient. In that case, you should estimate how much space is needed and request more space. The request should be sent to email@example.com.
In order to use Allas in Puhti or Mahti, first load the module allas:
module load allas
By default the allas-conf activates tools that use Allas with swift protocol. You can alternatively use S3 proptocol too, however in this document we discuss only swift based Allas usage.
Authentication is session specific and valid for eight hours at a time. You can be connected to only one Allas project at a time in one terminal session. However, simutaneous terminal sessions can use different Allas projects. The project you are using in Allas does not need to match the project you are using in Puhti or Mahti. You can refresh the authentication or change the target project at any time my running allas-conf again.
Allas client software options for Puhti and Mahti and other linux servers
The allas module provides several tools that you can use to move data between Allas and the computing server. You can cross-use the Allas clients as long as you access Allas with the same protocol (swift or S3). Below is a list of the Allas clients that are most commonly used in Puhti and Mahti:
- a-commands: (Swift, optionally S3) Easy and safe: a-commands
- rclone: (Swift, optionally S3) Advanced tool: rclone
- swift python client: (Swift) Native Swift client
- s3cmd: (S3) S3 client
More information about using Allas can be found from the Allas documentation:
The Allas documentation includes two tutorials that are especially designed for Puhti and Mahti users: