Allas access is based on CSC's customer projects. To be able to use Allas, you need to be a member of a CSC project that has the permission to use Allas. If you do not have a CSC account, you must first register as a CSC user and join or start a computing project for which Allas has been enabled. This can be done in the MyCSC user portal: https://my.csc.fi.
Once you have Allas enabled you can access it from any machine or server that is connected to internet. This can be a your laptop, supercomputer at CSC, virtual machine in cloud or enven your phone.
Accessing Allas from the web browser
At the moment CSC provides two web browser interfaces for Allas:
The OpenStack Horizon web interface in cPouta provides easy-to-use basic functions for data management in Allas:
SD Connect provides sensitive data oriented web interface for Allas. By default this interface encrypts the uploaded data so that it can be used in SD Desktop service. You can use SD Connect for non-sensitive data too, if you switch off the encryption.
Accessing Allas in the CSC computing environment and other Linux platforms
Puhti and Mahti servers at CSC support many different tools for using Allas, These include
- a-tools for basic use: (Swift, optionally S3)
- rclone providing some advanced functions:** (Swift, optionally S3)
- swift python client that provides wide range of functionalities (Swift)
- s3cmd an S3 client and persistent Allas connections:** (S3)
Note that the tools listed above utilize two different protocols: Swift and S3. Data uploaded using one protocol is not necessary compatible with another protocol.
The software listed above can be used in other Linux servers as well, e.g. a virtual machine running in cPouta or your own Linux-based laptop. In that case, you need to install the client software and configure the connection to Allas yourself. Instructions : allas-cli-utils
In Puhti and Mahti the Allas tools listed above are installed by CSC and provided through allas module. In order to use Allas in Puhti or Mahti, first load the Allas module:
module load allas
Authentication information is stored in the shell variables OS_AUTH_TOKEN and OS_STORAGE_URL and is valid for up to eight hours. However, you can refresh the authentication at any time my running allas-conf again. The environment variables are available only for that login session, so if you start another shell session, you need to authenticate again in there to access Allas.
Once Allas connection is congigured, you can start using Allas with the tools listed above.
Basic Allas operations with different tools.
|Tool||List objects in bucket buck_123||Upload file data1.txt to bucket buck_123||Download file data1.txt from bucket buck_123|
*For s3cmd, open Allas connection with command
allas-conf -m s3cmd
Accessing Allas with Windows or Mac
In addition to the Web interfaces listed above, you can access Allas from you Windows or Mac computer with locally installed client software. For example following tools can be used:
- Cyberduck provides easy to use graphical interface for moving data between local computer and Allas.
- Rclone is an command line tool that provides very effective way to use Allas.
- a-tools these Allas specific commands can be installed in Mac OSX machines but not to Windows
The list above is not complete or exclusive. Any tool that supports Swift or S3 protocols can in principle use Allas.
Copying files directly between object storages
Rclone can also be used to directly copy files from another object storage (e.g. Amazon S3, Google cloud, CREODIAS,...) to Allas. For this both credentials need to be stored in a Rclone configuration file in the users home directory (
.config/rclone/rclone.conf). An example is shown below:
[s3allas] type = s3 provider = Other env_auth = false access_key_id = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx endpoint = a3s.fi acl = private [otherobjectstorage] type = s3 provider = Other env_auth = false access_key_id = yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy secret_access_key = yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy endpoint = yourotherendpoint.com acl = private
source allas_conf --mode s3cmd .
After creating/updating this file, Rclone can be used to copy files
rclone copy otherobjectstorage:bucket-x/object-y s3allas:bucket-z/object-a
or list files from either Allas or the other object storage by using the respective name
rclone lsf otherobjectstorage:.
Other ways of accessing Allas
- Python Programmatic access to Allas.
- Geoscience related examples how Allas can be used in Python scripts
- Nextcloud front end Can be set up in Pouta to get additional functionality.
- R (Geoscience related example how Allas can be used in R scripts)