Skip to content

How to move data from Puhti to Allas and vice versa?

There are several options for moving data between Puhti and Allas. This page summarizes how to move data using a-commands, Swift, Rclone and S3cmd clients.

All the required packages and software needed for the clients are already installed on Puhti. To activate Allas and authenticate to a project you need to run the commands:

module load allas
allas-conf

Move data with a-commands

a-commands are easy-to-use tools for basic usage of Allas. The main commands to use for moving data between Puhti and Allas are:

  • a-put: Move data from Puhti to Allas
  • a-get: Move data from Allas to Puhti

Move data with Swift

The Swift client provides the options upload and download for moving data:

swift upload <bucket name> <file name>

swift download <bucket name> <file name>

For more information, see Swift client.

Move data with Rclone

Rclone is another client using which you can move data between Puhti and Allas. For example, you could create a bucket called 2000620-raw-data in Allas using the command:

rclone mkdir allas:2000620-raw-data

Uploading a file called file.dat to that bucket can be done using the rclone copy command:

rclone copy file.dat allas:2000620-raw-data/

Downloading the file back to Puhti is done with the same rclone copy command:

rclone copy allas:2000620-raw-data/file.dat .

Note

Another destination directory can be specified in the rclone copy command as well. If this directory does not exist, Rclone will create it for you.

rclone copy allas:2000620-raw-data/file.dat my-new-folder

For more information, see Using Allas with Rclone from Puhti.

Move data with S3cmd

For moving data between Puhti and Allas, S3cmd provides the functions:

For more information, see S3 client.