Skip to content

Using Allas with Python

CSC's instructions for installing Python and which libraries are required:
https://research.csc.fi/pouta-install-client.

Download the OpenStack RC File v3 as instructed in the last chapter 3.4.1.3 Configure your terminal environment for OpenStack.

The Python pip libraries required in the examples:
python-keystoneclient and python-swiftclient.

This page includes Python scripts for the following operations:

Function
Create a connection to server
Create a bucket
Upload an object
List buckets and objects
Download an object
Remove buckets and objects

Create a connection

This Python script creates a connection to the server:

from keystoneauth1 import session
from keystoneauth1.identity import v3
import swiftclient
import os

_authurl = os.environ['OS_AUTH_URL']
_auth_version = os.environ['OS_IDENTITY_API_VERSION']
_user = os.environ['OS_USERNAME']
_key = os.environ['OS_PASSWORD']
_os_options = {
    'user_domain_name': os.environ['OS_USER_DOMAIN_NAME'],
    'project_domain_name': os.environ['OS_USER_DOMAIN_NAME'],
    'project_name': os.environ['OS_PROJECT_NAME']
}

conn = swiftclient.Connection(
    authurl=_authurl,
    user=_user,
    key=_key,
    os_options=_os_options,
    auth_version=_auth_version
)

Alternatively, you can enter the information directly in the script from the downloaded RC file:

import swiftclient

_authurl = 'https://pouta.csc.fi:5001/v3'
_auth_version = '3'
_user = 'John'
_key = 'John1234'
_os_options = {
    'user_domain_name': 'Default',
    'project_domain_name': 'Default',
    'project_name': 'project_123456'
}

conn = swiftclient.Connection(
    authurl=_authurl,
    user=_user,
    key=_key,
    os_options=_os_options,
    auth_version=_auth_version
)

In the above example:

OS_AUTH_URL = _authurl = https://pouta.csc.fi:5001/v3
OS_IDENTITY_API_VERSION = _auth_version = 3
OS_USERNAME = _user = John
OS_PASSWORD = _key = John1234
OS_PROJECT_NAME = project_name = project_123456

Further information of the Keystone authentication:
https://docs.openstack.org/python-swiftclient/newton/client-api.html

Create a bucket

Create a new bucket using the following script:

bucket_name='snakebucket'
conn.put_container(bucket_name)

Upload an object

Upload an object called my_snake.txt to the bucket snakebucket:

object_name='my_snake.txt'
with open(object_name, 'r') as f:
    conn.put_object(bucket_name, object_name,
                    contents=f.read(),
                    content_type='text/plain')

List buckets and objects

List all buckets belonging to a project:

resp_headers, containers = conn.get_account()

for container in containers:
   print(container)

And all objects belonging to a bucket:

for info in conn.get_container('snakebucket')[1]:
    print('{0}\t{1}\t{2}'.format(info['name'], info['bytes'], info['last_modified']))

Download an object

Download an object:

my_obj = conn.get_object(bucket_name, object_name)[1]
with open('new_name_for_file.txt', 'w') as f:
    f.write(my_obj)

Please note: If you get the error

TypeError: write() argument must be str, not bytes

open the file in the binary mode

with open('new_name_for_file.txt', 'bw') as f:
    f.write(my_obj)

instead of the text mode.

Remove buckets and objects

Delete a bucket:

conn.delete_container(bucket_name)

Note: Only empty buckets can be removed.

Remove an object:

conn.delete_object(bucket_name, 'my_snake.txt')

Last edited Mon Jan 20 2020