tardis.apps.s3utils package

Submodules

tardis.apps.s3utils.api module

Additions to MyTardis’s REST API

class tardis.apps.s3utils.api.ReplicaAppResource(api_name=None)

Bases: tardis.tardis_portal.api.ReplicaResource

Extends MyTardis’s API for DFOs, adding in a download method for S3 objects

class Meta

Bases: tardis.tardis_portal.api.Meta

authorization

Authorisation class for Tastypie.

queryset
resource_name = 'replica'
base_fields = {'created_time': <tastypie.fields.DateTimeField object>, 'datafile': <tastypie.fields.ForeignKey object>, 'id': <tastypie.fields.IntegerField object>, 'last_verified_time': <tastypie.fields.DateTimeField object>, 'resource_uri': <tastypie.fields.CharField object>, 'uri': <tastypie.fields.CharField object>, 'verified': <tastypie.fields.BooleanField object>}
declared_fields = {}
download_dfo(request, **kwargs)

Download DataFileObject from S3 Object Store

prepend_urls()

A hook for adding your own URLs or matching before the default URLs.

tardis.apps.s3utils.apps module

class tardis.apps.s3utils.apps.S3UtilsConfig(app_name, app_module)

Bases: django.apps.config.AppConfig

name = 'tardis.apps.s3utils'

tardis.apps.s3utils.default_settings module

Default settings for s3utils app

tardis.apps.s3utils.default_settings.S3_SIGNED_URL_EXPIRY = 30

A short expiry (30 seconds) is used, because it is only intended to provide access long enough for an authenticated MyTardis user to be redirected to the signed URL.

tardis.apps.s3utils.urls module

Minimal urls.py, so we can do a reverse lookup for the ‘s3_api_download_dfo’ URL pattern.

‘s3_api_download_dfo’ is defined in the prepend_urls method of the ReplicaAppResource class in api.py

The API endpoint defined in this app is mapped to a URL in tardis/urls/api.py (along with API endpoints defined by other MyTardis apps).

tardis.apps.s3utils.utils module

Utilities for S3 objects

tardis.apps.s3utils.utils.calculate_checksums(dfo, compute_md5=True, compute_sha512=False)

Calculates checksums for an S3 DataFileObject instance. For files in S3, using the django-storages abstraction is inefficient - we end up with a clash of chunking algorithms between the download from S3 and MyTardis’s Python-based checksum calculation. So for S3 files, we calculate checksums using external binaries (md5sum and shasum) instead.

:param dfo : The DataFileObject instance :type dfo: DataFileObject :param compute_md5: whether to compute md5 default=True :type compute_md5: bool :param compute_sha512: whether to compute sha512, default=True :type compute_sha512: bool

Returns:the checksums as {‘md5sum’: result, ‘sha512sum’: result}
Return type:dict
tardis.apps.s3utils.utils.generate_presigned_url(dfo, expiry=None)

Generate a presigned URL for an S3 object

boto3 must be installed if you are using the storages.backends.s3boto3.S3Boto3Storage storage class, defined in the django-storages package

dfo : DataFileObject
The DataFileObject to generate the pre-signed URL for
expiry : int
The signed URL’s expiry in seconds
string
The pre-signed URL

Module contents