tardis.apps.s3utils package¶
Subpackages¶
Submodules¶
tardis.apps.s3utils.api module¶
Additions to MyTardis’s REST API
-
class
tardis.apps.s3utils.api.
ReplicaAppResource
(api_name=None)¶ Bases:
tardis.tardis_portal.api.ReplicaResource
Extends MyTardis’s API for DFOs, adding in a download method for S3 objects
-
class
Meta
¶ Bases:
tardis.tardis_portal.api.Meta
Authorisation class for Tastypie.
-
queryset
¶
-
resource_name
= 'replica'¶
-
base_fields
= {'created_time': <tastypie.fields.DateTimeField object>, 'datafile': <tastypie.fields.ForeignKey object>, 'id': <tastypie.fields.IntegerField object>, 'last_verified_time': <tastypie.fields.DateTimeField object>, 'resource_uri': <tastypie.fields.CharField object>, 'uri': <tastypie.fields.CharField object>, 'verified': <tastypie.fields.BooleanField object>}¶
-
declared_fields
= {}¶
-
download_dfo
(request, **kwargs)¶ Download DataFileObject from S3 Object Store
-
prepend_urls
()¶ A hook for adding your own URLs or matching before the default URLs.
-
class
tardis.apps.s3utils.apps module¶
tardis.apps.s3utils.default_settings module¶
Default settings for s3utils app
-
tardis.apps.s3utils.default_settings.
S3_SIGNED_URL_EXPIRY
= 30¶ A short expiry (30 seconds) is used, because it is only intended to provide access long enough for an authenticated MyTardis user to be redirected to the signed URL.
tardis.apps.s3utils.urls module¶
Minimal urls.py, so we can do a reverse lookup for the ‘s3_api_download_dfo’ URL pattern.
‘s3_api_download_dfo’ is defined in the prepend_urls method of the ReplicaAppResource class in api.py
The API endpoint defined in this app is mapped to a URL in tardis/urls/api.py (along with API endpoints defined by other MyTardis apps).
tardis.apps.s3utils.utils module¶
Utilities for S3 objects
-
tardis.apps.s3utils.utils.
calculate_checksums
(dfo, compute_md5=True, compute_sha512=False)¶ Calculates checksums for an S3 DataFileObject instance. For files in S3, using the django-storages abstraction is inefficient - we end up with a clash of chunking algorithms between the download from S3 and MyTardis’s Python-based checksum calculation. So for S3 files, we calculate checksums using external binaries (md5sum and shasum) instead.
:param dfo : The DataFileObject instance :type dfo: DataFileObject :param compute_md5: whether to compute md5 default=True :type compute_md5: bool :param compute_sha512: whether to compute sha512, default=True :type compute_sha512: bool
Returns: the checksums as {‘md5sum’: result, ‘sha512sum’: result} Return type: dict
-
tardis.apps.s3utils.utils.
generate_presigned_url
(dfo, expiry=None)¶ Generate a presigned URL for an S3 object
boto3 must be installed if you are using the storages.backends.s3boto3.S3Boto3Storage storage class, defined in the django-storages package
- dfo : DataFileObject
- The DataFileObject to generate the pre-signed URL for
- expiry : int
- The signed URL’s expiry in seconds
- string
- The pre-signed URL