tardis.apps.push_to package

Submodules

tardis.apps.push_to.apps module

class tardis.apps.push_to.apps.PushToConfig(app_name, app_module)

Bases: tardis.app_config.AbstractTardisAppConfig

name = 'tardis.apps.push_to'
verbose_name = 'Push To'

tardis.apps.push_to.exceptions module

exception tardis.apps.push_to.exceptions.NoSuitableCredential

Bases: Exception

This exception is thrown when attempting to find a suitable credential for a remote host but none is found

tardis.apps.push_to.models module

class tardis.apps.push_to.models.Credential(*args, **kwargs)

Bases: tardis.apps.push_to.models.KeyPair

A credential that may contain a password and/or key. The auth method chosen depends on the credentials available, allowed auth methods, and priorities defined by the SSH client.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

static generate_keypair_credential(tardis_user, remote_user, remote_hosts, bit_length=2048)

Generates and saves an RSA key pair credential. Credentials returned by this method are intended to be registered on remote systems before being used. @type tardis_user: User @type remote_user: str @type bit_length: int @type remote_hosts: list[RemoteHost] :return: the generated credential :rtype: object

get_client_for_host(remote_host)

Attempts to establish a connection with the remote_host using this credential object. The remote_host may be any host, but only those in the remote_hosts field are expected to work. @type remote_host: .RemoteHost :return: a connected SSH client :rtype: SSHClient

static get_suitable_credential(tardis_user, remote_host, remote_user=None)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
password

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

remote_hosts

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

remote_user

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

request_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

user

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

user_id
verify_remote_access(remote_host=None)

@type remote_host: RemoteHost

class tardis.apps.push_to.models.CredentialAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

form

alias of CredentialForm

media
class tardis.apps.push_to.models.CredentialForm(data=None, files=None, auto_id='id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None, use_required_attribute=None, renderer=None)

Bases: django.forms.models.ModelForm

class Meta

Bases: object

fields = '__all__'
model

alias of Credential

widgets = {'password': <django.forms.widgets.PasswordInput object>}
base_fields = {'key_type': <django.forms.fields.CharField object>, 'password': <django.forms.fields.CharField object>, 'private_key': <django.forms.fields.CharField object>, 'public_key': <django.forms.fields.CharField object>, 'remote_hosts': <django.forms.models.ModelMultipleChoiceField object>, 'remote_user': <django.forms.fields.CharField object>, 'user': <django.forms.models.ModelChoiceField object>}
declared_fields = {}
media
class tardis.apps.push_to.models.DBHostKeyPolicy

Bases: paramiko.client.MissingHostKeyPolicy

Host key verification policy based on the host key stored in the database.

missing_host_key(client, hostname, key)

@type key: PKey

class tardis.apps.push_to.models.KeyPair(*args, **kwargs)

Bases: django.db.models.base.Model

A key pair

class Meta

Bases: object

abstract = False
key
Returns:a subclass of PKey of the appropriate key type
Return type:PKey
Raises:ValidationError –
key_type

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

private_key

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

public_key

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

save(*args, **kwargs)

Save the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

class tardis.apps.push_to.models.OAuthSSHCertSigningService(*args, **kwargs)

Bases: django.db.models.base.Model

Connection parameters for an OAuth2 SSH certificate signing service. Supports certificate signing server available here: https://github.com/monash-merc/ssh-authz

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

allow_for_all

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

allowed_groups

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

allowed_remote_hosts

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

allowed_users

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

cert_signing_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

static get_available_signing_services(user)

Gets all SSH cert signing services available for a given user :param User user: User :return: allowed signing services :rtype: User

static get_oauth_service(user, service_id)

@type user: User @type service_id: int

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

nickname

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

oauth_authorize_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

oauth_check_token_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

oauth_client_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

oauth_client_secret

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

oauth_token_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
class tardis.apps.push_to.models.Progress(*args, **kwargs)

Bases: django.db.models.base.Model

Files copy progress

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

datafile

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

datafile_id
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

message

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
request

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

request_id
retry

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

timestamp

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class tardis.apps.push_to.models.RemoteHost(*args, **kwargs)

Bases: tardis.apps.push_to.models.KeyPair

A remote host that may be connected to via SSH

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

administrator

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

administrator_id
credential_set

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

host_name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

logo_img

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

nickname

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

oauthsshcertsigningservice_set

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

objects = <django.db.models.manager.Manager object>
port

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

request_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

class tardis.apps.push_to.models.RemoteHostAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

Hides the private key field, which is not necessary for host keys

fields = ['nickname', 'administrator', 'host_name', 'port', 'key_type', 'public_key', 'logo_img']
media
class tardis.apps.push_to.models.Request(*args, **kwargs)

Bases: django.db.models.base.Model

PushTo request

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

base_dir

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

credential

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

credential_id
get_next_by_timestamp(*, field=<django.db.models.fields.DateTimeField: timestamp>, is_next=True, **kwargs)
get_previous_by_timestamp(*, field=<django.db.models.fields.DateTimeField: timestamp>, is_next=False, **kwargs)
host

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

host_id
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

message

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

object_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

object_type

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
progress_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

timestamp

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

user

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

user_id

tardis.apps.push_to.oauth_tokens module

tardis.apps.push_to.oauth_tokens.get_token(request, oauth_service)

Returns the OAuth2 token from the current session :param Request request: django session object :param OAuthSSHCertSigningService oauth_service: an

OAuthSSHCertSigningService object
Returns:
Return type:string
tardis.apps.push_to.oauth_tokens.get_token_data(oauth_service, token)

Gets the OAuth2 user attributes using the supplied token :param OAuthSSHCertSigningService oauth_service: an

OAuthSSHCertSigningService object
Parameters:token (basestring) – an OAuth2 token
Returns:a json object of user attributes
Return type:dict
tardis.apps.push_to.oauth_tokens.set_token(request, oauth_service, token)

Stores the OAuth2 token in the current session :param Request request: django request object :param OAuthSSHCertSigningService oauth_service: an

OAuthSSHCertSigningService object
Parameters:token (basestring) – the OAuth2 token

tardis.apps.push_to.ssh_authz module

tardis.apps.push_to.ssh_authz.sign_certificate(credential, token, url)

An interface to the OAuth2 SSH certificate signing service @type credential: models.Credential

tardis.apps.push_to.tasks module

tardis.apps.push_to.tasks.complete_request(request_id)
tardis.apps.push_to.tasks.make_dirs(sftp_client, dir_list)

tardis.apps.push_to.urls module

tardis.apps.push_to.utils module

tardis.apps.push_to.utils.can_copy(sftp, object_type, object_id, path)
tardis.apps.push_to.utils.get_default_push_location(sftp_client)
tardis.apps.push_to.utils.is_directory(sftp, path)
tardis.apps.push_to.utils.list_subdirectories(sftp_client, path, show_hidden=False)
tardis.apps.push_to.utils.shell_escape(s)

tardis.apps.push_to.views module

tardis.apps.push_to.views.authorize_remote_access(request, remote_host_id, service_id=None)

Generates an SSH certificate using an OAuth2 SSH signing service :param Request request: request object :param basestring remote_host_id: remote host id :param basestring service_id: OAuth2 SSH certificate signing service id :return: an error message or OAuth2 redirects :rtype: HttpRedirect

tardis.apps.push_to.views.get_accessible_hosts(request, obj_type=None, push_obj_id=None)

Retrieves all accessible hosts (i.e. hosts for which the user already has credentials for) including push-to trigger URLs if the object type and id are supplied :param Request request: request object :param object obj_type: data type to be copied

(experiment, dataset or datafile)
Parameters:push_obj_id (int) – the database object id
Returns:json object with accessible hosts
Return type:HttpResponse
tardis.apps.push_to.views.get_credential(request, remote_host)

Fetches a suitable credential for the remote host, or raises an exception if none found :param Request request: request object :param RemoteHost remote_host: the RemoteHost for which a credential

should be found
Returns:the credential
Return type:object
Raises:NoSuitableCredential – raised when no credential is found
tardis.apps.push_to.views.get_push_url_for_host(remote_host, obj_type, push_obj_id)

Constructs a push-to URL to trigger data transfer :param RemoteHost remote_host: the RemoteHost to which data should be copied :param obj_type: data type to be copied (experiment, dataset or datafile) :type obj_type: object :param int push_obj_id: the database object id :return: a push-to URL :rtype: basestring

tardis.apps.push_to.views.get_signing_services(request, obj_type=None, push_obj_id=None)

Retrieves all certificate signing services and associated hosts including push-to trigger URLs if the object type and id are supplied :param Request request: request object :param class obj_type: data type to be copied (experiment, dataset or datafile) :param int push_obj_id: the database object id :return: json object with signing services and hosts :rtype: HttpResponse

tardis.apps.push_to.views.oauth_callback(request)

OAuth2 callback endpoint to continue the SSH certificate signing process :param Request request: request object :return: error message or redirect to the signing service with access token :rtype: HttpResponse

tardis.apps.push_to.views.oauth_callback_url(request)

Builds the oauth callback URL :param Request request: request object :return: callback URL :rtype: basestring

tardis.apps.push_to.views.render_error_message(request, message, status=500)
tardis.apps.push_to.views.render_success_message(request, message, status=200)
tardis.apps.push_to.views.validate_remote_path(request, remote_host_id)
tardis.apps.push_to.views.verify_redirect(request, redirect_url)

Module contents