cloudmesh-storage.cloudmesh.storage.queue.StorageQueue

Module Contents

Classes

StorageQueue

This class specifies a storage object queue, that allows the queuing of
class cloudmesh-storage.cloudmesh.storage.queue.StorageQueue.StorageQueue(source, destination, name='local', parallelism=4)

This class specifies a storage object queue, that allows the queuing of files to be copied between services.

The queue has a maximal parallelism that can be set to execute the copy in multiple threads.

Please note that actions only add modify the queue in the db, however, the run command executes them one by one.

It will be up to the method to guarantee order. For example, in case of a recursive copy it would make sense to create directories first.

status = ['completed', 'waiting', 'inprogress', 'canceled']
redgister_actions(self, put=None, get=None, delete=None, mkdir=None, list=None, cancel=None)
_copy_file(self, sourcefile, destinationfile)

adds a copy action to the queue

copies the file from the source service to the destination service using the file located in the path and storing it into the remote. If remote is not specified path is used for it.

The copy will not be performed if the files are the same.

Parameters

  • sourcefile –

  • destinationfile –

Returns

copy_file(self, sourcefile, destinationfile)

adds a copy action to the queue

copies the file from the source service to the destination service using the file located in the path and storing it into the remote. If remote is not specified path is used for it.

The copy will not be performed if the files are the same.

Parameters

  • sourcefile –

  • destinationfile –

Returns

copy_tree(self, sourcetree, destinationtree)

adds a tree to be copied to the queue it will recursively add all files within the tree

Parameters

  • sourcetree –

  • destinationtree –

Returns

sync(self, sourcetree, destinationtree)

just a more convenient name for copy_tree :param sourcetree: :param destinationtree: :return:

mkdir(self, service, path)

adds a mkdir action to the queue

create the directory in the storage service :param service: service must be either source or destination :param path: :return:

delete(self, service, path)

adds a deleta action to the queue

Parameters

  • service – service must be either source or destination

  • path –

Returns

delete(self, path, recursive=True)

adds a delete action to the queue

Parameters

path –

Returns

abstract status(self)

provides that status of the queue

{

“length”: 100, “completed”: 10, “waiting”: 80, “inprogress”: 10

}

Returns

cancel(self, id=None)

cancels a job with a specific id :param id: :return:

list(self, path, dir_only=False, recursive=False)

adds a list action to the queue

list the directory in the storage service :param service: service must be either source or destination :param path: :return:

action(self, specification)

executes the action identified by the specification. This is used by the run command.

Parameters

specification –

Returns

get_actions(self)
run(self)

runs the copy process for all jobs in the queue and completes when all actions are completed

Returns