cloudmesh-cloud.cloudmesh.vcluster.api.VirtualCluster

Module Contents

Classes

VirtualCluster

class cloudmesh-cloud.cloudmesh.vcluster.api.VirtualCluster.VirtualCluster(debug)

Bases: object

_config_validator(self)

validates the configuration of a run based on the information about its virtual cluster, runtime configuration and the job metadata

Returns

_clean_remote_in_parallel(self, target_node, remote_path)

This method is used to spawn processes to clean the remotes of a particular job.

Parameters

  • target_node – the node on which the data is going to be removed

  • remote_path – path of the data to be removed

Returns

_connection_test_in_parallel(self, target_node)

This method is used to test the connection to cluster nodes in parallel

Parameters

target_node – the node to which the connection is going to be tested

Returns

_create_config(self, config_name, proc_num, download_proc_num, download_later, input_type, output_type)

This method is used to create a runtime-configuration.

Parameters

  • config_name – name of the runtime configuration

  • proc_num – number of processes to be spawned in that runtime for submitting the jobs

  • download_proc_num – number number of processes to be spawned in that runtime for fetching the results

  • download_later – a flag indicating whether or not the script should wait for the results after the scripts are submitted

  • input_type – type of the input of the script to be run remotely

  • output_type – type of the output of the script to be run remotely

Returns

_create_vcluster(self, vcluster_name, cluster_list=(), computer_list=())

This method is used to create a virtual cluster

Parameters

  • vcluster_name – name of the virtual cluster

  • cluster_list – list of the clusters to be added to the virtual cluster

  • computer_list – list of the computers to be used from the previous parameter (cluster_list). If the computer_list is left empty, all of the computers will be used

Returns

static _execute_in_parallel(func_args)

This is a method used for running methods in parallel

Parameters

func_args –

Returns

_fetch_results_in_parallel(self, job_metadata, node_pid_tuple, all_pids)

This method is used to fetch the results from remote nodes.

Parameters

  • job_metadata – the dictionary containing the information about the previously submitted job

  • node_pid_tuple – the tuple containing destination node, destination pid and destination node index when the job was submitted

  • all_pids –

Returns

_run_remote_job_in_parallel(self, job_metadata, param_idx, params, all_pids)

This method is used to spawn remote processes in parallel

Parameters

  • job_metadata – contains the information about the job

  • param_idx – index of the parameters inputted as argument

  • params – the parameters inputted as argument for this run

  • all_pids – the manager used to take all pids of all submitted jobs

Returns

static _ssh(hostname, sshconfigpath, *args)

This method is used to create remove ssh connections

Parameters

  • hostname – hostname

  • sshconfigpath – path to sshconfig for connecting to remote node

  • args – the argument to be submitted via ssh

Returns

static _scp(hostname, sshconfigpath, *args)

This method is used for scp from and to remote

Parameters

  • hostname – hostname

  • sshconfigpath – ssh config file

:param args:arguments for using while copying :return:

static add_suffix_to_path(path, suffix)

This method is used to add suffix to a path

Parameters

  • path – path

  • suffix – suffix

Returns

clean_remote(self, job_name, proc_num)

This method is used to spawn processes for cleaning the remote nodes

Parameters

  • job_name – name of previously submitted job for which the nodes are going to be cleaned

  • proc_num – number of processes used for cleaning the remote nodes

Returns

connection_test(self, vcluster_name, proc_num)

This method is used for spawning processes for testing the connections to remote nodes of a vcluster

Parameters

  • vcluster_name – name of the virtual cluster the nodes of which are going to be tested

  • proc_num – number of processes used for testing the remote nodes

Returns

create(self, *args, **kwargs)

This is a caller for creator functions including config creator and vcluster creator

Parameters

  • args –

  • kwargs –

Returns

destroy(self, target, key)

Used to remove virtual clusters and runtime configs

Parameters

  • target – type of entity to be removed

  • key – keyname of the entity to be removed

Returns

fetch(self, job_name)

This method is used to fetch results from remote nodes

Parameters

job_name – the previously submitted job name

Returns

list(self, target, max_depth, current_depth=1, input_dict=None)

listing the current virtual clusters based on the vcluster_conf file.

Parameters

  • target – name of the virtual cluster to be listed

  • max_depth – depth of information to be shown

  • input_dict – used for recursion for depth of higher than 1

  • current_depth – current depth of printing information

Returns

run(self, job_name, cluster_name, config_name, script_path, argfile_path, outfile_name, remote_path, local_path, params_list, suffix, overwrite)

This method is used to create a job, validate it and run it on remote nodes

Parameters

  • job_name – name of the job to create

  • cluster_name – cluster on which the job is gonna run

  • config_name – name of the configuration based on which the job is going to run

  • script_path – path of the script to be run remotely

  • argfile_path – path of the file that has to be passed to the file as an argument if any

  • outfile_name – output filename resulted from running the script , if any

  • remote_path – path in the remotes on which the script is gonna be copied to and ran from

  • local_path – local path to which the results are gonna be copied

  • params_list – list of the parameters that are going to be passed to the script if any

  • suffix – suffix of the filenames in the job

  • overwrite – if the job already exists, this flag overwrites the previous job with the same name

Returns

set_param(self, target, name, parameter, value)

Used to set a specific parameter in the configuration

Parameters

  • target – the entity type on which the parameter is going to be set, e.g. runtime-config

  • name – the entity name on which the parameter is going to be set, e.g. test-config32

  • parameter – name of the parameter to be set

  • value – value of that parameter to be set

Returns