Cloudmesh Client Toolkit

The cloudmesh client toolkit is a lightweight client interface of accessing heterogeneous clouds, clusters, and workstations right from the users computer. The user can manage his own set of resources he would like to utilize. Thus the user has the freedom to customize their cyber infrastructure they use. Cloudmesh client includes an API, a commandline client, and a commandline shell. It strives to abstract backends to databases that are used to manage the workflow utilizing the different infrastructure and also the services. Switching for example to stage virtual machines from openstack clouds to amazon is as simple as specifying the name of the cloud. Moreover, cloudmesh client can be installed on Linux, MacOSX, and even Windows. Currently we support backends to SLURM, SSH, Openstack, Heat. In the past we supported AWS and Azure. We are in the process of integrating them back into the client.

This documentation and code is available on github at:

• Documentation:

  • on github: http://cloudmesh.github.io/client/
  • on rtd: http://cloudmesh-client.readthedocs.org/

• Code: https://github.com/cloudmesh/client

• Automated build reports and documentation:

  • Documentation: http://cloudmesh-client.readthedocs.org
  • Code: https://travis-ci.org/cloudmesh/client

Overview

• Prefix
  • Repository
    • Automated Builds and Reports
  • Contact
  • Authors
  • Conventions
  • Feature Requests
• Introduction
  • Where to go next?

Short Manuals

• Quickstart
  • Setup
  • Help
  • Defaults and Variables
  • Accessing Clouds
  • Virtual Machines
  • VM Quickstart
  • Microsoft Azure
  • HPC
• Reference Card
  • Shell
  • Shell commands that expire after a session
  • Clouds
  • Comet
  • HPC
• Example Scripts
  • Comment
  • Terminal Commands
  • Executing Shell commands
  • Executing Python
  • Variables
  • Group
  • Keys
  • VMs
  • Copy
  • Security Groups
  • Nova
  • Network
  • HPC
  • Cluster
  • Cloud
• Comet Virtual Cluster
  • Introduction
  • Links
  • Eligibility and expectation to get/run a virtual cluster on comet
  • Is Comet virtual cluster a right solution for your project?
  • Steps to get a virtual cluster on comet
  • FAQ

User Manuals

• Setup
  • Upgrade
  • Preparation
  • Installation
  • Configuration
• User Manual
  • Shell Commands
  • Cloud Commands
  • HPC Commands
  • Comet Commands
  • Proposed Commands
• Commands
  • akey
  • banner
  • check
  • clear
  • cloud
  • cluster
  • color
  • comet
  • context
  • debug
  • default
  • deploy
  • echo
  • EOF
  • exec
  • flavor
  • group
  • h
  • hadoop
  • help
  • history
  • hpc
  • image
  • info
  • inventory
  • key
  • launcher
  • limits
  • list
  • load
  • man
  • network
  • nist
  • nova
  • open
  • pause
  • portal
  • py
  • q
  • quit
  • quota
  • refresh
  • register
  • reservation
  • reset
  • rsync
  • secgroup
  • select
  • server
  • setup
  • shell
  • shell_exec
  • sleep
  • ssh
  • stack
  • submit
  • sync
  • test
  • timer
  • usage
  • var
  • vc
  • verbose
  • version
  • vm
  • workflow

Teaching

• Exercises
  • Assignment A: Prerequisite
  • Assignment B: IaaS
  • Assignment C: Ansible
  • Assignment D: Key Management

Tutorials

• Comet Cloudmesh Tutorial
  • Status of this Tutorial
  • Overview
  • Requirements
  • Install Cloudmesh Client tool to access Comet VC
  • Using Cloudmesh Client tool to access Comet VC
  • Virtual Cluster Architecture
  • Getting access to your cluster
  • Install the VC front-end node
  • Configuring the front-end node
  • Install VC Compute Nodes
  • DEMO1: Verifying InfiniBand Performance
  • DEMO2: OSU Micro-Benchmarks
  • DEMO3: Bigdata analytical software stack deployment and example run
  • DEMO4 (optional): OpenFOAM
  • DEMO5 (optional): Julia
• Comet Big Data Tutorial
  • Status of this Tutorial
  • Overview
  • Requirements
  • Preparation
  • Sanity Check
  • Deploy the Stack
  • Usage
  • Word Count
  • What’s next?

Developer Manuals

• API
  • Cloud Database
  • Updating an element in the database
• Hacking
  • Contributing
  • PRIORITIZATON
  • Requirements file
  • Editor
  • Documentation
  • Git

Appendix

• Appendix
  • Developer Notes
    • OpenStack API Examples
  • Changes
    • Summary of Changes
    • Changelog
      • Other
        • 3.1.0 (2016-03-09)
        • 3.0.0 (2016-03-04)
        • 2.1.3 (2016-02-18)
        • 2.1.2 (2016-02-15)
        • 2.1.1 (2016-02-15)
        • 2.0.5 (2016-02-11)
        • 1.1.6 (2016-02-09)
      • New
      • Changes
        • 1.1.4 (2016-02-01)
      • New
      • Changes
      • Fix
      • Other
        • 1.1.3 (2016-01-26)
      • New
      • Changes
      • Fix
      • Other
        • 1.1.2 (2016-01-13)
        • 1.1.1 (2016-01-13)
      • New
      • Other
        • 1.1.0 (2016-01-13)
      • New
      • Changes
      • Fix
      • Other
        • 1.0.2 (2015-11-25)
      • New
      • Other
        • 1.0.1 (2015-11-25)
        • 1.0.0 (2015-11-24)
      • New
      • Changes
      • Other
        • 0.7.7 (2015-11-18)
      • New
      • Changes
      • Other
        • 0.7.4 (2015-11-15)
      • New
      • Changes
      • Fix
      • Other
        • 0.7.3 (2015-10-31)
      • New
      • Changes
      • Other
        • 0.7.2 (2015-10-30)
      • New
      • Changes
      • Fix
      • Other
        • 0.7.0 (2015-09-30)
      • New
      • Changes
      • Fix
      • Other
        • 0.6.9 (2015-09-06)
        • 0.6.8 (2015-09-06)
        • 0.6.7 (2015-09-05)
        • 0.6.6 (2015-09-04)
        • 0.6.5 (2015-09-04)
        • 0.6.4 (2015-09-02)
        • 0.6.3 (2015-09-01)
        • 0.6.2 (2015-09-01)
        • 0.6.1 (2015-08-31)
        • 0.6.0 (2015-08-29)
        • 0.5.8 (2015-06-28)
        • 0.5.7 (2015-06-17)
        • 0.5.6 (2015-06-17)
        • 0.5.5 (2015-06-17)
        • 0.5.4 (2015-06-17)
    • Changelog
      • %%version%% (unreleased)
      • 3.1.2 (2016-03-18)
      • 3.1.1 (2016-03-16)
        • Changes
        • Other
      • 3.1.0 (2016-03-09)
      • 3.0.3 (2016-03-05)
      • 3.0.2 (2016-03-05)
      • 3.0.1 (2016-03-05)
      • 3.0.0 (2016-03-04)
        • Changes
        • Fix
        • Other
      • 2.2.1 (2016-02-20)
      • 2.2.0 (2016-02-20)
      • 2.1.3 (2016-02-18)
        • Changes
        • Other
      • 2.1.2 (2016-02-15)
        • Changes
        • Other
      • 2.1.1 (2016-02-15)
        • Changes
        • Other
      • 2.1.0 (2016-02-11)
      • 2.0.5 (2016-02-11)
      • 2.0.4 (2016-02-11)
      • 2.0.3 (2016-02-10)
      • 2.0.2 (2016-02-10)
      • 2.0.1 (2016-02-10)
      • 2.0.0 (2016-02-10)
      • 1.1.6 (2016-02-09)
        • New
        • Changes
        • Other
      • 1.1.5 (2016-02-02)
      • 1.1.4 (2016-02-01)
        • New
        • Changes
        • Fix
        • Other
      • 1.1.3 (2016-01-26)
        • New
        • Changes
        • Fix
        • Other
      • 1.1.2 (2016-01-13)
      • 1.1.1 (2016-01-13)
        • New
        • Other
      • 1.1.0 (2016-01-13)
        • New
        • Changes
        • Fix
        • Other
      • 1.0.2 (2015-11-25)
        • New
        • Other
      • 1.0.1 (2015-11-25)
      • 1.0.0 (2015-11-24)
        • New
        • Changes
        • Other
      • 0.7.7 (2015-11-18)
        • New
        • Changes
        • Other
      • 0.7.6 (2015-11-15)
      • 0.7.5 (2015-11-15)
      • 0.7.4 (2015-11-15)
        • New
        • Changes
        • Fix
        • Other
      • 0.7.3 (2015-10-31)
        • New
        • Changes
        • Other
      • 0.7.2 (2015-10-30)
        • New
        • Changes
        • Fix
        • Other
      • 0.7.1 (2015-09-30)
      • 0.7.0 (2015-09-30)
        • New
        • Changes
        • Fix
        • Other
      • 0.6.9 (2015-09-06)
      • 0.6.8 (2015-09-06)
      • 0.6.7 (2015-09-05)
      • 0.6.6 (2015-09-04)
      • 0.6.5 (2015-09-04)
      • 0.6.4 (2015-09-02)
      • 0.6.3 (2015-09-01)
      • 0.6.2 (2015-09-01)
      • 0.6.1 (2015-08-31)
      • 0.6.0 (2015-08-29)
      • 0.5.8 (2015-06-28)
      • 0.5.7 (2015-06-17)
      • 0.5.6 (2015-06-17)
      • 0.5.5 (2015-06-17)
      • 0.5.4 (2015-06-17)
  • ToDos

Future and Old

• Docker
• Stack
  • Questions
  • About BDS
    • BDS Requirements
    • Using BDS
  • Integrating BDS with Cloudmesh Client
    • Proposed Cloudmesh Changes
      • cm stack
      • cm hadoop
      • .cloudmesh/stack.yml
    • Use Case: Hadoop with Spark, HBase, Drill
  • Implementation Overview
    • Sanity Check cm stack check
    • Initialization cm stack init
    • Listing Stacks cm stack list
    • Switching Projects cm stack project
    • Deploying Onto Nodes cm stack deploy
    • Deploying Hadoop with Addons cm hadoop

Code

• cloudmesh_client package
  • Subpackages
    • cloudmesh_client.api package
      • Subpackages
      • Module contents
    • cloudmesh_client.cloud package
      • Subpackages
      • Submodules
      • cloudmesh_client.cloud.ListResource module
      • cloudmesh_client.cloud.attributes module
      • cloudmesh_client.cloud.counter module
      • cloudmesh_client.cloud.experiment module
      • cloudmesh_client.cloud.flavor module
      • cloudmesh_client.cloud.group module
      • cloudmesh_client.cloud.hpc module
      • cloudmesh_client.cloud.image module
      • cloudmesh_client.cloud.inventory module
      • cloudmesh_client.cloud.ip module
      • cloudmesh_client.cloud.key module
      • cloudmesh_client.cloud.launcher module
      • cloudmesh_client.cloud.limits module
      • cloudmesh_client.cloud.list module
      • cloudmesh_client.cloud.network module
      • cloudmesh_client.cloud.nova module
      • cloudmesh_client.cloud.quota module
      • cloudmesh_client.cloud.register module
      • cloudmesh_client.cloud.reservation module
      • cloudmesh_client.cloud.secgroup module
      • cloudmesh_client.cloud.stack module
      • cloudmesh_client.cloud.sync module
      • cloudmesh_client.cloud.usage module
      • cloudmesh_client.cloud.vc module
      • cloudmesh_client.cloud.vm module
      • cloudmesh_client.cloud.workflow module
      • Module contents
    • cloudmesh_client.comet package
      • Submodules
      • cloudmesh_client.comet.authenticate module
      • cloudmesh_client.comet.cluster module
      • cloudmesh_client.comet.comet module
      • Module contents
    • cloudmesh_client.common package
      • Subpackages
      • Submodules
      • cloudmesh_client.common.BaseConfigDict module
      • cloudmesh_client.common.ConfigDict module
      • cloudmesh_client.common.DictDB module
      • cloudmesh_client.common.DynamicClass module
      • cloudmesh_client.common.Error module
      • cloudmesh_client.common.FlatDict module
      • cloudmesh_client.common.FlatDict2 module
      • cloudmesh_client.common.GitInfo module
      • cloudmesh_client.common.LibcloudDict module
      • cloudmesh_client.common.LogUtil module
      • cloudmesh_client.common.Printer module
      • cloudmesh_client.common.SSHkey module
      • cloudmesh_client.common.Shell module
      • cloudmesh_client.common.StopWatch module
      • cloudmesh_client.common.TableParser module
      • cloudmesh_client.common.borg module
      • cloudmesh_client.common.dotdict module
      • cloudmesh_client.common.hostlist module
      • cloudmesh_client.common.menu module
      • cloudmesh_client.common.ssh_config module
      • cloudmesh_client.common.todo module
      • cloudmesh_client.common.type module
      • cloudmesh_client.common.util module
      • cloudmesh_client.common.vmname module
      • Module contents
    • cloudmesh_client.db package
      • Subpackages
      • Submodules
      • cloudmesh_client.db.CloudmeshDatabase module
      • Module contents
    • cloudmesh_client.deployer package
      • Subpackages
      • Module contents
    • cloudmesh_client.etc package
      • Module contents
    • cloudmesh_client.extend package
      • Submodules
      • cloudmesh_client.extend.my module
      • cloudmesh_client.extend.summary module
      • Module contents
    • cloudmesh_client.platform package
      • Subpackages
      • Module contents
    • cloudmesh_client.provider package
      • Subpackages
      • Module contents
    • cloudmesh_client.shell package
      • Subpackages
      • Submodules
      • cloudmesh_client.shell.cm module
      • cloudmesh_client.shell.cm_shell_status module
      • cloudmesh_client.shell.command module
      • cloudmesh_client.shell.console module
      • Module contents
    • cloudmesh_client.system package
      • Submodules
      • cloudmesh_client.system.rack module
      • cloudmesh_client.system.who module
      • Module contents
  • Submodules
  • cloudmesh_client.default module
  • cloudmesh_client.exc module
  • cloudmesh_client.locations module
  • cloudmesh_client.logger module
  • cloudmesh_client.main module
  • cloudmesh_client.setup module
  • cloudmesh_client.shell_commad module
  • cloudmesh_client.var module
  • cloudmesh_client.version module
  • Module contents

#.. toctree:: # :caption: outdated # :maxdepth: 4

# choco # cloudmesh_base # cloudmesh_client # commands_nova # commands_register # docker # man # old1 # system-cygwin

Indices and tables

• Index
• Module Index
• Search Page

Todo

• ToDos
• Using python 2 and 3 for development
  • Install python 3.5 and 2.7.11
    • OSX
• Python Guidelines
  • Exceptions
  • Developing Compatible Code
    • print
    • raise
    • traceback
    • exception
    • integer division
    • metaclass
    • strings
    • import
    • Dict
    • File
    • input
    • bytechar