Documentation

• 1: Linux
• 1.1: Shell
• 1.1.1: Linux Shell
• 1.1.2: Perl One Liners
• 1.1.3: ZSH
• 1.2: SSH
• 1.2.1: Introduction to SSH
• 1.2.2: SSH on Windows
• 1.3: Recordings
• 1.4: Refcards
• 2: Web
• 2.1: Hugo Themes
• 3: Hardware
• 3.1: Boards: Raspberry PI
• 3.2: Pi Clusters on the Internet
• 3.3: Cases: Pi Cluster Cases
• 3.4: Parts
• 3.4.1: Individual Parts
• 3.4.2: Parts for a 3 node cluster
• 3.4.3: Parts for a 8 node cluster
• 4: Software
• 4.1: Autogenerating Analytics Rest Services
• 4.2: Link Collection abut PI Clusters and Parts
• 4.3: Cloudmesh Pi Burner
• 4.4: Fortan
• 4.5: Kubernetes
• 4.6: Motherboard LED
• 4.7: Run at Boot
• 5: Contributors
• 5.1: Erin Seliger
• 5.2: Fidel Leal
• 5.3: Gregor von Laszewski
• 5.4: Venkata Sai Dhakshesh Kolli

1 - Linux

1.1 - Shell

1.1.1 - Linux Shell

We introduce you to a number of useful shell commands. You may ask:

“Why is he so keen on telling me all about shells as I do have a beautiful GUI?”

You will soon learn that A GUI may not be that suitable if you like to manage 10, 100, 1000, 10000, … virtual machines. A commandline interface could be mcuh simpler and would allow scripting.

History

LINUX is a reimplementation by the community of UNIX which was developed in 1969 by Ken Thompson and Dennis Ritchie of Bell Laboratories and rewritten in C. An important part of UNIX is what is called the kernel which allows the software to talk to the hardware and utilize it.

In 1991 Linus Torvalds started developing a Linux Kernel that was initially targeted for PC’s. This made it possible to run it on Laptops and was later on further developed by making it a full Operating system replacement for UNIX.

Shell

One of the most important features for us will be to access the computer with the help of a shell. The shell is typically run in what is called a terminal and allows interaction to the computer with commandline programs.

There are many good tutorials out there that explain why one needs a linux shell and not just a GUI. Randomly we picked the first one that came up with a google query. This is not an endorsement for the material we point to, but could be a worth while read for someone that has no experience in Shell programming:

• http://linuxcommand.org/lc3_learning_the_shell.php

Certainly you are welcome to use other resources that may suite you best. We will however summarize in table form a number of useful commands that you may als find even as a RefCard.

• http://www.cheat-sheets.org/#Linux

We provide in the next table a number of useful commands that you want to explore. For more information simply type man and the name of the command. If you find a useful command that is missing, please add it with a Git pull request.

The command man

On Linux you find a rich set of manual pages for thes commands. Try to pick one and execute:

$ man ls

You will see somthing like this

LS(1)                     BSD General Commands Manual                    LS(1)

NAME
     ls -- list directory contents

SYNOPSIS
     ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]

DESCRIPTION
     
     For each operand that names a file of a type other than directory,
     ls displays its name as well as any requested, associated
     information.  For each operand that names a file of type directory,
     ls displays the names of files contained within that directory, as
     well as any requested, associated information.

     If no operands are given, the contents of the current directory are
     displayed.  If more than one operand is given, non-directory
     operands are displayed first; directory and non-directory operands
     are sorted separately and in lexicographical order.

     The following options are available:

     -@      Display extended attribute keys and sizes in long (-l) output.

     -1      (The numeric digit ``one''.)  Force output to be one entry
             per line.  This is the default when output is not to a terminal.

     -A      List all entries except for . and ...  Always set for the
             super-user.

     -a      Include directory entries whose names begin with a dot (.).


     ... on purpose cut ... instead try it yourslef

Multi-command execution

One of the important features is that one can execute multiple commands in the shell.

To execute command 2 once command 1 has finished use

command1; command2

To execute command 2 as soon as command 1 forwards output to stdout use

command1; command2

To execute command 1 in the background use

command1 &

Keyboard Shortcuts

These shortcuts will come in handy. Note that many overlap with emacs short cuts.

bashrc, bash_profile or zprofile

Usage of a particular command and all the attributes associated with it, use man command. Avoid using rm -r command to delete files recursively. A good way to avoid accidental deletion is to include the following in the file .bash_profile or .zprofile on macOS or .bashrc on other platforms:

alias rm='rm -i'
alias mv='mv -i'
alias h='history'

Makefile

Makefiles allow developers to coordinate the execution of code compilations. This not only includes C or C++ code, but any translation from source to a final format. For us this could include the creation of PDF files from latex sources, creation of docker images, and the creation of cloud services and their deployment through simple workflows represented in makefiles, or the coordination of execution targets.

As makefiles include a simple syntax allowing structural dependencies they can easily adapted to fulfill simple activities to be executed in repeated fashion by developers.

An example of how to use Makefiles for docker is provided at

• http://jmkhael.io/makefiles-for-your-dockerfiles/.

An example on how to use Makefiles for LaTeX is provided at

• https://github.com/cloudmesh/book/blob/master/Makefile.

Makefiles include a number of rules that are defined by a target name. Let us define a target called hello that prints out the string “Hello World”.

hello:
    @echo "Hello World"

Important to remember is that the commands after a target are not indented just by spaces, but actually by a single TAB character. Editors such as emacs will be ideal to edit such Makefiles, while allowing syntax highlighting and easy manipulation of TABs. Naturally other editors will do that also. Please chose your editor of choice. One of the best features of targets is that they can depend on other targets. Thus, iw we define

hallo: hello
    @echo "Hallo World"

our makefile will first execute hello and than all commands in hallo. As you can see this can be very useful for defining simple dependencies.

In addition we can define variables in a makefile such as

HELLO="Hello World"

hello:
    @echo $(HELLO)

and can use them in our text with $ invocations.

Moreover, in sophisticated Makefiles, we could even make the targets dependent on files and a target rules could be defined that only compiles those files that have changed since our last invocation of the Makefile, saving potentially a lot of time. However, for our work here we just use the most elementary makefiles.

For more information we recommend you to find out about it on the internet. A convenient reference card is available at

• http://www.cs.jhu.edu/~joanne/unixRC.pdf.

chmod

The chmod command stand for change mode and changes the access permissions for a given file system object(s). It uses the following syntax: chmod [options] mode[,mode] file1 [file2…]. The option parameters modify how the process runs, including what information is outputted to the shell:

Modes specify which rights to give to which users. Potential users include the user who owns the file, users in the file’s Group, other users not in the file’s Group, and all, and are abbreviated as u, g, o, and a respectively. More than one user can be specified in the same command, such as chmod –v ug(operator)(permissions) file.txt . If no user is specified, the command defaults to a. Next, a + or - indicates whether permissions should be added or removed for the selected user(s). The permissions are as follows:

More than one permission can be also be used in the same command as follows:

$ chmod –v o+rw file.txt

Multiple files can also be specified:

$ chmod a-x,o+r file1.txt file2.txt

Exercises

E.Linux.1

Familiarize yourself with the commands

E.Linux.2

Find more commands that you find useful and add them to this page.

E.Linux.3

Use the sort command to sort all lines of a file while removing duplicates.

E.Linux.4

Should there be other commands listed in the table with the Linux commands If so which? Create a pull request for them.

E.Linux.5

Write a section explaining chmod. Use letters not numbers

E.Linux.6

Write a section explaining chown. Use letters not numbers

E.Linux.7

Write a section explaining su and sudo

E.Linux.8

Write a section explaining cron, at, and crontab

1.1.2 - Perl One Liners

Perl is a programming language that used to be very popular with system administrators. It predates Python. It has some very powerful regular expression abilities allowing you to easily do things on the commandline that woul otherwise thake many hours. Here ar some useful perl one line commands.

Strip trailing whitespace from a file

$ perl -lpe 's/\s*$//' FILENAME

Replace wrong quote

$ perl -i -p -e "s/’/'/g;"  *.md

Remove ^M from file

$ perl -p -i -e 's/\r\n$/\n/g'

1.1.3 - ZSH

Introduction

Z shell (zsh) is an alternative to bash. It is used as an interactive shell or command interpreter. Zsh has been chosen by apple as a replacement for bash. A large number of plugins for zsh is avalable at the Web site Oh My Zsh.

Features of zsh include:

• commandline completion
• global history that can be shared in shells
• build in file globing
• multiline commands
• spell correction
• compatibillity modes to impersonate other shells
• themes for prompts
• in addition to which a wher ecommand
• shortcut to names directories with ~

In principal it does not matter much whcih shell you use as long as you use to set up your envireonment properly. While bash zupporst .bash_profile and .bashrc, zsh supports ~/.zprofile and ~/.zshrc

A good overview of the loading process is documented at

• https://medium.com/@rajsek/zsh-bash-startup-files-loading-order-bashrc-zshrc-etc-e30045652f2e

Setting up zsh on an older OSX is relatively simple.

$ brew install zsh

To add Oh My Zsh you can do:

$ sh -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"

To chnge the default shell to zsh you can than execute

$ chsh -s $(which zsh)

To activate the shell you can as usal do

$ source ~/.zshrc

However if you start a new terminal, you do not have to do this as it is added automatically.

To use a number of useful plugins, you can activate them with

$ plugins=(osx git colored-man colorize pip python brew zsh-syntax-highlighting zsh-autosuggestions)

If you like to change the theme you can find a large number at

• https://github.com/ohmyzsh/ohmyzsh/wiki/Themes

Using zsh on your operating systems

For other operationg systems see

• https://github.com/ohmyzsh/ohmyzsh/wiki/Installing-ZSH

zsh on Windows 10

To install zsh on Windows 10, please look at

• https://www.howtogeek.com/258518/how-to-use-zsh-or-another-shell-in-windows-10/

1.2 - SSH

1.2.1 - Introduction to SSH

Secure Shell is a network protocol allowing users to securely connect to remote resources over the internet. In many services we need to use SSH to assure that we protect he messages send between the communicating entities. Secure Shell is based on public key technology requiring to generate a public-private key pair on the computer. The public key will than be uploaded to the remote machine and when a connection is established during authentication the public private key pair is tested. If they match authentication is granted. As many users may have to share a computer it is possible to add a list of public keys so that a number of computers can connect to a server that hosts such a list. This mechanism builds the basis for networked computers.

In this section we will introduce you to some of the commands to utilize secure shell. We will reuse this technology in other sections to for example create a network of workstations to which we can log in from your laptop. For more information please also consult with the SSH Manual.

Whatever others tell you, the private key should never be copied to another machine. You almost always want to have a passphrase protecting your key.

ssh-keygen

The first thing you will need to do is to create a public private key pair. Before you do this check whether there are already keys on the computer you are using:

ls ~/.ssh

If there are files named id_rsa.pub or id_dsa.pub, then the keys are set up already, and we can skip the generating keys step. However you must know the passphrase of the key. If you forgot it you will need to recreate the key. However you will lose any ability to connect with the old key to the resources to which you uploaded the public key. So be careful.

To generate a key pair use the command ssh-keygen. This program is commonly available on most UNIX systems and most recently even Windows 10.

To generate the key, please type:

$ ssh-keygen -t rsa -C <comment>

The comment will remind you where the key has been created, you could for example use the hostname on which you created the key.

In the following text we will use localname to indicate the username on your computer on which you execute the command.

The command requires the interaction of the user. The first question is:

Enter file in which to save the key (/home/localname/.ssh/id_rsa):

We recommend using the default location ~/.ssh/ and the default name id_rsa. To do so, just press the enter key.

The second and third question is to protect your ssh key with a passphrase. This passphrase will protect your key because you need to type it when you want to use it. Thus, you can either type a passphrase or press enter to leave it without passphrase. To avoid security problems, you MUST chose a passphrase.

It will ask you for the location and name of the new key. It will also ask you for a passphrase, which you MUST provide. Please use a strong passphrase to protect it appropriately. Some may advise you (including teachers and TA’s) to not use passphrases. This is WRONG as it allows someone that gains access to your computer to also gain access to all resources that have the public key. Only for some system related services you may create passwordless keys, but such systems need to be properly protected.

Not using passphrases poses a security risk!

Make sure to not just type return for an empty passphrase:

Enter passphrase (empty for no passphrase):

and:

Enter same passphrase again:

If executed correctly, you will see some output similar to:

Generating public/private rsa key pair.
Enter file in which to save the key (/home/localname/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/localname/.ssh/id_rsa.
Your public key has been saved in /home/localname/.ssh/id_rsa.pub.
The key fingerprint is:
34:87:67:ea:c2:49:ee:c2:81:d2:10:84:b1:3e:05:59 localname@indiana.edu

+--[ RSA 2048]----+
|.+...Eo= .       |
| ..=.o + o +o    |
|O.  = ......     |
| = .   . .       |
+-----------------+

Once, you have generated your key, you should have them in the .ssh directory. You can check it by:

$ cat ~/.ssh/id_rsa.pub

If everything is normal, you will see something like:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXJH2iG2FMHqC6T/U7uB8kt
6KlRh4kUOjgw9sc4Uu+Uwe/kshuispauhfsjhfm,anf6787sjgdkjsgl+EwD0
thkoamyi0VvhTVZhj61pTdhyl1t8hlkoL19JVnVBPP5kIN3wVyNAJjYBrAUNW
4dXKXtmfkXp98T3OW4mxAtTH434MaT+QcPTcxims/hwsUeDAVKZY7UgZhEbiE
xxkejtnRBHTipi0W03W05TOUGRW7EuKf/4ftNVPilCO4DpfY44NFG1xPwHeim
Uk+t9h48pBQj16FrUCp0rS02Pj+4/9dNeS1kmNJu5ZYS8HVRhvuoTXuAY/UVc
ynEPUegkp+qYnR user@myemail.edu

The directory ~/.ssh will also contain the private key id_rsa which you must not share or copy to another computer.

Never, copy your private key to another machine or check it into a repository!

To see what is in the .ssh directory, please use

$ ls ~/.ssh

Typically you will se a list of files such as

authorized_keys
id_rsa
id_rsa.pub
known_hosts

In case you need to change your change passphrase, you can simply run ssh-keygen -p command. Then specify the location of your current key, and input (old and) new passphrases. There is no need to re-generate keys:

ssh-keygen -p

You will see the following output once you have completed that step:

Enter file in which the key is (/home/localname/.ssh/id_rsa):
Enter old passphrase:
Key has comment '/home/localname/.ssh/id_rsa'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

ssh-add

Often you wil find wrong information about passphrases on the internet and people recommending you not to use one. However it is in almost all cases better to create a key pair and use ssh-add to add the key to the current session so it can be used in behalf of you. This is accomplished with an agent.

The ssh-add command adds SSH private keys into the SSH authentication agent for implementing single sign-on with SSH. ssh-add allows the user to use any number of servers that are spread across any number of organizations, without having to type in a password every time when connecting between servers. This is commonly used by system administrators to login to multiple server.

ssh-add can be run without arguments. When run without arguments, it adds the following default files if they do exist:

• ~/.ssh/identity - Contains the protocol version 1 RSA authentication identity of the user.
• ~/.ssh/id_rsa - Contains the protocol version 1 RSA authentication identity of the user.
• ~/.ssh/id_dsa - Contains the protocol version 2 DSA authentication identity of the user.
• ~/.ssh/id_ecdsa - Contains the protocol version 2 ECDSA authentication identity of the user.

To add a key you can provide the path of the key file as an argument to ssh-add. For example,

ssh-add ~/.ssh/id_rsa

would add the file ~/.ssh/id_rsa

If the key being added has a passphrase, ssh-add will run the ssh-askpass program to obtain the passphrase from the user. If the SSH_ASKPASS environment variable is set, the program given by that environment variable is used instead.

Some people use the SSH_ASKPASS environment variable in scripts to provide a passphrase for a key. The passphrase might then be hard-coded into the script, or the script might fetch it from a password vault.

The command line options of ssh-add are as follows:

SSH Add and Agent

To not always type in your password, you can use ssh-add as previously discussed

It prompts the user for a private key passphrase and add it to a list of keys managed by the ssh-agent. Once it is in this list, you will not be asked for the passphrase as long as the agent is running.with your public key. To use the key across terminal shells you can start an ssh agent.

To start the agent please use the following command:

$ eval `ssh-agent`

or use

$ eval "$(ssh-agent -s)"

It is important that you use the backquote, located under the tilde (US keyboard), rather than the single quote. Once the agent is started it will print a PID that you can use to interact with later

To add the key use the command

$ ssh-add

To remove the agent use the command

kill $SSH_AGENT_PID

To execute the command upon logout, place it in your .bash_logout (assuming you use bash).

On OSX you can also add the key permanently to the keychain if you do toe following:

ssh-add -K ~/.ssh/id_rsa

Modify the file .ssh/config and add the following lines:

Host *
  UseKeychain yes
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_rsa

Using SSH on Mac OS X

Mac OS X comes with an ssh client. In order to use it you need to open the Terminal.app application. Go to Finder, then click Go in the menu bar at the top of the screen. Now click Utilities and then open the Terminal application.

Using SSH on Linux

All Linux versions come with ssh and can be used right from the terminal.

Using SSH on Raspberry Pi 3/4

SSH is available on Raspbian. However, to ssh into the PI you have to activate it via the configuration menu.

Accessing a Remote Machine

Once the key pair is generated, you can use it to access a remote machine. To dod so the public key needs to be added to the authorized_keys file on the remote machine.

The easiest way to do tis is to use the command ssh-copy-id.

$ ssh-copy-id user@host

Note that the first time you will have to authenticate with your password.

Alternatively, if the ssh-copy-id is not available on your system, you can copy the file manually over SSH:

$ cat ~/.ssh/id_rsa.pub | ssh user@host 'cat >> .ssh/authorized_keys'

Now try:

$ ssh user@host

and you will not be prompted for a password. However, if you set a passphrase when creating your SSH key, you will be asked to enter the passphrase at that time (and whenever else you log in in the future). To avoid typing in the password all the time we use the ssh-add command that we described earlier.

$ ssh-add

SSH Port Forwarding :o2:

:o2: TODO: Add images to illustrate the concepts

SSH Port forwarding (SSH tunneling) creates an encrypted secure connection between a local computer and a remote computer through which services can be relayed. Because the connection is encrypted, SSH tunneling is useful for transmitting information that uses an unencrypted protocol.

Prerequisites

• Before you begin, you need to check if forwarding is allowed on the SSH server you will connect to.
• You also need to have a SSH client on the computer you are working on.

If you are using the OpenSSH server:

$ vi /etc/ssh/sshd_config

and look and change the following:

AllowTcpForwarding = Yes
GatewayPorts = Yes

Set the GatewaysPorts variable only if you are going to use remote port forwarding (discussed later in this tutorial). Then, you need to restart the server for the change to take effect.

How to Restart the Server

If you are on:

• Linux, depending upon the init system used by your distribution, run:

   $ sudo systemctl restart sshd
   $ sudo service sshd restart
  

  Note that depending on your distribution, you may have to change the service to ssh instead of sshd.

• Mac, you can restart the server using:

  $ sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
  $ sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
  

• Windows and want to set up a SSH server, have a look at MSYS2 or Cygwin.

Types of Port Forwarding

There are three types of SSH Port forwarding:

Local Port Forwarding

Local port forwarding lets you connect from your local computer to another server. It allows you to forward traffic on a port of your local computer to the SSH server, which is forwarded to a destination server. To use local port forwarding, you need to know your destination server, and two port numbers.

Example 1:

$ ssh -L 8080:www.cloudcomputing.org:80 <host>

Where <host> should be replaced by the name of your laptop. The -L option specifies local port forwarding. For the duration of the SSH session, pointing your browser at http://localhost:8080/ would send you to http://cloudcomputing.com

Example 2:

This example opens a connection to the www.cloudcomputing.com jump server, and forwards any connection to port 80 on the local machine to port 80 on intra.example.com.

$ ssh -L 80:intra.example.com:80 www.cloudcomputing.com

Example 3:

By default, anyone (even on different machines) can connect to the specified port on the SSH client machine. However, this can be restricted to programs on the same host by supplying a bind address:

$ ssh -L 127.0.0.1:80:intra.example.com:80 www.cloudcomputing.com

Example 4:

$ ssh -L 8080:www.Cloudcomputing.com:80 -L 12345:cloud.com:80 <host>

This would forward two connections, one to www.cloudcomputing.com, the other to www.cloud.com. Pointing your browser at http://localhost:8080/ would download pages from www.cloudcomputing.com, and pointing your browser to http://localhost:12345/ would download pages from www.cloud.com.

Example 5:

The destination server can even be the same as the SSH server.

$ ssh -L 5900:localhost:5900 <host>

The LocalForward option in the OpenSSH client configuration file can be used to configure forwarding without having to specify it on command line.

Remote Port Forwarding

Remote port forwarding is the exact opposite of local port forwarding. It forwards traffic coming to a port on your server to your local computer, and then it is sent to a destination. The first argument should be the remote port where traffic will be directed on the remote system. The second argument should be the address and port to point the traffic to when it arrives on the local system.

$ ssh -R 9000:localhost:3000 user@clodcomputing.com

SSH does not by default allow remote hosts to forwarded ports. To enable remote forwarding add the following to: /etc/ssh/sshd_config

GatewayPorts yes

$ sudo vim /etc/ssh/sshd_config

and restart SSH

$ sudo service ssh restart

After completing the previous steps you should be able to connect to the server remotely, even from your local machine. ssh -R first creates an SSH tunnel that forwards traffic from the server on port 9000 to your local machine on port 3000.

Dynamic Port Forwarding

Dynamic port forwarding turns your SSH client into a SOCKS proxy server. SOCKS is a little-known but widely-implemented protocol for programs to request any Internet connection through a proxy server. Each program that uses the proxy server needs to be configured specifically, and reconfigured when you stop using the proxy server.

$ ssh -D 5000 user@clodcomputing.com

The SSH client creates a SOCKS proxy at port 5000 on your local computer. Any traffic sent to this port is sent to its destination through the SSH server.

Next, you’ll need to configure your applications to use this server. The Settings section of most web browsers allow you to use a SOCKS proxy.

ssh config

Defaults and other configurations can be added to a configuration file that is placed in the system. The ssh program on a host receives its configuration from

• the command line options
• a user-specific configuration file: ~/.ssh/config
• a system-wide configuration file: /etc/ssh/ssh_config

Next we provide an example on how to use a config file

Tips

Use SSH keys

• You will need to use ssh keys to access remote machines

No blank passphrases

• In most cases you must use a passphrase with your key. In fact if we find that you use passwordless keys to futuresystems and to chameleon cloud resources, we may elect to give you anF for the assignment in question. There are some exceptions, but they will be clearly communicated to you in class. You will as part of your cloud drivers license test explain how you gain access to futuresystems and chameleon to explicitly explain this point and provide us with reasons what you can not do.

A key for each server

• Under no circumstances copy the same private key on multiple servers. This violates security best practices. Create for each server a new private key and use their public keys to gain access to the appropriate server.

Use SSH agent

• So as to not to type in all the time the passphrase for a key, we recommend using ssh-agent to manage the login. This will be part of your cloud drivers license.

  But shut down the ssh-agent if not in use

keep an offline backup, put encrypt the drive

• You may for some of our projects need to make backups of private keys on other servers you set up. If you like to make a backup you can do so on a USB stick, but make sure that access to the stick is encrypted. Do not store anything else on that key and look it in a safe place. If you lose the stick, recreate all keys on all machines.

References

• The Secure Shell: The Definitive Guide, 2 Ed (O’Reilly and Associates)

SSH Exercises

E.SSH.1:

Create an SSH key pair

E.SSH.2:

Upload the public key to git repository you use.

E.SSH.3:

What is the output of a key that has a passphrase when executing the following command. Test it out on your key

 $ grep ENCRYPTED ~/.ssh/id_rsa

E.SSH.4

Get an account on futuresystems.org (if you are authorized to do so). Upload your key to https://futuresystems.org. Login to india.futuresystems.org. Note that this could take some time as administrators need to approve you. Be patient.

E.SSH.5:

What can happen if you copy your private key to a machine on the network?

E.SSH.6:

Should I share my provate key with others?

E.SSH.7:

Assume I participate in a video conference call and I accidently share my private key. What should I do?

E.SSH.8:

Assume I participate in a video conference call and I accidently share my public key. What should I do?

1.2.2 - SSH on Windows

For this class we recommend that you use a virtual machine via virtual box and use the Linux ssh instructions. The information here is just provided for completeness and no support will be offered for native windows support.

Windows users need to have some special software to be able to use the SSH commands. If you have one that you are comfortable with and know how to setup key pairs and access the contents of your public key, please feel free to use it.

On Windows you have a couple of options on running Linux commands such as ssh. At this time it may be worth while to try the OpenSSH Client available for Windows, although it is in beta. If you like to use other methods we have included alternatives.

OpenSSH Client on Windows

software to be able to run it directly from the Windows commandline including PowerShell.

However it is as far as we know not activated by default so you need to follow some setup scripts. Also this software is considered beta and its development and issues can be found at

• https://github.com/PowerShell/openssh-portable

Fortunately, the software is already distributed with Winodws 10, but may not yet been activated. What you have to do is to install it by going to Settings > Apps and click Manage optional features under Apps & features.

Next, Click on the Add feature. You will be presented with a list in which you scroll down, till you find OpenSSH Client (Beta). Click on it and invoke Install.

After the install has completed, you can use the ssh command. Just type it in the commandshell or PowerShell

PS C:\Users\gregor> ssh

Naturally you can now use it just as on Linux or macOS. and use it to login to other resources

PS C:\Users\gregor> ssh myname@computer.example.com

see also the MS SSH Guide for the newest up dates.

Due to the availability of SSH on Windows 10, we no longer recommend using Cygwin SSH, PuTTY or Chocolatey. However we kept thise sections here for completness.

GitBash

A realy great tool for Windows is made avalable via

• https://gitforwindows.org/

Here you can find gitbash that provides you with a terminal in which you can natively execute linux commands such as cd, ls and many more. It also includes ssh and ssh-keygen. which you will need if you want o interface with Linux machines hosted in a cloud.

You can also enable the Git GUI as you may be used to doing things form GUI’s. However soon you will find out why in this class we typicaly do not much via GUIs. However if you like them you can also integrate git in the Windows Explorer. This could be beneficial fo you during development of your project or keep up with what others do on git.

Makefiles on Windows

Makefiles can easily be accessed also on windows while installing gitbash. Please reed to the internet or search in this handbook for more information about gitbash.

:o2: Please contribute to this section on how to install make on wondows natively. Here is some information to start with Make on Windows

Using SSH from Cygwin

One established way of using ssh is from using cygwin.

http://cygwin.com/install.html Cygwin contains a collection of GNU and Open Source tools providing Linux like functionality on Windows. A DLL is available that exposes the POSIX API functionality.

A list of supported commands is available at

https://cygwin.com/packages/package_list.html Please be minded that in order for cygwin to function easily the Windows user name should not include spaces. However, as the setup in windows encourages to use the full name when you buy and setup a machine it may not be convenient to use. However, we just recommend that you create yourself a new username and use this if you like to use cygwin.

You can selectively install from the cygwin setup terminal which software you like to use, obviously you may want to use ssh

SSH from putty

As you will see the process is somewhat cumbersome and when you compare it with the commandline tools available, we do recommend using them instead.

PuTTY allows you to access the SSH, Telnet and Rlogin network protocols from windows.

https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html Although PuTTY has been out there for many years and served the community well, it is not following the standard ssh command line syntax when invoked from a command shell.

putty -ssh user@host.name

In addition to using ssh, it also provides a copy command.

pscp user@host.name:"\"remote filename with spaces\"" local_filename

Putty is best known for its GUI configuration application to manage several machines as demonstrated next. Once you have downloaded it and opened PuTTYgen, you will be presented with a a key generator window (images provided by chameleon cloud) (see Figure [F:putty-key]{reference-type=“ref” reference=“F:putty-key”}).

\FIGURE{htb} {0.5} {images/chameleon/putty2.png} {Key generation window} {F:putty-key}

To generate a key you click the Generate button which is blue. The PuTTY Key Generator (see Figure [F:putty-pass]{reference-type=“ref” reference=“F:putty-pass”}) will then ask you to move your mouse around the program’s blank space to generate “randomness” for your key. You must enter a “Key passphrase” and then confirm the passphrase.

\FIGURE{htb} {0.5} {images/chameleon/putty3.png} {Key generation window} {F:putty-pass}

Next you need to save both the public and private keys into a file of your choice using the “Save public key” and “Save private key” buttons. We suggest you name something obvious like “public_key.pub” and “private_key” so that you can distinguish between the two.

Before closing this window, select the entire public key and copy it with “Control-C”. Please note that everything should be copied, including “ssh-rsa”. This will be used when importing the key pair to Openstack.

At this time, the public key has been created and copied. Now you can use the public key and upload it to systems you like to login to.

Chocolatey

Another approach is to use it in Powershell with the help of chocolatey. Other options may be better suited for you and we leave it up to you to make this decision.

Chocolatey is a software management tool that mimics the install experience that you have on Linux and macOS. It has a repository with many packages. The packages are maintained by the community and you need to evaluate security implications when installing packages hosted on chocolatey just as you have to do if you install software on Linux and macOS from their repositories. Please be aware that there could be malicious code offered in the chocolatey repository although the distributors try to remove them.

The installation is sufficiently explained at

https://chocolatey.org/install Once installed you have a command choco and you should make sure you have the newest version with:

choco upgrade chocolatey

Now you can browse packages at

https://chocolatey.org/packages Search for openssh and see the results. You may find different versions. Select the one that most suits you and satisfies your security requirements as well as your architecture. Lets assume you chose the Microsoft port, than you can install it with:

choco install openssh

Naturally, you can also install cygwin and ptty over chocolatey. A list of packages can be found at

https://chocolatey.org/packages Packages of interest include

• emacs: choco install emacs
• pandoc: choco install pandoc
• LaTeX: choco install miktex
• jabref: choco install jabref
• pycharm: choco install pycharm-community
• lyx: choco install lyx
• python 2: choco install python2
• python 3: choco install python
• pip: choco install pip
• virtualbox: choco install virtualbox
• vagrant: choco install vagrant

Before installing any of them evaluate if you need them and identify security risks.

1.3 - Recordings

In Linux and OSX it is possible to easly record and replay a terminal session. There are several programs that can do this. A good overview article about it is given at

• https://linoxide.com/linux-how-to/linux-terminal-recording-tools/

From these programs we have tried

• script, which is preinstalled and you do not have it immediatly available
• TermRecord that also depends on ttyrec, so you need to install this as well

TermRecord

OSX

$ brew install ttyrec
$ pip install TermRecord

To use it simply invoke

$ TermRecord -o recording.html

To replay open the html page in a browser. Befor eyou publish it for example in the docs folder in a github repository, please make sure you have not exposed any information that allows others to compromise your system.

In case you use chrome you can replay it also from the commandline with

$ google-chrome /tmp/test.html

Screencapture

On macOS you can use the screen capture command use

OSX SHIT-COMMAND - 4 Screen capture

or say on the commandline

$ screencapture a.gif

Do not use screenshots when filing bugs

In general it is a bad practice to use screenshots whne filing bugs. The reason is that they can not easily parsed or content can be copied to replicate the bug. Thus avoid using screenshots when using for example github issues if you can.

1.4 - Refcards

We present you with a list of useful short reference cards. This cards can be extremely useful to remind yourself about some important commands and features. Having them could simplify your interaction with the systems, We not only collected here some refcards about Linux, but also about other useful tools and services.

If you like to add new topics, let us know via your contribution (see the contribution section).

Collections

• CheatSheets

Editors

• Emacs
• Vi
• Vim

Documentation

• LaTeX
• RST
• Markdown (GitHub
• Markdown (commonmark)
• Markdown (Hugo choices)
• Markdown (Hugo Syntax)

Linux

• Linux
• Makefile
• Git

Cloud/Virtualization

• Openstack
• Openstack
• vagrant

SQL

• SQL

Languages

• R

Python

• Python
• PythonData
• Numpy/Pandas
• PythonTutorial
• Python
• Python
• PythonAPIIndex
• Python3

2 - Web

2.1 - Hugo Themes

When creating Web pages often you are looking for an easy way to organize documentation tha is in a more structured form mthan simply a blog.

For this reason the following hugo themes may be useful:

Docsy

• Docsy: https://themes.gohugo.io/docsy/

  • cons: no modules, needs tto switch from master to main
  • pro: nice front page, we used this a lot, this Web sit is create with it
  

Dot

• Dot: https://themes.gohugo.io/dot-hugo-documentation-theme/

  

Syna

• Syna https://themes.gohugo.io/syna/

  

Compose

• Compose: https://themes.gohugo.io/compose/

  • pro: modules, menu with dynamic location update
  

Learn

• Learn: https://themes.gohugo.io/hugo-theme-learn/

  
  • pro colapsible menu menu, next button on pages

Hugo Clarity

A nice Blog theme

• hugo-clarity: https://themes.gohugo.io/hugo-clarity/

  

Dynamic Web Site (non documentation focused)

Although not a documentation oriented Wen page, The following recieved our attention as it show a very dynamic look. However it could be a bit distracting.

So one could develop two repos. One as frontpage with a splashy theme, the oster tha just focusses on the documentation

• Omega: https://themes.gohugo.io/omega-hugo-theme/

  

Converting Hugo to ePub

The following git repo contains documentation on how to convert hugo Web sites to epub

• https://github.com/weitblick/epub

An article in german is available at

• https://www.raspberrypi-spy.co.uk/wp-content/uploads/2012/11/Raspberry-Pi-Mounting-Hole-Template.pdf

The following script stes this up for you automatically:

git clone https://github.com/weitblick/epub.git
cd epub
cd exampleSite
mkdir themes
cd themes
git clone https://github.com/weitblick/epub.git
cd epub 
rm -rf .git
rm -rf exampleSite   # remove the example site in the theme
cd ../..
hugo
bash ./deploy.sh

3 - Hardware

3.1 - Boards: Raspberry PI

Raspberry PI’s are a convenient cheap compute platform that allow us to explore create cloud clusters with various software that otherwise would not be accessible to most. The point is not to create a complex compute platform, but to create a testbed in which we can explore configuration aspects and prepare benchmarks that are run on larger and expensive cloud environments. In addition Raspberry Pis can be used as a simple Linux terminal to log into other machines.

We will give a small introduction to the platform next.

Raspberry PI 4B

Figure 1: Pi 4B board

The board has the following properties ¹:

• Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
• 2GB, 4GB or 8GB LPDDR4-3200 SDRAM (depending on model)
• 2.4 GHz and 5.0 GHz IEEE 802.11ac wireless, Bluetooth 5.0, BLE
• Gigabit Ethernet
• 2 USB 3.0 ports; 2 USB 2.0 ports.
• Raspberry Pi standard 40 pin GPIO header (fully backwards compatible with previous boards)
• 2 × micro-HDMI ports (up to 4kp60 supported)
• 2-lane MIPI DSI display port
• 2-lane MIPI CSI camera port
• 4-pole stereo audio and composite video port
• H.265 (4kp60 decode), H264 (1080p60 decode, 1080p30 encode)
• OpenGL ES 3.0 graphics
• Micro-SD card slot for loading operating system and data storage
• 5V DC via USB-C connector (minimum 3A*)
• 5V DC via GPIO header (minimum 3A*)
• Power over Ethernet (PoE) enabled (requires separate PoE HAT)
• Operating temperature: 0 – 50 degrees C ambient

Important for the cluster is the following comment:

• A good quality 2.5A power supply can be used if downstream USB peripherals consume less than 500mA in total.

Raspberry PI 3 B+

We plan to purchase a number of them so we can conduct performance experiments and leverage the faster hardware. The newest Raspberry PI 3 B+ is shown in Figure 2.

Figure 2: Raspberry PI 3B+ board

The board has the following properties:

• Broadcom BCM2837B0, Cortex-A53 (ARMv8) 64-bit SoC @ 1.4GHz
• 1GB LPDDR2 SDRAM
• 2.4GHz and 5GHz IEEE 802.11.b/g/n/ac wireless LAN
• Bluetooth 4.2, BLE
• Gigabit Ethernet over USB 2.0 (maximum throughput 300 Mbps)
• Extended 40-pin GPIO header
• Full-size HDMI
• 4 USB 2.0 ports
• CSI camera port for connecting a Raspberry Pi camera
• DSI display port for connecting a Raspberry Pi touchscreen display
• 4-pole stereo output and composite video port
• Micro SD port for loading your operating system and storing data
• 5V/2.5A DC power input
• Power-over-Ethernet (PoE) support (requires separate PoE HAT)

Raspberry PI 3 B

Till February 2018 the Raspberry PI 3 B was the newest model. Within this class we have access to about 100 of them. The Raspberry PI 3 B is shown in Figure 3.

Figure 3: Raspberry PI 3 B board

The board has the following properties:

• Quad Core 1.2GHz Broadcom BCM2837 64bit CPU
• 1GB RAM
• BCM43438 wireless LAN and Bluetooth Low Energy (BLE) on board
• 40-pin extended GPIO
• 4 USB 2 ports
• 4 Pole stereo output and composite video port
• Full size HDMI
• CSI camera port for connecting a Raspberry Pi camera
• DSI display port for connecting a Raspberry Pi touchscreen display
• Micro SD port for loading your operating system and storing data
• Switched Micro USB power source up to 2.5A

Raspberry PI Zero

In addition to the PI 3’s another interesting platform is the PI Zero, which is a very low-cost system that can serve as IoT board. However, it is also powerful enough to run more sophisticated applications on it. The newest Raspberry PI Zero is shown in Figure 4.

Figure 4: Raspbery Pi Zero ²

The board has the following properties:

• 1GHz single-core CPU
• 512MB RAM
• Mini HDMI port
• Micro USB OTG port
• Micro USB power
• HAT-compatible 40-pin header
• Composite video and reset headers
• CSI camera connector (v1.3 only)

Pin Layout

The PI 4B, 3B+, 3B, and Zero come with a number of pins that can be used to attach sensors. It is convenient to have the pinout available for your project. Hence, we provide a pinout layout in Figure 5. Other Pis will have a different pinout and you will have to locate them on the internet.

Figure 5. Pi pinout

Mounting diagram

In case you need to drill wholes in cases or plexiglas for mounting, the following mounting diagram is very useful:

• Mounting Diagram in PDF

Resources

Detailed information about it are available at

• https://www.raspberrypi.org/documentation/hardware/raspberrypi/README.md

3.2 - Pi Clusters on the Internet

There is a large number of projects related to creating Pi clusters on the internet. They vary in size and software installed on them. Naturally a Pi cluster is a useful training and development environment for research organizations and thus many bigger projects are located at universities as well as government labs. However, we also have many projects done by enthusiasts.

We distinguish two different efforts. First, we often find projects that target the creation of cases for such clusters and second, we find projects that develop software fort these clusters. This section will provide an overview of these activities and provide links to these activities.

Cluster Cases

Lego

When looking at the placement of the wholes on the Raspberry Pi, the width between the wholes on the small side seems to be exactly 7 Lego Technic beam wholes apart. This has the advantage that one could build a quick frame form Lego pieces such as a

• 2 * 11 ($0.22) piece [link]
• 2 * two piece * 4 * 4 ($0.192) [link]
• A number of connector pins [link] [link]

The cost is about $0.25 per piece = $2.74 per pi.

So if we are having 100 pis we end up with $274. However we also need still to get screw and Lego connectors which we at this time have not counted and included in this calculation.

Naturally Lego’s have been explored by others

• Images on google
• University of Southampton: [link] Instructions are no longer at the original link
• Lego Technic: [link], [link]
• Lego Technic [link]
• Old style Lego [link]

Other ideas using Lego’s include:

• Compact case: Single board No screws [link]
• Zero 3D print Thingverse [link]
• B+ 3D Pi case [link]

Other interesting but not cluster related links include

• Lego hat [link]
• Brick Pi [link]

Beast by resion.io

This company has provided some larger designs for Raspberry Pi clusters and tries to create a modular system to put a number of Pis on plates that than can be connected.

• Beast 3 [link]
• Beast 2 [link]
• Beast 2 Tile [link]

Clusters

Additional links which could be useful include:

• SDSC Raspberry Pi2 MPI and Tiled Wall Viz Cluster [link]
• Six Common Errors When Building a Raspberry Pi Cluster [link]
• 5 Most Popular Raspberry Pi Cluster Supercomputer Projects [link]
• 10 amazing Raspberry Pi clusters [link]
• Build an OctaPi [link]

An older document on how to create an MPI cluster is located at

• Pi 2 MPI cluster, Boise State [link]

Swarm

How to set up docker swarm is documented here

• Pi Docker Cluster [link]

3.3 - Cases: Pi Cluster Cases

NAS (1 Pi)

Although a NAS is not really a compute cluster the Pi has used many times to build a Network Attached Storage (NAS) server. In this configuration a HDD is attached to the Raspberry and the network features of the Raspberry is used to access the disk drive via software installed on the PI that make this easily possible. Many tutorials exists on the Web that help setting op such a device.

We like to hear from you if you have successfully developed such a NAS and provide us with such links. Links that may help include:

• https://hackmypi.com/NASpi.php

ClusterHat (4 Zero + 1 Pi)

The smallest cluster we came across is actually a hybrid cluster in which 4 Pi zeros attached to a Raspberry Pi 3. This sis achieved via an add on board to the Pi 3 allowing to plug in PI=i Zeros:

• https://clusterhat.com/

The Cluster HAT (Hardware Attached on Top) allows to attach 4 Raspberry Pi Zeros via to be attached to a regular Raspberry PI 3 to simulate a small cluster.

According to the Web Site it supports the following features:

• USB Gadget Mode: Ethernet and Serial Console.
• Onboard 4 port USB 2.0 hub.
• Raspberry Pi Zeros powered via Controller Pi GPIO (USB optional).
• Individual Raspberry Pi Zero power controlled via the Controller Pi GPIO (I2C).
• Connector for Controller Serial Console (FTDI Basic).
• Controller Pi can be rebooted without interrupting power to Pi Zeros (network recovers on boot).

Although this setup seems rather appealing, the issue is with obtaining Pi Zeros for the regional price of $5. Typically users can only by one for that price and must pay shipping. To by more one has to buy a kit for about $20. However, for that amount of money it may just be worth while to get Pi 3’s instead of zero’s. Nevertheless the form factor is rather appealing.

Additional information can be found at:

• https://www.raspberrypi.org/magpi/clusterhat-review-cluster-hat-kit/

Cluster Case With Cooling (5 Pi’s)

Many instructions on the Web exist describing how to build clusters with 3 or more Pi’s. One of the considerations that we have to think about is that we may run rather demanding applications on such clusters causing heat issues. To eliminate them we must provide proper cooling. In some cluster projects cooling is not adequately addressed. Hence we like to provide an example that discusses in detail how to add a fan and what the fan has for an impact on the temperature.

• http://climbers.net/sbc/add-fan-raspberry-pi/
• http://climbers.net/sbc/diy-raspberry-pi-3-cluster/

From the previous Web page we find the following information as shown in the following table. From the data in the table it is clear that we need to keep the Pi from throttling while being in a case by adding a fan as obvious from experiment No. 2.

Table: Temperature comparison of fan impact

No. Case Fan Direction RPM Idle 100% Load Performance

1 no no - - 41.0C 75.5C OK (barely) 2 yes no - - 45.0C 82.5C throttles 3 yes 5V in unknown 37.9C 74.5C OK (barely) 4 yes 7V in 800 35.6C 69.5C OK 5 yes 12V in 1400 32.5C 61.1C OK 6 yes 7V out 800 34.5C 66.4C OK

Interesting is also the design of the case that uses snaps instead of screws to affix the walls to each other. The case layout can be found at:

• http://climbers.net/sbc/diy-raspberry-pi-3-cluster-2017/

Octapi (8 Pi’s)

A set of instructions on building an is available at

• https://projects.raspberrypi.org/en/projects/build-an-octapi

Bitscope Case (40 Pi’s)

A company from Australia called BitScope Designs offers a number of cases that leverage their Pi Blade boards allowing up to four Pis to be put together and sharing the same power supply. The blades are shown in the next Figure. The rack to place 10 of them is shown in the Figure after that.

The cost of the blade rack is $ 795.45 + $60.00 shipping + import tax. This may originally sound expensive when compared to a single case, however as we can store 40 Pis in them and they can share the power-supply and reduce cabling we think this case is quite interesting overall due to its price-point of $20 per Pi.

BitScope Cluster (144 Pi’s)

• https://www.youtube.com/watch?v=78H-4KqVvrg

Together with LANL a new cluster module that holds 144 Pis is developed. This system is targeted to be placed into a rack to create a large Pi cluster. The cost for such a module is about $15K.

The next Figure shows the module.

The next figure shows how multiple modules can be placed into a single rack.

Additional information about this form factor can be found at the following links:

• https://cluster.bitscope.com/solutions
• https://www.pcper.com/news/General-Tech/BitScope-Unveils-Raspberry-Pi-Cluster-2880-CPU-Cores-LANL-HPC-RD
• http://my.bitscope.com/store/
• http://my.bitscope.com/store/?p=view\&i=item+7
• http://www.newark.com/bitscope/bb04b/quattro-pi-board-raspberry-pi/dp/95Y0643
• http://linuxgizmos.com/rpi-expansion-boards-support-up-to-40-pi-clusters/

Oracle Cluster (1060 Pi’s)

Oracle has displayed at Oracle World 2019 a 1060 node Raspberry Pi Cluster.

More images are available at this (link)[https://imgur.com/gallery/wx1hZ5D]. the supercomputer features scores of racks with 21 Raspberry Pi 3 B+ boards each. The system is used to demonstrate Oracle Autonomous Linux source.

Build Your Own 5 Node Pi Cluster

To experiment with building an elementary cluster one does not need to have a big budget. Such clusters are often dedicated to research tasks and are bound into security protocols that do not allow direct access. Instead it is possible to build such a cluster based on Raspberry Pi’s yourself if you are willing to spend the money or if you have access to Pi’s that you may loan from your department.

Table Parts lists one such possible parts list that will allow you to build a cluster for up to 5 nodes. However make sure to buy at least 3 Raspberry Pi’s with the appropriate memory. At minimum we recommend you get the 32GB SD card. We do not recommend any smaller as otherwise you will run out of memory. Additionally, you can add memory and disks on the USB ports. If you attach a HDD, make sure it has an external power supply and do not drive it from the USB power as otherwise the PI becomes unstable. A fan is at this time not yet included.

Naturally it is possible to modify the parts list and adapt. If you find better parts let us know. We have not included any case and you are welcome to share your suggestions with the class. For a case we are looking also for a good solution for a fan.

We suggest that when you build the cluster to do it on a table with a large white paper or board, or a tablecloth and take pictures of the various stages of the build so we can include it in this document.

Initially we just put Raspbian as Operating system on the SD cards and test out each PI. To do so you will naturally need an SD card writer that you can hook up to your computer if it does not have one. As you will have to potentially do this more than once it is not recommended to buy an SD card with the OS on it. Buy the SD card writer instead so you can redo the flashing of the card when needed. In addition to the SD card you need a USB mouse and keyboard and a monitor or TV with HDMI port.

Locate setup instructions and write a section in markdown that we will include here once it is finished. The section is to be managed on github.

¹ items were replaced with similar ² item was not available

Assembling the Pi Cluster

Figure	Description
	First, aluminum and copper heat syncs need to be attached to each Pi. The two aluminum heat syncs are attached to the Broadcom chip and the SMSC Ethernet controller located on the top of the Pi. The blades of the heat syncs are parallel to the longer side of the Pi as shown in black aluminum fanned heat syncs are attached to the top of the pi as shown.
	Flat copper heat sync is attached to the bottom of the pi as shown.
	After attaching the heat syncs, threaded hexagonal spacer supports are used to connect the Pis together. A fully-assembled 5-node Pi cluster is shown.
	Each node of the cluster is then attached to the switch using an Ethernet cables and to the power supply using a USB cables. The fully wired cluster is shown.

Virtual Raspberry Cluster

It is possible to create a virtual raspberry PI cluster while for example using virtual box. This requires two steps. First the deployment of a virtualized Raspberry PI. There are many resources on the internet that describe how to do that. Some of them are

• https://raspberrytips.com/run-raspberry-in-virtual-machine/
• http://dbakevlar.com/2015/08/emulating-a-raspberry-pi-on-virtualbox/

The next step includes the deployment of multiple VMs emulating Raspberry’s. Naturally each should have its own name so you can distinguish them. Instead of just using the GUI, it would be important to find out how to start them from a command line as a shell script as well as tear them down.

Next you will need to make sure you can communicate from the Pi’s to each other. This is naturally the same as on a real cluster

Note: We recently set up such a cluster and found it was extremely slow on an older MacBook Pro with 16GB memory. Thus make sure you have up to date hardware and lots of memory. For this reason we recommend doing a hradware version. Also remember that some software does require a lot of memory to be allocated to the VM. Make sure that you plan ahead before you start the virtualization route. It will be frustrating to work on a real slow virtual cluster.

3.4 - Parts

The parts are collected in a YAML file located at:

You can add parts you used by doing a pull request. Please also send us a picture of your cluster.

3.4.1 - Individual Parts

Parts

This section will have the list of tools and parts that we recommend you get to assemble a PI cluster

To add parts please visit the yaml file and add them to the parts list

Prices are in dollar as found at the time by an online retailers. We list only the cheapest prices found. Price updates may be listed in comments. We may round to the next largest dollar amount.

	vendor	description	included	price	comment	image
0	Cana Kit	Raspberry Pi 4 8GBSKU: PI4-8GB	1	75
1	Amazon	Heat Sinks	12	12.99
2	Amazon	SD Card 64GB EVO Select	1	9.99
3	Amazon	SanDisk 32GB Ultra microSDHC	1	10.99	Only buy those for PI3B+, for 4 use the cheaper afaster and bigger 64GB cards
4	Amazon	Duo SD Card Reader	1	7.99
5	Amazon	Compatible with Raspberry Pi 4 Power Supply with ON/Off Switch	1	9.99
6	Amazon	Compatible with Raspberry Pi 4 Power Supply with ON/Off Switch	1	9.99	cable is build in the utronics seems betetr as we can take it out
7	Amazon	USB Charger 10-Port 120W	1	39.99	Warning when using the power switsh listed in the table, you must always switch it on and off from the build in switch. Never put the power supply on a power strip that has its own switch and use that switch to switch on and off via the power supply and leave the switch on the supply just open. THis coudl destroy your supply.
8	Amazon	Anker 60W 6-Port USB Wall Charger	1	27.99	pay attention to get as much W as possible
9	Amazon	60 Ports USB Charging Station	1	119.99
10	Amazon	Cable with switch	2	8.89	has 2 cables included and as the 3 cable option is not available making tis a good option
11	Amazon	Power USB Switch Type-C Cable	1	5.89	expensive
12	Amazon	Cable with switch	1	9.99	3 pack, unavailable
13	Amazon	4 Pack Cable with switch	4	12	these are from USB-C to USB-C to go to a USB charger you stil need a cable or converter to USB-A
14	Amazon	Individual power switches	6	19.99	Please be casefull using them as they may lead to damage to PI power supplies in case of power spickes.
15	Amazon	50 Pcs M2 X 20mm + 5mm Hex Hexagonal Threaded Spacer	50	11.49	fit very good, a bit thin. This is the cheapest solution
16	Amazon	Spacers	50	11.85	slightoy too big screw portion, will work, but buy oothers next time
17	Amazon	HONJIE M2.5 x 11mm	50	7.56	2 of them stacked are needed between two Pis
18	Amazon	Standoffs	10	5.49
19	Amazon	CanaKit Raspberry Pi 4 Micro HDMI Cable - 6 Feet (Pack of 2)	2	9.99
20	Amazon	TP-Link 8 Port Gigabit Ethernet Network Switch	1	19.99
21	Amazon	Ethernet cable	6	6.64
22	Amazon	Case	1	59.39	The price of this case has significantly increase and is now about $139. Find alternatives. This case is used to custom fit multiple Pis. Dependent on How many PIs you fit in, buy 90 degree power cables. And get 2 feet versions or longer. To take the case shelf out you need to drill out the rivets ifrom the shelf. This case has a swing door for easy access. You wll need a USB to 12v converter to use the fans or a separate power supply for it. It may be easier to buy the $50 case without fans and buy 120mm 5V fans that you can directly attach to the USB. This case is big.
23	Amazon	Fan cable	1	10.99
24	Amazon	Case 8 layers	1	40	see commend in the 8 layer case
25	Amazon	Case 8 layers	1	64.99	not recommend due to to not able to use the faster SD Cards, works well for PI3B+. For PI4 and 64GB card the voltage of the SDCard needs to be increased, but that means its slower
26	Amazon	Case 8 layers	1	90	see comment in 8 layer case
27	Amazon	Case 8 layers	1	12	Heat Sinks and fans included
28	Amazon	Cloudlet Case, 7 Pi	1	70
29	Amazon	Utronics 1U Rack mount, 4 Pi’s	1	40
30	Amazon	Utronics 1U Rack mount with HDMI redirection, 4 Pi’s	1	60
31	Amazon	UCTRONICS 19 inch 3U Rack Mount	1	125	Comes only 8 trays that costs $100 and will have an open gap if you do not buy an extra 4 trays for $25. The frame can hold 12 trays
32	Amazon	MyElectronics 3U Rackmount, 12 Pi’s	1	139	see also UTRONICS

3.4.2 - Parts for a 3 node cluster

Parts

This section will have the list of tools and parts that we recommend you get to assemble a PI cluster

Note: If you at a later time like to update to more than 6 nodes, you need a different power supply or buy a second one of these. See our 8 node cluster list for an alternative.

To add parts please visit the yaml file and add them to the parts list

Prices are in dollar as found at the time on online retailers.

https://github.com/cloudmesh/pi/blob/main/parts/README-parts-list.yml

	vendor	description	included	price	count	total	comment	image
0	Cana Kit	Raspberry Pi 4 8GBSKU: PI4-8GB	1	75.0	3	225.0
1	Amazon	Heat Sinks	1	12.99	1	12.99
2	Amazon	SD Card 64GB EVO Select	1	9.99	3	29.97
3	Amazon	Duo SD Card Reader	1	7.99	1	7.99
4	Amazon	Compatible with Raspberry Pi 4 Power Supply with ON/Off Switch	1	9.99	3	29.97
5	Amazon	50 Pcs M2 X 20mm + 5mm Hex Hexagonal Threaded Spacer	50	11.49	1	11.49	fit very good, a bit thin
6	Amazon	CanaKit Raspberry Pi 4 Micro HDMI Cable - 6 Feet (Pack of 2)	2	9.99	1	9.99
7	Amazon	TP-Link 8 Port Gigabit Ethernet Network Switch	1	19.99	1	19.99
8	Amazon	Ethernet cable	6	6.64	1	6.64
9						========
10						354.03

3.4.3 - Parts for a 8 node cluster

Parts

This section will have the list of tools and parts that we recommend you get to assemble a PI cluster

Prices are in dollar as found at the time on online retailers.

To add parts please visit the yaml file and add them to the parts list

	vendor	description	included	price	count	total	comment	image
0	Cana Kit	Raspberry Pi 4 8GBSKU: PI4-8GB	1	75.0	8	600.0
1	Amazon	Heat Sinks	12	12.99	1	12.99
2	Amazon	SD Card 64GB EVO Select	1	9.99	8	79.92
3	Amazon	SanDisk 32GB Ultra microSDHC	1	10.99	8	87.92	Only buy those for PI3B+, for 4 use the cheaper afaster and bigger 64GB cards
4	Amazon	Duo SD Card Reader	1	7.99	1	7.99
5	Amazon	USB Charger 10-Port 120W	1	39.99	1	39.99	Warning when using the power switsh listed in the table, you must always switch it on and off from the build in switch. Never put the power supply on a power strip that has its own switch and use that switch to switch on and off via the power supply and leave the switch on the supply just open. THis coudl destroy your supply.
6	Amazon	Cable with switch	2	8.89	4	35.56	has 2 cables included and as the 3 cable option is not available making tis a good option
7	Amazon	CanaKit Raspberry Pi 4 Micro HDMI Cable - 6 Feet (Pack of 2)	2	9.99	1	9.99
8	Amazon	TP-Link 8 Port Gigabit Ethernet Network Switch	1	19.99	1	19.99
9	Amazon	Ethernet cable	6	6.64	2	13.28
10						========
11						907.63

4 - Software

4.1 - Autogenerating Analytics Rest Services

1. Prerequisite

It is also assumed that the user has installed and has familiarity with the following:

• Install cloudmesh-openapi using the developer install as documented here
• Python 3.8.x
• Linux Command line
• Working in a python environment

2. The Python Code

First, let us ensure we are in the correct directory. If you followed the cloudmesh-openapi installation directions as dictated in the installation guide, simply navigate to the root directory of cloudmesh-openapi. Notice how we are still working in our python virtual environment ENV3 from the installation guide.

(ENV3) > pwd
~/cm/cloudmesh-openapi

Let us take a look at the PipelineAnova SVM example code.

A Pipeline is a pipeline of transformations to apply with a final estimator. Analysis of variance (ANOVA) is used for feature selection. A Support vector machine SVM is used as the actual learning model on the features.

Use your favorite editor to look at it (whether it be vscode, vim, nano, etc). We will use emacs

(ENV3) > emacs ./tests/Scikitlearn-experimental/sklearn_svm.py

The class within this file has two main methods to interact with (except for the file upload capability which is added at runtime)

@classmethod
def train(cls, filename: str) -> str:
    """
    Given the filename of an uploaded file, train a PipelineAnovaSVM
    model from the data. Assumption of data is the classifications 
    are in the last column of the data.

    Returns the classification report of the test split
    """
    # some code...

@classmethod
def make_prediction(cls, model_name: str, params: str):
    """
    Make a prediction based on training configuration
    """
    # some code...

Note the parameters that each of these methods takes in. These parameters are expected as part of the stateless request for each method.

3. Generating the OpenAPI YAML file

Let us now use the python code from above to create the openapi YAML file that we will deploy onto our server. To correctly generate this file, use the following command:

(ENV3) > cms openapi generate PipelineAnovaSVM 
    \ --filename=./tests/Scikitlearn-experimental/sklearn_svm.py
    \ --import_class
    \ --enable_upload

Let us digest the options we have specified:

• --filename indicates the path to the python file in which our code is located
• --import_class notifies cms openapi that the YAML file is generated from a class. The name of this class is specified as PipelineAnovaSVM
• --enable_upload allows the user to upload files to be stored on the server for reference. This flag causes cms openapi to auto-generate a new python file with the upload method appended to the end of the file. For this example, you will notice a new file has been added in the same directory as sklearn_svm.py. The file is aptly called: sklearn_svm_upload-enabled.py

4. The OpenAPI YAML File (optional)

If Section 2 above was correctly, cms will have generated the corresponding openapi YAML file. Let us take a look at it.

(ENV3) > emacs ./tests/Scikitlearn-experimental/sklearn_svm.yaml

This YAML file has a lot of information to digest. The basic structure is documented here. However, it is not necessary to understand this information to deploy RESTful APIs.

However, take a look at paths: on line 9 in this file. Under this section, we have several different endpoints for our API listed. Notice the correlation between the endpoints and the python file we generated from.

5. Starting the Server

Using the YAML file from Section 2, we can now start the server.

(ENV3) > cms openapi server start ./tests/Scikitlearn-experimental/sklearn_svm.yaml

The server should now be active. Navigate to http://localhost:8080/cloudmesh/ui.

6. Interacting With the Endpoints

6.1 Uploading the Dataset

We now have a nice user inteface to interact with our newly generated API. Let us upload the data set. We are going to use the iris data set in this example. We have provided it for you to use. Simply navigate to the /upload endpoint by clicking on it, then click Try it out.

We can now upload the file. Click on Choose File and upload the data set located at ./tests/Scikitlearn-experimental/iris.data. Simply hit Execute after the file is uploaded. We should then get a 200 return code (telling us that everything went ok).

6.2 Training on the Dataset

The server now has our dataset. Let us now navigate to the /train endpoint by, again, clicking on it. Similarly, click Try it out. The parameter being asked for is the filename. The filename we are interested in is iris.data. Then click execute. We should get another 200 return code with a Classification Report in the Response Body.

6.3 Making Predictions

We now have a trained model on the iris data set. Let us now use it to make predictions. The model expects 4 attribute values: sepal length, seapl width, petal length, and petal width. Let us use the values 5.1, 3.5, 1.4, 0.2 as our attributes. The expected classification is Iris-setosa.

Navigate to the /make_prediction endpoint as we have with other endpoints. Again, let us Try it out. We need to provide the name of the model and the params (attribute values). For the model name, our model is aptly called iris (based on the name of the data set).

As expected, we have a classification of Iris-setosa.

7. Clean Up (optional)

At this point, we have created and trained a model using cms openapi. After satisfactory use, we can shut down the server. Let us check what we have running.

(ENV3) > cms openapi server ps
openapi server ps

INFO: Running Cloudmesh OpenAPI Servers

...

[{'name': 'sklearn_svm', 'pid': 7496, 'spec': './tests/Scikitlearn-experimental/sklearn_svm.yaml'}]
+-------------+------+--------------------------------------------------+
| name        | pid  | spec                                             |
+-------------+------+--------------------------------------------------+
| sklearn_svm | 7496 | ./tests/Scikitlearn-                             |
|             |      | experimental/sklearn_svm.yaml                    |
+-------------+------+--------------------------------------------------+

We can stop the server with the following command:

(ENV3) > cms openapi server stop sklearn_svm

We can verify the server is shut down by running the ps command again.

(ENV3) > 
openapi server ps

INFO: Running Cloudmesh OpenAPI Servers

[]
None

8. Assignments

Many ML models follow the same basic process for training and testing:

1. Upload Training Data
2. Train the model
3. Test the model

Using the PipelineAnovaSVM code as a template, write python code for a new model and deploy it as a RESTful API as we have done above. Train and test your model using the provided iris data set. There are plenty of examples that can be referenced here

9. References

• SciKit-Learn: Machine learning in Python, 0.24.1, Feb 2021

4.2 - Link Collection abut PI Clusters and Parts

Software

OS

• RaspberyOS, 32 bit
• Ubuntu
• Ubuntu 20.04 Server, 64bit
• Ubuntu 20.10 Server, 64bit
• Ubuntu 20.10 Desktop, 64bit

Booting and Startup

• USB mass storage boot, raspberypi.org
• Five Ways to Run a Program On Your Raspberry Pi At Startup
• How to Run a Script at Boot on Raspberry Pi, Tom’s Hardware

SDCard Creation

• Cludhmesh Pi Burn Very good for burning multiple cars for clusters with easy setup of a cluster of PIs that are fully connected with each other.
• Pi Imager, raspberrypi.org Best suited for burning individuall cards. Requires additional work after burning, such as setting hostname, and enable ssh. Time consuming when setting up a cluster.
• Pi Bakery Easy setup of individual Pis, with the ability to add post instalation information such as Wifi, hostname and others. GUI driven. Not targeted towards Cluster setups.

PXE Boot

• Pi4 Network booting, raspberrypi.org
• Tutoria Network boot your Raspberry Pig, raspberrypi.org

Network of Workstations

• Cludhmesh Pi Burn Setting up a cluster of Pis easily including ssh, keys, and networking.

SLURM

• Slurm on Pi zero, raspberrypi.org Forum post

MPI

• Build a Raspberry Pi cluster computer, The MagPi magazine
• Installing MPI for Python on a Raspberry Pi Cluster, The New Stack
• Building a Raspberry Pi Cluster. Part III - OpenMPI, Python, and Parallel, Garrett Mills, Medium Medium requires a subscription if you read more than a small number of articles

Kubernetes

• see our special section on kubernetes

Network Configurations

mDNS

• Going headless with Avahi Zeroconf MDNS (Pi4, maybe Pi 3B, etc) - Raspberry Pi Forums

Mesh Network

• Workshop to create a sensor application over a WiFi Mesh network, binnes, GitHub,
• DIY: How to Create a Home Mesh WiFi using Raspberry Pi | iotTrends.tech

4.3 - Cloudmesh Pi Burner

WARNING: This program is designed for a Raspberry Pi. Instructions to use Linux are included in the FAQ. We are working on support for macOS and Windows 10. If you want to help us port to any of these OSes, please contact laszewski@gmail.com

• Cloudmesh Pi Burner for SD Cards
  • Introduction
  • Nomenclature
  • Quickstart for Bridged WiFi
    • Requirements
    • Manager Pi
    • Burning Multiple SD Cards with a Single Burner
    • Connecting Pis to the Internet via Bridge
  • Set up of the SSH keys and SSH tunnel
  • Manual Pages
    • Manual Page for the burn command
    • Manual Page for the bridge command
    • Manual Page for the host command
    • Manual Page for the pi command
  • FAQ and Hints
    • Can I use the LEDs on the PI Motherboard?
    • How can I use pycharm, to edit files or access files in general from my Laptop on the PI?
    • How can I enhance the get script?
    • Can I use a Mesh Network for the setup?
    • Can I use cms burn on Linux?
    • What packages do I need to run the info command on macOS
    • Are there any unit tests?
    • Using Pi Imager to setup a Manager Pi with headless access
    • Single Card Burning
    • How to update firmware?
    • How to burn a cluster using Linux
  • Alternatives
    • What is the status of the implementation?
  • How can I contribute Contributing

Introduction

cms burn is a program to burn many SD cards for the preparation of building clusters with Raspberry Pi’s. It allows users to create readily bootable SD cards that have the network configured and contain a public ssh key from your machine that you used to configure the cards. Thus, little setup is needed for a cluster. Another unique feature is that you can burn multiple cards in a row, each with their individual setup such as hostnames and ipadresses.

Nomenclature

• Commands proceeded with pi@mangerpi:$ are to be executed on the Rasperry Pi with the name managerpi.

• Commands with (ENV3) pi@managerpi:$ are to be executed in a virtula ENV using Python 3 on the Raspberry Pi with the name managerpi

Quickstart for Bridged WiFi

To provide you with a glimpse on what you can do with cms burn, we have provided this quickstart guide that will create one manager PI and several workers.

This setup is intended for those who have restricted access to their home network (ie. cannot access router controls). For example, those on campus WiFis or regulated apartment WiFis.

The Figure 1 describes our network configuration. We have 5 Raspberry Pi 4s: 1 manager and 4 workers. We have WiFi access, but we do not necessarily have access to the router’s controls.

We also have a network switch, where the manager and workers can communicate locally, but we will also configure the manager to provide internet access to devices on the network switch via a “network bridge”.

Figure 1: Pi Cluster setup with bridge network

Requirements

For the quickstart we have the following requirements:

• SD Cards and Raspberry Pis

• You will need an SD card writer (USB-A) to burn new cards We recommend that you invest in a USB 3.0 SDCard writer as they are significantly faster and you can resuse them on PI'4s

Manager Pi

First we need to configure the Manager Pi

Step 0. Burn Manager Pi SD Card

Using Raspberry Pi imager, burn an SD card with Raspberry Pi OS (32-bit) with desktop and recommended applications. You may use your normal system to burn such a card including Windows, macOS, or Linux.

You will then want a method of accessing this manager Pi. You may either use SSH (recommended) or monitor desktop environment (easiest) to access it. We highly recommend changing the password on the Pi as soon as you have access. This is because the pi is initialized with default user pi and default password raspberry. This is critical if you are on a shared network where anyone can attempt to access your pi.

Monitor Desktop Environment: You will need a monitor, keyboard, and mouse. This is the easiest approach as Raspberry Pi OS provides a very nice user interface with an easy-to-follow setup process for connecting to WiFi and other such tasks.

SSH Environment: You may consider enabling SSH access to your Pi so that you may access the file system from your preferred machine.

Headless Configuration: See section 3 of enabling ssh for instructions on how to enable SSH headlessly. Similarly, how to enable WiFi headlessly. Additionally you can check out the FAQ for step-by-step instructions. Using Pi Imager to setup a Manager Pi with headless access.

Update the firmware: See the FAQ [How to update firmware?]> (#how-to-update-firmware)

Step 1. Installing Cloudmesh on the Manager Pi

Open a new terminal screen on the Manager Pi. Here we assume the hostname is managerpi. However, this is of no importance in relation to topics of this guide.

Update pip. The simple curl command below will generate an ssh-key, update your system, and install cloudmesh.

pi@managerpi:~ $ pip install pip -U
pi@managerpi:~ $ curl -Ls http://cloudmesh.github.io/get/pi | sh -                

This will take a moment…

Step 2. Reboot

The installation script updates your system. Reboot for effect.

pi@managerpi:~ $ sudo reboot

Step 3. Download the latest Raspberry Pi Lite OS

The following command will download the latest images for Raspberry Lite OS.

(ENV3) pi@managerpi:~ $ cms burn image get latest-lite

We can verify our image’s downloaded with the following.

(ENV3) pi@managerpi:~ $ cms burn image ls

Note. For our cluster we use light, but if you like to use other versions please see this note. We can use the following command to list the current Raspberry Pi OS versions (full and lite)

(ENV3) pi@managerpi:~ $ cms burn image versions --refresh

This will list the Tags and Types of each available OS. We can then modify the image get command for versions we are interested in. For example,

(ENV3) pi@managerpi:~ $ cms burn image get full-2020-05-28

Step 4. Setup SD Card Writer

Plug your SD Card Writer into the Pi. Ensure you have an SD Card inserted into your writer. Run the following command to find the path to your SD Card.

(ENV3) pi@managerpi:~ $ cms burn info
...
# ----------------------------------------------------------------------
# SD Cards Found
# ----------------------------------------------------------------------

+----------+----------------------+----------+-----------+-------+------------------+---------+-----------+-----------+
| Path     | Info                 | Readable | Formatted | Empty | Size             | Aaccess | Removable | Writeable |
+----------+----------------------+----------+-----------+-------+------------------+---------+-----------+-----------+
| /dev/sda | Generic Mass-Storage | True     | True      | False | 64.1 GB/59.7 GiB | True    | True      |           |
+----------+----------------------+----------+-----------+-------+------------------+---------+-----------+-----------+

cms burn info has other useful information, but for the purposes of this guide we omit it.

We can see from the information displayed that our SD card’s path is /dev/sda. Of course, this may vary.

Burning Multiple SD Cards with a Single Burner

Step 0. Ensure the first SD card is inserted into the burner.

We can run cms burn info again as we did above to verify our SD card is connected.

Step 1. Burning the Cards

cms burn supports logical incremenation of numbers/characters.

For example, red00[1-2] is interpreted by cms burn as [red001, red002]. Similarly, red[a-c] is interpreted by cms burn as [reda, redb, redc].

We can burn 2 SD cards as follows:

!! WARNING VERIFY THE DEVICE IS CORRECT. REFER TO CMS BURN !!

(ENV3) pi@managerpi:~ $ cms burn create --hostname=red00[1-4] --ip=10.1.1.[2-5] --device=/dev/sda --tag=latest-lite

The user will be prompted to swap the SD cards after each card burn if there are still remaining cards to burn.

Step 2. Boot the cluster

After all cards are burned. Turn off the cluster, insert the cards, and turn the cluster back on.

We can now proceed to the next section where we configure our bridge.

Connecting Pis to the Internet via Bridge

Figure 1 depicts how the network is set up with the help of the bridge command.

Figure 1: Networking Bridge

Step 0. Review and Setup

At this point we assume that you have used cms burn to create all SD cards for the Pi’s with static IP addresses in the subnet range 10.1.1.0/24 (excluding 10.1.1.1. See step 1 for details)

We are also continuing to use managerpi (which is where we burn the worker SD cards).

We will now use cms bridge to connect the worker Pis to the internet. Let us again reference the diagram of our network setup. You should now begin connecting your Pis together via network switch (unmanaged or managed) if you have not done so already. Ensure that managerpi is also connected into the network switch.

We will assign the eth0 interface of the managerpi to be 10.1.1.1, and it will act as the default gateway for the workers. The workers IPs were set during the create command.

Step 1. Configuring our Bridge

We can easily create our bridge as follows.

(ENV3) pi@managerpi:~ $ cms bridge create --interface='wlan0'

We should now reboot.

(ENV3) pi@managerpi:~ $ sudo reboot

Note the --interface option indicates the interface used by the manager pi to access the internet. In this case, since we are using WiFi, it is likely wlan0. Other options such as eth0 and eth1 exist for ethernet connections.

Step 2. Verifying internet connection

We should now be able to see our workers.

arp -a

Note it may take a few minutes for them to populate in the neighbor table. If you want to speed this up try to ping them individually.

ping red001

At this point, our workers should have internet access. Let us SSH into one and ping google.com to verify. Ensure you have booted your workers and connected them to the same network switch as the manager.

(ENV3) pi@managerpi:~ $ ssh red001

pi@red001:~ $ ping google.com
PING google.com (142.250.64.238) 56(84) bytes of data.
64 bytes from mia07s57-in-f14.1e100.net (142.250.64.238): icmp_seq=1 ttl=106 time=48.2 ms
64 bytes from mia07s57-in-f14.1e100.net (142.250.64.238): icmp_seq=2 ttl=106 time=48.3 ms
64 bytes from mia07s57-in-f14.1e100.net (142.250.64.238): icmp_seq=3 ttl=106 time=47.9 ms
64 bytes from mia07s57-in-f14.1e100.net (142.250.64.238): icmp_seq=4 ttl=106 time=47.10 ms
64 bytes from mia07s57-in-f14.1e100.net (142.250.64.238): icmp_seq=5 ttl=106 time=48.5 ms
^C
--- google.com `ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 9ms
rtt min/avg/max/mdev = 47.924/48.169/48.511/0.291 ms

Note how we are able to omit the pi user and .local extension. We have successfuly configured our bridge. Our pis are now ready to cluster.

Set up of the SSH keys and SSH tunnel

One important aspect of a cluster is to setup authentication via ssh in a convenient way, so we can easily login from the laptop to each of the PI workers and the PI manager. Furthermore, we like to be able to login from the PI manager to each of the workers. In addition, we like to be able to login between the workers.

We have chosen a very simple setup while relying on ssh tunnel.

To simplify the setup of this we have developed a command cms host that gathers and scatters keys onto all machines, as well as sets up the tunnels.

It is essential that that the key on the laptop must not be password less. This is also valid for any machine that is directly added to the network such as in the mesh notwork.

To avoid password less keys we recommend you to use ssh-add or ssh-keychain which will ask you for one.

The manual page for cms host is provided in the Manual Pages section.

Step 1. On the manager create ssh keys for each of the workers.

(ENV3) pi@managerpi:~ $ cms host key create red00[1-3]

Step 2. On the manager gather the worker, manager, and your laptop public ssh keys into a file.

(ENV3) pi@managerpi:~ $ cms host key gather red00[1-3],you@yourlaptop.local keys.txt

Step 3. On the manager scatter the public keys to all the workers and manager ~/.ssh/authorized_hosts file

(ENV3) pi@managerpi:~ $ cms host key scatter red00[1-3],localhost keys.txt

Step 4. Remove undeeded keys.txt file

(ENV3) pi@managerpi:~ $ rm keys.txt

Step 5. Verify SSH reachability from worker to manager and worker to worker.

(ENV3) pi@managerpi:~ $ ssh red001
pi@red001:~ $ ssh managerpi.local

BUG: The workers still currently still need to use .local after names. This will be resolved by the inventory create update.

(ENV3) pi@managerpi:~ $ exit
pi@red001:~ $ ssh red002.local
pi@red002:~ $ exit
pi@red001:~ $ exit

Step 6. (For Bridge setup) Create SSH tunnels on the manager to enable ssh acces from your laptop to the workers

For now we manually install autossh, to test the new cms host tunnel program. Later we add it to the main manager setup script.

(ENV3) pi@managerpi:~ $ yes y | sudo apt install autossh
(ENV3) pi@managerpi:~ $ cms host tunnel create red00[1-3]

Step 7. (For Bridge setup) Copy the specified command output to your ~/.ssh/config file on your laptop

host tunnel create red00[1-3]

Using wlan0 IP = 192.168.1.17
Using cluster hostname = managerpi

Tunnels created.

Please place the following in your remote machine's (i.e. laptop)
`~/.ssh/config` file to alias simple ssh access (i.e. `ssh red001`).

# ----------------------------------------------------------------------
# copy to ~/.ssh/config on remote host (i.e laptop)
# ----------------------------------------------------------------------

Host red001
     HostName managerpi.local
     User pi
     Port 8001

Host red002
     HostName managerpi.local
     User pi
     Port 8002

Host red003
     HostName managerpi.local
     User pi
     Port 8003

Note: We will in future provide an addition to the command so you can remove and add them directly from the commandline

cms host tunnel config create red00[1-3]
cms host tunnel config delete red00[1-3]

Step 8. (For Bridge setup) Verify SSH reachability from the laptop to workers

you@yourlaptop:~ $ ssh red001

Manual Pages

Manual Page for the burn command

Note to execute the command on the commandline you have to type in cms burn and not jsut burn.

  burn firmware check
  burn firmware update
  burn install
  burn load --device=DEVICE
  burn format --device=DEVICE
  burn imager [TAG...]
  burn mount [--device=DEVICE] [--os=OS]
  burn unmount [--device=DEVICE] [--os=OS]
  burn network list [--ip=IP] [--used]
  burn network
  burn info [--device=DEVICE]
  burn image versions [--refresh] [--yaml]
  burn image ls
  burn image delete [--image=IMAGE]
  burn image get [--url=URL] [TAG...]
  burn backup [--device=DEVICE] [--to=DESTINATION]
  burn copy [--device=DEVICE] [--from=DESTINATION]
  burn shrink [--image=IMAGE]
  burn create [--image=IMAGE]
              [--device=DEVICE]
              [--hostname=HOSTNAME]
              [--ip=IP]
              [--sshkey=KEY]
              [--blocksize=BLOCKSIZE]
              [--dryrun]
              [--passwd=PASSWD]
              [--ssid=SSID]
              [--wifipassword=PSK]
              [--format]
              [--tag=TAG]
              [--inventory=INVENTORY]
              [--name=NAME]
  burn sdcard [TAG...] [--device=DEVICE] [--dryrun]
  burn set [--hostname=HOSTNAME]
           [--ip=IP]
           [--key=KEY]
  burn enable ssh
  burn wifi --ssid=SSID [--passwd=PASSWD] [--country=COUNTRY]
  burn check [--device=DEVICE]
  burn mac --hostname=HOSTNAME

Options:
  -h --help              Show this screen.
  --version              Show version.
  --image=IMAGE          The image filename,
                         e.g. 2019-09-26-raspbian-buster.img
  --device=DEVICE        The device, e.g. /dev/sdX
  --hostname=HOSTNAME    The hostname
  --ip=IP                The IP address
  --key=KEY              The name of the SSH key file
  --blocksize=BLOCKSIZE  The blocksise to burn [default: 4M]

Arguments:
    TAG                  Keyword tags to identify an image
                         [default: latest]
Files:
  This is not fully thought through and needs to be documented
  ~/.cloudmesh/images
    Location where the images will be stored for reuse

Description:
    cms burn create --inventory=INVENTORY --device=DEVICE --name=NAME

        Will refer to a specified cloudmesh inventory file (see cms help inventory).
        Will search the configurations for NAME inside of INVENTORY and will burn
        to DEVICE. Supports parameter expansion.

    cms burn create --passwd=PASSWD

         if the passwd flag is added the default password is
         queried from the commandline and added to all SDCards

         if the flag is omitted login via the password is
         disabled and only login via the sshkey is allowed

  Network

    cms burn network list

        Lists the ip addresses that are on the same network

         +------------+---------------+----------+-----------+
         | Name       | IP            | Status   | Latency   |
         |------------+---------------+----------+-----------|
         | Router     | 192.168.1.1   | up       | 0.0092s   |
         | iPhone     | 192.168.1.4   | up       | 0.061s    |
         | red01      | 192.168.1.46  | up       | 0.0077s   |
         | laptop     | 192.168.1.78  | up       | 0.058s    |
         | unkown     | 192.168.1.126 | up       | 0.14s     |
         | red03      | 192.168.1.158 | up       | 0.0037s   |
         | red02      | 192.168.1.199 | up       | 0.0046s   |
         | red        | 192.168.1.249 | up       | 0.00021s  |
         +------------+----------------+----------+-----------+

    cms burn network list [--used]

        Lists the used ip addresses as a comma separated parameter
        list

           192.168.50.1,192.168.50.4,...

    cms burn network address

        Lists the own network address

         +---------+----------------+----------------+
         | Label   | Local          | Broadcast      |
         |---------+----------------+----------------|
         | wlan0   | 192.168.1.12   | 192.168.1.255  |
         +---------+----------------+----------------+

    cms burn firmware check

        checks if the firmware on the Pi is up to date

    cms burn firmware update

        checks and updates the firmware on the Pi

    cms burn install

        installs a program to shrink img files. THis is
        useful, after you created a backup to make the
        backup smaller and allow faster burning in case of
        recovery

    cms burn load --device=DEVICE

        loads the sdcard into the USB drive. Thi sis similar to
        loading a cdrom drive. It s the opposite to eject

    cms burn format --device=DEVICE

        formats the SDCard in the specified device. Be
        careful it is the correct device.  cms burn info
        will help you to identifying it

    cms burn mount [--device=DEVICE] [--os=OS]

        mounts the file systems available on the SDCard

    cms burn unmount [--device=DEVICE] [--os=OS]

        unmounts the mounted file systems from the SDCard

    cms burn info [--device=DEVICE]

        provides useful information about the SDCard

    cms burn image versions [--refresh] [--yaml]

        The images that you like to burn onto your SDCard
        can be cached locally with the image command.  The
        available images for the PI can be found when
        using the --refresh option. If you do not specify
        it it reads a copy of the image list from our
        cache

    cms burn image ls

        Lists all downloaded images in our cache. You can
        download them with the cms burn image get command

    cms burn image delete [--image=IMAGE]

        deletes the specified image. The name can be found
        with the image ls command

    cms burn image get [--url=URL] [TAG...]

        downloads a specific image or the latest
        image. The tag are a number of words separated by
        a space that must occur in the tag that you find
        in the versions command

    cms burn backup [--device=DEVICE] [--to=DESTINATION]

        backs up a SDCard to the given location

    cms burn copy [--device=DEVICE] [--from=DESTINATION]

        copies the file form the destination on the SDCard
        this is the same as the SDCard command. we will in
        future remove one

    cms burn shrink [--image=IMAGE]

        shrinks the size of a backup or image file that
        is on your local file system. It can only be used
        for .img files

    cms burn create [--image=IMAGE]
                    [--device=DEVICE]
                    [--hostname=HOSTNAME]
                    [--ip=IP]
                    [--sshkey=KEY]
                    [--blocksize=BLOCKSIZE]
                    [--dryrun]
                    [--passwd=PASSWD]
                    [--ssid=SSID]
                    [--wifipassword=PSK]
                    [--format]

        This command  not only can format the SDCard, but
        also initializes it with specific values

    cms burn sdcard [TAG...] [--device=DEVICE] [--dryrun]

        this burns the sd card, see also copy and create

    cms burn set [--hostname=HOSTNAME]
                 [--ip=IP]
                 [--key=KEY]
                 [--mount=MOUNTPOINT]

        this sets specific values on the sdcard after it
        has ben created with the create, copy or sdcard
        command

        a --ssh is missing from this command

    cms burn enable ssh [--mount=MOUNTPOINT]

        this enables the ssh server once it is booted

    cms burn wifi --ssid=SSID [--passwd=PASSWD] [--country=COUNTRY]

        this sets the wifi ssid and password after the card
        is created, copied, or the sdcard is used.

        The option country option expects an ISO 3166-1
        two digit country code. The default is "US" and
        the option not required if suitable. See
        https://en.wikipedia.org/wiki/ISO_3166-1 for other
        countries.

    cms burn check [--device=DEVICE]

        this command lists the parameters that were set
        with the set or create command

Examples: ( \ is not shown)

   > cms burn create --image=2019-09-26-raspbian-buster-lite
   >                 --device=/dev/mmcblk0
   >                 --hostname=red[5-7]
   >                 --ip=192.168.1.[5-7]
   >                 --sshkey=id_rsa

   > cms burn image get latest

   > cms burn image get https://downloads.raspberrypi.org/
   >   raspbian_lite/images/
   >   raspbian_lite-2018-10-11/2018-10-09-raspbian-stretch-lite.zip

   > cms burn image delete 2019-09-26-raspbian-buster-lite

Manual Page for the bridge command

Note to execute the command on the commandline you have to type in cms bridge and not jsut bridge.

Options:
    --interface=INTERFACE  The interface name [default: eth1]
                           You can also specify wlan0 if you want
                           to bridge through WIFI on the manager
                           eth0 requires a USB to WIFI adapter

   --ip=IP  The ip address to assign on the eth0 interface,
            ie. the listening interface [default: 10.1.1.1]

  --dns=NAMESERVER  The ip address of a nameserver to set statically
           For example, --dns=8.8.8.8,8.8.4.4 will use google
           nameservers

Description:

  Command used to set up a bride so that all nodes route the traffic
  trough the manager PI.

  bridge create [--interface=INTERFACE] [--ip=IP] [--dns=NAMESERVER]
      creates the bridge on the current device.
      A reboot is required.

Manual Page for the host command

Note to execute the command on the commandline you have to type in cms host and not jsut host.

    host scp NAMES SOURCE DESTINATION [--dryrun]
    host ssh NAMES COMMAND [--dryrun] [--output=FORMAT]
    host config NAMES [IPS] [--user=USER] [--key=PUBLIC]
    host check NAMES [--user=USER] [--key=PUBLIC]
    host key create NAMES [--user=USER] [--dryrun] [--output=FORMAT]
    host key list NAMES [--output=FORMAT]
    host key gather NAMES [--authorized_keys] [FILE]
    host key scatter NAMES FILE
    host tunnel create NAMES [--port=PORT]
    host mac NAMES [--eth] [--wlan] [--output=FORMAT]

This command does some useful things.

Arguments:
    FILE   a file name

Options:
    --dryrun   shows what would be done but does not execute
    --output=FORMAT  the format of the output
    --port=PORT starting local port for tunnel assignment

Description:

    host scp NAMES SOURCE DESTINATION

      TBD

    host ssh NAMES COMMAND

      runs the command on all specified hosts
      Example:
           ssh red[01-10] "uname -a"

    host key create NAMES
      create a ~/.ssh/id_rsa and id_rsa.pub on all hosts specified
      Example:
          ssh key create "red[01-10]"

    host key list NAMES

      list all id_rsa.pub keys from all hosts specifed
       Example:
           ssh key list red[01-10]

    host key gather HOSTS FILE

      gathers all keys from file FILE including the one from localhost.

          ssh key gather "red[01-10]" keys.txt

    host key scatter HOSTS FILE

      copies all keys from file FILE to authorized_keys on all hosts,
      but also makes sure that the users ~/.ssh/id_rsa.pub key is in
      the file.

      1) adds ~/.id_rsa.pub to the FILE only if its not already in it
      2) removes all duplicated keys

      Example:
          ssh key scatter "red[01-10]"

    host key scp NAMES FILE

      copies all keys from file FILE to authorized_keys on all hosts
      but also makes sure that the users ~/.ssh/id_rsa.pub key is in
      the file and removes duplicates, e.g. it calls fix before upload

      Example:
          ssh key list red[01-10] > pubkeys.txt
          ssh key scp red[01-10] pubkeys.txt

    host config NAMES IPS [--user=USER] [--key=PUBLIC]

      generates an ssh config file tempalte that can be added to your
      .ssh/config file

      Example:
          cms host config "red,red[01-03]" "198.168.1.[1-4]" --user=pi

    host check NAMES [--user=USER] [--key=PUBLIC]

      This command is used to test if you can login to the specified
      hosts. It executes the hostname command and compares it.
      It provides a table  with a sucess column

      cms host check "red,red[01-03]"

          +-------+---------+--------+
          | host  | success | stdout |
          +-------+---------+--------+
          | red   | True    | red    |
          | red01 | True    | red01  |
          | red02 | True    | red02  |
          | red03 | True    | red03  |
          +-------+---------+--------+

    host tunnel create NAMES [--port=PORT]

      This command is used to create a persistent local port
      forward on the host to permit ssh tunnelling from the wlan to
      the physical network (eth). This registers an autossh service in
      systemd with the defualt port starting at 8001.

      Example:
          cms host tunnel create red00[1-3]

    host mac NAMES

      returns the list of mac addresses of the named pis.

Manual Page for the pi command

Note to execute the command on the commandline you have to type in cms pi and not jsut pi.

Note: Please note that the command hadoop, spark, and k3 are experimental and do not yet work. In fact the hadoop and spark deployment are not fullfilling our standard and should not be used. They will be put into a different command soon so they are not confusing the used in this README. The command is likely to be called pidev. Once the command is graduated it will be moved into the main command pi.

There is some very usefull aditional information about how to use the LED and temperature monitoring programs at

• https://github.com/cloudmesh/cloudmesh-pi-cluster/blob/main/README.md

  pi led reset [NAMES]
  pi led (red|green) VALUE
  pi led (red|green) VALUE NAMES [--user=USER]
  pi led list NAMES [--user=USER]
  pi led blink (red|green) NAMES [--user=USER] [--rate=SECONDS]
  pi led sequence (red|green) NAMES [--user=USER] [--rate=SECONDS]
  pi temp NAMES [--rate=RATE] [--user=USER] [--output=FORMAT]
  pi free NAMES [--rate=RATE] [--user=USER] [--output=FORMAT]
  pi load NAMES [--rate=RATE] [--user=USER] [--output=FORMAT]
  pi script list SERVICE [--details]
  pi script list SERVICE NAMES
  pi script list
  pi wifi SSID [PASSWORD] [--dryrun]

This command does some useful things.

Arguments:
    FILE   a file name

Options:
    -f      specify the file


Description:

  This command switches on and off the LEDs of the specified
  PIs. If the hostname is omitted. It is assumed that the
  code is executed on a PI and its LED are set. To list the
  PIs LED status you can use the list command

  Examples:

      cms pi led list  "red,red[01-03]"

          lists the LED status of the given hosts

      cms pi led red off  "red,red[01-03]"

          switches off the led of the given PIs

      cms pi led red on  "red,red[01-03]"

          switches on the led of the given PIs

      cms pi led red blink  "red,red[01-03]"

          switches on and off the led of the given PIs

      cms pi led red sequence  "red,red[01-03]"

          goes in sequential order and switches on and off
          the led of the given PIs

FAQ and Hints

Here, we provide some usefule FAQs and hints.

Can I use the LEDs on the PI Motherboard?

Typically this LED is used to communicate some system related information. However cms pi can controll it to switch status on and off. This is helpful if you like to showcase a particular state in the PI. Please look at the manual page. An esample is

$ cms pi led red off HOSTNAME

that when executed on the PI (on which you also must have cms installed you switch the red LED off. For more options see the manual page

How can I use pycharm, to edit files or access files in general from my Laptop on the PI?

This is easily possible with the help of SSHFS. To install it we refer you to See also: https://github.com/libfuse/sshfs SSHFS: add manager to .ssh/config onlocal machine

Let us assume you like to edit fles on a PI that you named red

Please craete a `./.ssh/config file that containes the following:

 Host red
      HostName xxx.xxx.xxx.xxx
      User pi
      IdentityFile ~/.ssh/id_rsa.pub

Now let us create a directory in which we mount the remote PI directories

mkdir manager
sshfs manager: manager -o auto_cache

How can I enhance the get script?

Instead of using the link

• http://cloudmesh.github.io/get/pi

please use

• https://raw.githubusercontent.com/cloudmesh/get/main/pi

This allows us to test also modifications to the get script before we push them to the official community repository.

You can create a pull request at

• https://github.com/cloudmesh/get/blob/main/pi/index.html

Can I use a Mesh Network for the setup?

This section is still under development.

In case you have a Mesh Network, the setup can typically be even more simplifies as we can attach the unmanaged router directly to a Mesh node via a network cable. IN that case the node is directly connected to the internet and uses the DHCP feature from the Mesh router (see Figure 2).

Figure 2: Networking with Mesh network

You will not need the bridge command to setup the network.

Can I use cms burn on Linux?

Not everything is supported.

To download the latest Raspberry Pi OS Lite image use

cms burn image get latest-lite

To see what SDCard writers you have attached, you can use the command

cms burn info

It will issue a probe of USB devices and see if SDCards can be found.

Identify the /dev/sdX, where X is a letter such as b,c,d, … It will likely never be a.

sudo apt-get install pv cms burn sdcard –dev=/dev/sdX cms burn mount –device=/dev/sdX cms burn enable ssh cms burn unmount

Take the SDCard into the PI and set it up there. as documented.

What packages do I need to run the info command on macOS

brew install libusb
nrew install pv

Access to ext4: For the more advanced features of burn you will need full write access to the ext4 partition on your SDCard that is created when you burn it. Unfortunately the tools that used t be freely available see no longerto work properly, so you could use extFS for Mac by Paragon Software which does cost $40 for a license.

Are there any unit tests?

As cms burn may delete format, delete, and remove files during unit testing users are supposed to first review the tests before running them. Please look at the source and see if you can run a test.

we have the following tests:

• pytest -v --capture=no tests/test_01_image.py
  • This test removes files forom ~/.cloudmesh/cmburn/images
  • See also: test_01_image.py

Using Pi Imager to setup a Manager Pi with headless access

This FAQ will provide step by step instructions for burning and accessing a headless manager pi. We will include instructions for either wifi access to the pi or local ethernet connection.

If you have restricted WIFI that requires you to register you devices MAC address via a web browser (think hotel wifi access page), you might not be able to continue with a headless setup.

This FAQ references instructions from <https://www.raspberrypi. org/documentation/remote-access/ssh/> and <https://www.raspberrypi. org/documentation/configuration/wireless/headless.md>

Step 1.

Download and install the Pi Imager software from raspberrypi.org <https://www. raspberrypi.org/software/>.

Step 2.

Launch the Pi Imager software, insert an SD card reader and SD card into your laptop.

Step 3.

Choose the OS in the Pi Imager interface. We will use **Raspberry Pi OS (32-BIT) with the Raspberry Pi desktop.

Step 4.

Choose the SD card in the Pi Imager interface. If you do not see an SD card and a reader is plugged into your laptop, remove and re-insert the sd card reader.

Step 5.

Push the ‘Write’ button and confirm the settings to burn the OS to your SD card. You may need to put in the SUDO password to burn the card. This will take some time. USB 3.0 devices are faster than USB 2.0. Make sure your cable is USB 3.0 as well.

Step 6.

Mount the SD card. This can be accomplished easily in Linux by unplugging and replugging in the device. On Pi and Linux you should see the boot partition at /media/$USER/boot (where user is you username) and on MacOS at /Volumes/boot. I will use Linux for the example. Substitute as required for MacOS.

Step 7.

Enable SSH access to the SD card. At the command prompt

you@yourlaptop:~ $ cd /media/$USER/boot
you@yourlaptop:/media/$USER/boot $ touch ssh

This creates and empty file named ssh in the boot partition. On the first boot, this enables the SSH service, and then empty ssh file will be automatically deleted.

Step 8.

If only have wireless access to your Pi. You need to setup the wireless configuration.

If you have restricted WIFI that requires you to register you devices MAC address via a web browser (think hotel wifi access page), you might not be able to continue with a headless setup.

Otherwise you can continue without this step if you have ethernet access between your laptop and pi (either via switch or direct cable). After plugging into a shared switch with the Pi, or directly to it, you will need to make sure you see a link local address on your ethernet port on your laptop. It should look something like 169.254.X.X. If you do not see this investigate how to setup a link local ip on your OS.

you@yourlaptop:/media/$USER/boot $ nano /media/$USER/boot/wpa_supplicant.conf

#Insert this into the file and save (CTRL-X, Y, Enter).

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=<Insert 2 letter ISO 3166-1 country code here e.g. US>

network={
 ssid="<Name of your wireless LAN>"
 psk="<Password for your wireless LAN>"
}

Step 9.

Unmount and eject the SD card.

you@yourlaptop:~ $ sudo umount /media/$USER/boot
you@yourlaptop:~ $ sudo umount /media/$USER/rootfs

Step 10.

Boot a pi with the SD card. Wait a few minutes and try to access it via SSH. Use the Raspi OS default username “pi” and “raspberry”.

you@yourlaptop:~ $ ssh pi@raspberrypi.local

Step 11.

Change the password.

pi@raspberrypi:~ $ passwd

Step 12.

Change the hostname to managerpi.

pi@raspberrypi:~ $ sudo raspi-config
<1. System Options>
<S4 Hostname>
enter
managerpi
<Finish>
Would you like to reboot?
<Yes>

Step 13.

Wait a minute or two and reconnect. Now using the new hostname.

you@yourlaptop:~ $ ssh pi@managerpi.local

Step 14.

You are all done. You are ready to proceed with Quickstart for Bridged WiFi. You will now witness the magic of how cms burn automates this process for you.

pi@managerpi:~ $ 

Single Card Burning

Step 0. Ensure the SD card is inserted.

We can run cms burn info again as we did above to verify our SD card is connected.

Step 1. Burning the SD Card

Choose a hostname for your card. We will use red001 with ip 10.1.1.2. The IP address 10.1.1.1 is reserved for the burner pi (ie. managerpi).

Note we are using the subnet 10.1.1.0/24 in this guide. We currently recommend you do the same, otherwise the WiFi bridge will not configure correctly. We will change this in the future to support other Private IP Ranges

!! WARNING VERIFY THE DEVICE IS CORRECT. REFER TO CMS BURN !!

(ENV3) pi@managerpi:~ $ cms burn create --hostname=red001 --ip=10.1.1.2 --device=/dev/sda --tag=latest-lite

Wait for the card to burn. Once the process is complete, it is safe to remove the SD card.

We can now proceed to the bridge setup

How to update firmware?

To update the firmware reference the raspi documentation

Or follow the simple instructions below.

pi@managerpi:~ $ sudo apt update
pi@managerpi:~ $sudo apt full-upgrade
pi@managerpi:~ $sudo reboot
pi@managerpi:~ $sudo rpi-eeprom-update -a
pi@managerpi:~ $sudo reboot

How to burn a cluster using Linux

This will setup the same cluster seen in Quickstart for Bridged WiFi using only a Linux machine. Pi imager and a manual manager pi setup process is not required using this method.

Prerequisites

• We recommend Python 3.8.2 Python or newer.
• We recommend pip version 21.0.0 or newer
• You have a private and public ssh key named ~/.ssh/id_rsa and ~/. ssh/id_rsa.pub

Install Cloudmesh

Create a Python virtual environment ENV3 in which to install cloudmesh. This will keep cloudmesh and its dependecies seperate from your default environment.

Always make sure to source this environment when working with cloudmesh.

you@yourlaptop:~ $ python -m venv ~/ENV3
you@yourlaptop:~ $ source ~/ENV3/bin/activate 
you@yourlaptop:~ $ mkdir cm
you@yourlaptop:~ $ cd cm
you@yourlaptop:~ $ pip install cloudmesh-installer
you@yourlaptop:~ $ cloudmesh-installer get pi 

Create a Manager Pi

Step 1. Get the latest RaspiOS-full image

you@yourlaptop:~ $ cms burn image get latest-full

Step 1. Insert and SD card to your laptop and identify the sd card device name using:

you@yourlaptop:~ $ cms burn info

Step 2. Burn the manager pi.

!! WARNING VERIFY THE DEVICE IS CORRECT. REFER TO CMS BURN !!

you@yourlaptop:~ $ cms burn create --hostname=managerpi --tag=latest-full--device=/dev/sdX --ssid=your_wifi --wifipassword=your_password

Create the workers

Step 1. Download the latest RaspiOS-lite image

cms burn image get latest-lite

Step 2. Burn the workers

!! WARNING VERIFY THE DEVICE IS CORRECT. REFER TO CMS BURN !!

cms burn create --hostname=red00[1-4] --ip=10.1.1.[2-5] --device=/dev/sdX --tag=latest-lite

Step 3. Turn off the cluster, insert sd cards, turn on cluster, and connect to the manager pi.

you@yourlaptop:~ $ ssh pi@mangerpi.local

Step 4. Update and install cloudmesh on your manager pi

Update pip. The simple curl command below will generate an ssh-key, update your system, and install cloudmesh.

pi@managerpi:~ $ pip install pip -U
pi@managerpi:~ $ curl -Ls http://cloudmesh.github.io/get/pi | sh -                
pi@managerpi:~ $ sudo reboot

Step 4. Enable the bridge on the mangerpi.

See section Connecting Pis to the Internet via Bridge

Step 5. Generate and distribute SSH keys

See section Set up of the SSH keys and SSH tunnel

Step 6. Enjoy your Pi cluster :)

Alternatives

There are several alternatives to make the setup easier:

• Using Ansible after you have created the SDCards via PIImager. THis however requires still the discovery of the hosts on the network and additional steps.
• PiBakery can burn cards while allowing startup scripts and naming hosts. Although the GUI is nice it is also a limiting factor as each card shoudl have a different hostname
• Using DHCP to get ip addresses automatically. THis is a solution we also used but do not present here
• PXE or network booting whch allsows you to boot from the network. For larger PI clusters this requires multiple Servers so that the network is not everwhelmed. Starting the cluster takes much longer.

What is the status of the implementation?

• for macOS, only the image commands have unit tests
• firmware does not have a unit test

empty = not yet implemented

• • = verified throug unit test either by ANthony or Gregor
• • broken

• TODO1 = todo for boot fs, rootfs not supported

• 2 = change and add –ssd so its uniform 1 = get needs to use the image versions refresh cache

How can I contribute Contributing

The code uses a variety of cloudmesh components. This mainly includes

• GitHub cloudmesh-pi-burn
• GitHub cloudmesh-pi-cluster

Additional cloudmesh components ar used. For example:

• GitHub cloudmesh-pi-cluster
• GitHub cloudmesh-pi-cluster
• GitHub cloudmesh-pi-cluster

4.4 - Fortan

You may find still that many scientific programs are written in fortran. For some smaller fortran programs it is even possible to run them on a Raspberry pi.

Installation

Naturally you will need to install a fortran compiler, which you can do with

pi$ sudo apt-get install gfortran

Example

To test it out store the following program into hello.f90

program hello
    print *, "Hello World!"
end program hello

Now you can compile it with

pi$ gfortran -o hello hello.f90

Execute it with

pi$ hello

4.5 - Kubernetes

TBD

• Identified as useful: https://opensource.com/article/20/3/kubernetes-raspberry-pi-k3s

Complete to this format:

• title, author, publication, date

fedora

• Kubernetes on Raspberry Pi 4 on Fedora, Matthias Runge, 04 January 2021 kubespray
• apacheconna2017 unclear firmware update, ppt, fedora

x86 via docker

• Raspberry Pi 4 Kubernetes Cluster with x86 Support, Ajay Patel sd cards may fail on heavy RW

Raspbian

• Running a Kubernetes Cluster with Dashboard on Raspberry Pi 4, Jeppe Andersen Blog, Nov 3, 2019 dashboard
• codesqueak/k18srpi4: Kubernetes on a Raspberry Pi 4 Cluster, Nov 3, 2019 Raspbian Buster Lite / Kernel 4:19

Ubuntu

• Ubuntu 20.10 on Raspberry Pi delivers the full Linux desktop and micro clouds, Ubuntu
• Build a Kubernetes cluster with the Raspberry Pi, Chris Collins, Opensource.com, 05 Jun 2020 Ubuntu easy to follow, uses upto date ubuntu kubernetes install
• How Raspberry Pi and Kubernetes work together, The Enterprisers Project just a pointer to Collins page
• How to build a Raspberry Pi Kubernetes cluster using MicroK8s, Ubuntu easy to follow, discusses cgroups in cmmdline.txt
• Kubernetes on 64 Bit OS Raspberry Pi 4 · Robert Sirchia Ubuntu 19.10
• Running kubernetes on Raspberry Pi - Viktor Andersson Ubuntu likely outdated, has comment on large sd cards, mentions PI3 and Pi4
• mhausenblas/kube-rpi: Kubernetes on Raspberry Pi 4 with 64-bit OS tp link wireless to wired, ubuntu 19.10

RancherOS

• Rancher Docs: Raspberry Pi Pi3
• Home, Raspberry Pi Dramble, Jeff Gerling Github Needs evaluation

k8s

• Kubernetes (K8s) Private Cloud with Raspberry Pi 4s - DEV Community
• KubeSail Blog, MicroK8s on the Raspberry Pi 4

k3s

• k3s-io/k3s: Lightweight Kubernetes
• How Rancher Labs’ K3s Makes It Easy to Run Kubernetes at the Edge – The New Stack
• K3s: Lightweight Kubernetes
• Tutorial: Install a Highly Available K3s Cluster at the Edge – The New Stack
• Raspberry Pi Cluster Part 2: ToDo API running on Kubernetes with k3s
• Self-hosting Kubernetes on your Raspberry Pi Reports Pi 3 have issues with kubernetes, k3s, arcade

Hyperoid, k3s

• Everything I know about Kubernetes I learned from a cluster of Raspberry Pis, Jeff Geerling, November 26, 2019 k3s-ansible, nice videos, lists many other options for k*, requires to go in various parts
• K3s vs k8s – What’s the difference between K8s and k3s - Civo.com

TuringPI

• https://turingpi.com/installing-kubernetes-cluster-on-the-turingpi/

k3sup

• Kubernetes Homelab with Raspberry Pi and k3sup

microk8

• MicroK8s tutorials

Medium.com

Requires fee, if you read several articles (I have not read them)

• Walk-through — install Kubernetes to your Raspberry Pi in 15 minutes, Alex Ellis, Mar 1, 2020·9, Medium requires access to medium.com
• Kubernetes 1.17 on a Raspberry Pi 4 Cluster, Jamie Duncan, Dec 16, 2019·5, Medium
• Step Stepguide — Kubernetes Cluster on Raspberry Pi 4B: kubernetes
• Building a kubernetes cluster on Raspberry Pi and low-end equipment. Part 1, Eduard Iskandarov, ITNEXT

Services

Azure SQL Edge

• Building a Raspberry Pi cluster to run Azure SQL Edge on Kubernetes, DBA From The Cold

4.6 - Motherboard LED

Using Cloudmesh to Control the LEDs

The most convenient way to control the motherboard LEDs is via Cloudmesh. It includes a special command that can even be called from a remote Pi using ssh once you hve set up the cluster with cloudmesh.

This command switches on and off the LEDs of the specified PIs. If the hostname is omitted. It is assumed that the code is executed on a PI and its LED are set.

The specialty of this program is that it not only can query the current state, but also allows you to set multiple LEDs on different motherboards at the same time with a convenient parameterized syntax.

Examples

We present a number of examples on how to use this command

cms pi led list  "red,red[01-03]"

lists the LED status of the given hosts

cms pi led red off  "red,red[01-03]"

switches off the led of the given PIs

cms pi led red on  "red,red[01-03]"

switches on the led of the given PIs

cms pi led red blink  "red,red[01-03]"

switches on and off the led of the given PIs

cms pi led red sequence  "red,red[01-03]"

goes in sequential order and switches on and off the led of the given PIs

Syntax

The command syntax is

cms pi led reset [NAMES]
cms pi led (red|green) VALUE
cms pi led (red|green) VALUE NAMES [--user=USER]
cmspi led list NAMES [--user=USER]

Using Motherboard LEDs from Shell Scripts

The Raspberry pi contains an LED that can also be used to provide us with some information as to the status of the PI. It is usually used for reporting the power status.

The green LED can be made blinking as follows in root

echo 1 > /sys/class/leds/led0/brightness
echo 0 > /sys/class/leds/led0/brightness

Controlling the Motheboard LEDs remotely via ssh

Naturally this ac be done via a remote command if your ssh keys are uploaded and your originating computer is added to the authorized_keys. Now you can can control them via ssh

ssh pi@red03 "echo 1 > led; sudo cp led /sys/class/leds/led0/brightness"
ssh pi@red03 "echo 0 > led; sudo cp led /sys/class/leds/led0/brightness"

This is extremely useful as it allows us to check if we the OS is available and we can access the PI.

One strategy is to for example switch the light of, once it is booted, so we can see which board may be in trouble.

4.7 - Run at Boot

Run Commands at Boot time

In many cases we need to provide configurations and programs that run at boot time. A number of different methods exist to run commands and programs at boot time.

We will be focusing here only a few of them

rc.local

On your Pi you will find under /etc/rc.local a file in which you can list programs that are started up at boot time. The programs should success fully run and exit with the status 0, and they must not continuously run in which case they need to be started in background.

To make sure you do not forget it, simply add the following line at the end of your program

exit 0

indicating that the start was successful. Programs in rc.local must use the absolute file path.

Crontab

Crontab is a service the schedules jobs that can run at various times repeatedly. For example we can use crontab to run commands every hour, every day, every half hour or other time intervals or at reboot.

To use crontab follow these steps

1. Open a terminal and enter the command

   $ crontab -e
   

   If you are doing this for the first time, you will be asked to chose an editor. Please, choose your favorite editor

2. To run the program at boot time, add the following line to the at the end of the file

    @reboot <command>
   

Let us look at an example and assume we have test.py program in your home directory at /home/pi/test.py. Once you add it to crontab with

 @reboot python /home/pi/test.py

it will be run at boot time

It is important to provide the absolute path to the file. In case your file produces output you need to redirect it into a file

  @reboot python /home/pi/test.py > /home/pi/test.log

• When the raspberry pi reboots, the program will run automatically.

References

A good introduction to the various methods is provided at

• https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/

For more information on crontab see for example:

• http://www.adminschoice.com/crontab-quick-reference

5 - Contributors

5.1 - Erin Seliger

Erin Seliger is an undergraduate student at Indiana University in the Intelligent Systems Engineering programming. For piplanet she is contributing to using mpi on raspberry pis.

Web Page: https://github.com/eselige

E-mail: ejseliger@gmail.com

5.2 - Fidel Leal

Fidel Leal contributes to cloudmesh

Web Page: https://github.com/flealc

E-mail: fidelxleal@gmail.com

5.3 - Gregor von Laszewski

Gregor von Laszewski is an Assistant Director in the Digital Science Center (DSC) in the School of Informatics and Computing at Indiana University. He holds a position as Adjunct Professor in the Intelligent Systems Engineering Department.

His current interest and projects include cloud computing, big data, and scientific impact metrics. He initiated the Cloudmesh project which hybrid multicloud toolkit to enable cloud computing across various Cloud and Container IaaS such as OpenStack, AWS, Azure, docker, docker swarm, and Kubernetes.

He received a Masters Degree in 1990 from the University of Bonn, Germany, and a Ph.D. in 1996 from Syracuse University in computer science. He held a position at Argonne National Laboratory from Nov. 1996 – Aug. 2009, where he was last a scientist and a fellow of the Computation Institute at University of Chicago. During his last two years at ANL he was on sabbatical and an Associate Professor and the Director of an institution at Rochester Institute of Technology, focusing on Cyberinfrastructure.

Past appointments include Adjunct Professor positions at the Computer Science Department at Indiana University, Associate Professor at Rochester Institute of Techology, Adjunct Professor at University of North Texas. He has also taught on a voluntary basis at Illinois Institute of Technology.

He provided leadership to interface with the with San Diego Supercomputing Center on virtual clusters for XSEDE comet. Previously, he was the architect of FutureGrid, one of the first successful clouds in US academia. He was previously involved in Grid computing since the term was coined. Before that, he initiated and led the Java Commodity Grid Kit which provided the basis for hundreds of Grid related projects, including the Globus Toolkit.

Web Page: https://laszewski.github.io/

E-mail: laszewski@gmail.com

5.4 - Venkata Sai Dhakshesh Kolli

Venkata Sai Dhakshesh Kolli completed his B.S. (2020) and is currently working on his M.S. (2022) in Computer Science and MBA (2022) in Finance at Indiana University Bloomington. His past experiences include internships in software engineering with Liberty Mutual Insurance and tech startups Clear Software and Eureka Fintech, and projects on COVID-19 and quantitative trading. His interests include investing, financial technologies, and financial compliance.

Web Page: https://github.com/dkkolli

E-mail: vekolli.iu@gmail.com