Shell

• 1: Linux Shell
• 2: Perl One Liners
• 3: ZSH

1 - Linux Shell

We introduce you to a number of useful shell commands. You may ask:

“Why is he so keen on telling me all about shells as I do have a beautiful GUI?”

You will soon learn that A GUI may not be that suitable if you like to manage 10, 100, 1000, 10000, … virtual machines. A commandline interface could be mcuh simpler and would allow scripting.

History

LINUX is a reimplementation by the community of UNIX which was developed in 1969 by Ken Thompson and Dennis Ritchie of Bell Laboratories and rewritten in C. An important part of UNIX is what is called the kernel which allows the software to talk to the hardware and utilize it.

In 1991 Linus Torvalds started developing a Linux Kernel that was initially targeted for PC’s. This made it possible to run it on Laptops and was later on further developed by making it a full Operating system replacement for UNIX.

Shell

One of the most important features for us will be to access the computer with the help of a shell. The shell is typically run in what is called a terminal and allows interaction to the computer with commandline programs.

There are many good tutorials out there that explain why one needs a linux shell and not just a GUI. Randomly we picked the first one that came up with a google query. This is not an endorsement for the material we point to, but could be a worth while read for someone that has no experience in Shell programming:

• http://linuxcommand.org/lc3_learning_the_shell.php

Certainly you are welcome to use other resources that may suite you best. We will however summarize in table form a number of useful commands that you may als find even as a RefCard.

• http://www.cheat-sheets.org/#Linux

We provide in the next table a number of useful commands that you want to explore. For more information simply type man and the name of the command. If you find a useful command that is missing, please add it with a Git pull request.

The command man

On Linux you find a rich set of manual pages for thes commands. Try to pick one and execute:

$ man ls

You will see somthing like this

LS(1)                     BSD General Commands Manual                    LS(1)

NAME
     ls -- list directory contents

SYNOPSIS
     ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]

DESCRIPTION
     
     For each operand that names a file of a type other than directory,
     ls displays its name as well as any requested, associated
     information.  For each operand that names a file of type directory,
     ls displays the names of files contained within that directory, as
     well as any requested, associated information.

     If no operands are given, the contents of the current directory are
     displayed.  If more than one operand is given, non-directory
     operands are displayed first; directory and non-directory operands
     are sorted separately and in lexicographical order.

     The following options are available:

     -@      Display extended attribute keys and sizes in long (-l) output.

     -1      (The numeric digit ``one''.)  Force output to be one entry
             per line.  This is the default when output is not to a terminal.

     -A      List all entries except for . and ...  Always set for the
             super-user.

     -a      Include directory entries whose names begin with a dot (.).


     ... on purpose cut ... instead try it yourslef

Multi-command execution

One of the important features is that one can execute multiple commands in the shell.

To execute command 2 once command 1 has finished use

command1; command2

To execute command 2 as soon as command 1 forwards output to stdout use

command1; command2

To execute command 1 in the background use

command1 &

Keyboard Shortcuts

These shortcuts will come in handy. Note that many overlap with emacs short cuts.

bashrc, bash_profile or zprofile

Usage of a particular command and all the attributes associated with it, use man command. Avoid using rm -r command to delete files recursively. A good way to avoid accidental deletion is to include the following in the file .bash_profile or .zprofile on macOS or .bashrc on other platforms:

alias rm='rm -i'
alias mv='mv -i'
alias h='history'

Makefile

Makefiles allow developers to coordinate the execution of code compilations. This not only includes C or C++ code, but any translation from source to a final format. For us this could include the creation of PDF files from latex sources, creation of docker images, and the creation of cloud services and their deployment through simple workflows represented in makefiles, or the coordination of execution targets.

As makefiles include a simple syntax allowing structural dependencies they can easily adapted to fulfill simple activities to be executed in repeated fashion by developers.

An example of how to use Makefiles for docker is provided at

• http://jmkhael.io/makefiles-for-your-dockerfiles/.

An example on how to use Makefiles for LaTeX is provided at

• https://github.com/cloudmesh/book/blob/master/Makefile.

Makefiles include a number of rules that are defined by a target name. Let us define a target called hello that prints out the string “Hello World”.

hello:
    @echo "Hello World"

Important to remember is that the commands after a target are not indented just by spaces, but actually by a single TAB character. Editors such as emacs will be ideal to edit such Makefiles, while allowing syntax highlighting and easy manipulation of TABs. Naturally other editors will do that also. Please chose your editor of choice. One of the best features of targets is that they can depend on other targets. Thus, iw we define

hallo: hello
    @echo "Hallo World"

our makefile will first execute hello and than all commands in hallo. As you can see this can be very useful for defining simple dependencies.

In addition we can define variables in a makefile such as

HELLO="Hello World"

hello:
    @echo $(HELLO)

and can use them in our text with $ invocations.

Moreover, in sophisticated Makefiles, we could even make the targets dependent on files and a target rules could be defined that only compiles those files that have changed since our last invocation of the Makefile, saving potentially a lot of time. However, for our work here we just use the most elementary makefiles.

For more information we recommend you to find out about it on the internet. A convenient reference card is available at

• http://www.cs.jhu.edu/~joanne/unixRC.pdf.

chmod

The chmod command stand for change mode and changes the access permissions for a given file system object(s). It uses the following syntax: chmod [options] mode[,mode] file1 [file2…]. The option parameters modify how the process runs, including what information is outputted to the shell:

Modes specify which rights to give to which users. Potential users include the user who owns the file, users in the file’s Group, other users not in the file’s Group, and all, and are abbreviated as u, g, o, and a respectively. More than one user can be specified in the same command, such as chmod –v ug(operator)(permissions) file.txt . If no user is specified, the command defaults to a. Next, a + or - indicates whether permissions should be added or removed for the selected user(s). The permissions are as follows:

More than one permission can be also be used in the same command as follows:

$ chmod –v o+rw file.txt

Multiple files can also be specified:

$ chmod a-x,o+r file1.txt file2.txt

Exercises

E.Linux.1

Familiarize yourself with the commands

E.Linux.2

Find more commands that you find useful and add them to this page.

E.Linux.3

Use the sort command to sort all lines of a file while removing duplicates.

E.Linux.4

Should there be other commands listed in the table with the Linux commands If so which? Create a pull request for them.

E.Linux.5

Write a section explaining chmod. Use letters not numbers

E.Linux.6

Write a section explaining chown. Use letters not numbers

E.Linux.7

Write a section explaining su and sudo

E.Linux.8

Write a section explaining cron, at, and crontab

2 - Perl One Liners

Perl is a programming language that used to be very popular with system administrators. It predates Python. It has some very powerful regular expression abilities allowing you to easily do things on the commandline that woul otherwise thake many hours. Here ar some useful perl one line commands.

Strip trailing whitespace from a file

$ perl -lpe 's/\s*$//' FILENAME

Replace wrong quote

$ perl -i -p -e "s/’/'/g;"  *.md

Remove ^M from file

$ perl -p -i -e 's/\r\n$/\n/g'

3 - ZSH

Introduction

Z shell (zsh) is an alternative to bash. It is used as an interactive shell or command interpreter. Zsh has been chosen by apple as a replacement for bash. A large number of plugins for zsh is avalable at the Web site Oh My Zsh.

Features of zsh include:

• commandline completion
• global history that can be shared in shells
• build in file globing
• multiline commands
• spell correction
• compatibillity modes to impersonate other shells
• themes for prompts
• in addition to which a wher ecommand
• shortcut to names directories with ~

In principal it does not matter much whcih shell you use as long as you use to set up your envireonment properly. While bash zupporst .bash_profile and .bashrc, zsh supports ~/.zprofile and ~/.zshrc

A good overview of the loading process is documented at

• https://medium.com/@rajsek/zsh-bash-startup-files-loading-order-bashrc-zshrc-etc-e30045652f2e

Setting up zsh on an older OSX is relatively simple.

$ brew install zsh

To add Oh My Zsh you can do:

$ sh -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"

To chnge the default shell to zsh you can than execute

$ chsh -s $(which zsh)

To activate the shell you can as usal do

$ source ~/.zshrc

However if you start a new terminal, you do not have to do this as it is added automatically.

To use a number of useful plugins, you can activate them with

$ plugins=(osx git colored-man colorize pip python brew zsh-syntax-highlighting zsh-autosuggestions)

If you like to change the theme you can find a large number at

• https://github.com/ohmyzsh/ohmyzsh/wiki/Themes

Using zsh on your operating systems

For other operationg systems see

• https://github.com/ohmyzsh/ohmyzsh/wiki/Installing-ZSH

zsh on Windows 10

To install zsh on Windows 10, please look at

• https://www.howtogeek.com/258518/how-to-use-zsh-or-another-shell-in-windows-10/