4.3.3. Interactive Python

Python can be used interactively. Start by entering the interactive loop by executing the command:

$ python

You should see something like the following:

Python 2.7.13 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

The >>> is the prompt for the interpreter. This is similar to the shell interpreter you have been using.

Tip

Often we show the prompt when illustrating an example. This is to provide some context for what we are doing. If you are following along you will not need to type in the prompt.

This interactive prompt does the following:

  • read your input commands
  • evaluate your command
  • print the result of evaluation
  • loop back to the beginning.

This is why you may see the interactive loop referred to as a REPL: Read-Evaluate-Print-Loop.

4.3.4. Python 3 Features in Python 2

As mentioned earlier, we assume you will use Python 2.7.X because there are still some libraries that haven’t been ported to Python 3. However, there are some features of Python 3 we can and want to use in Python 2.7. Before we do anything else, we need to make these features available to any subsequent code we write:

>>> from __future__ import print_function, division

Note

The first of these imports allows us to use the print function to output text to the screen, instead of the print statement, which Python 2 uses. This is simply a design decision that better reflects Python’s underlying philosophy.

Note

The second of these imports makes sure that the division operator behaves in a way a newcomer to the language might find more intruitive. In Python 2, division / is floor division when the arguments are integers, meaning that 5 / 2 == 2, for example. In Python 3, division / is true division, thus 5 / 2 == 2.5.

4.3.5. Statements and Strings

Let us explore the syntax of Python. Type into the interactive loop and press Enter:

>>> print("Hello world from Python!")
Hello world from Python!

What happened: the print function was given a string to process. A string is a sequence of characters. A character can be a alphabetic (A through Z, lower and upper case), numeric (any of the digits), white space (spaces, tabs, newlines, etc), syntactic directives (comma, colon, quotation, exclamation, etc), and so forth. A string is just a sequence of the character and typically indicated by surrounding the characters in double quotes.

Tip

Standard output is discussed in the ../../lesson/linux/shell lesson.

So, what happened when you pressed Enter? The interactive Python program read the line print “Hello world from Python!”, split it into the print statement and the “Hello world from Python!” string, and then executed the line, showing you the output.

4.3.6. Variables

You can store data into a variable to access it later. For instance, instead of:

>>> print('Hello world from Python!')

which is a lot to type if you need to do it multiple times, you can store the string in a variable for convenient access:

>>> hello = 'Hello world from Python!'
>>> print(hello)
Hello world from Python!

4.3.7. Data Types

4.3.7.1. Booleans

A boolean is a value that indicates truthness of something. You can think of it as a toggle: either “on” or “off”, “one” or “zero”, “true” or “false”. In fact, the only possible values of the boolean (or bool) type in Python are:

  • True
  • False

You can combine booleans with boolean operators:

  • and
  • or
>>> print(True and True)
True
>>> print(True and False)
False
>>> print(False and False)
False
>>> print(True or True)
True
>>> print(True or False)
True
>>> print(False or False)
False

4.3.7.2. Numbers

The interactive interpreter can also be used as a calculator. For instance, say we wanted to compute a multiple of 21:

>>> print(21 * 2)
42

We saw here the print statement again. We passed in the result of the operation 21 * 2. An integer (or int) in Python is a numeric value without a fractional component (those are called floating point numbers, or float for short).

The mathematical operators compute the related mathematical operation to the provided numbers. Some operators are:

  • * — multiplication
  • / — division
  • + — addition
  • - — subtraction
  • ** — exponent

Exponentiation is read as x**y is x to the yth power:

\[x^y\]

You can combine floats and ints:

>>> print(3.14 * 42 / 11 + 4 - 2)
13.9890909091
>>> print(2**3)
8

Note that operator precedence is important. Using parenthesis to indicate affect the order of operations gives a difference results, as expected:

>>> print(3.14 * (42 / 11) + 4 - 2)
11.42
>>> print(1 + 2 * 3 - 4 / 5.0)
6.2
>>> print( (1 + 2) * (3 - 4) / 5.0 )
-0.6

4.3.8. REPL (Read Eval Print Loop)

We have so far seen a few examples of types: strings, bools, ints, and floats. A type indicates that values of that type support a certain set of operations. For instance, how would you exponentiate a string? If you ask the interpreter, this results in an error:

>>> "hello"**3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int'

There are many different types beyond what we have seen so far, such as dictionariess, lists, sets. One handy way of using the interactive python is to get the type of a value using type():

>>> type(42)
<type 'int'>
>>> type(hello)
<type 'str'>
>>> type(3.14)
<type 'float'>

You can also ask for help about something using help():

>>> help(int)
>>> help(list)
>>> help(str)

Tip

Using help() opens up a pager. To navigate you can use the spacebar to go down a page w to go up a page, the arrow keys to go up/down line-by-line, or q to exit.

4.3.9. Module Management

A module allows you to logically organize your Python code. Grouping related code into a module makes the code easier to understand and use. A module is a Python object with arbitrarily named attributes that you can bind and reference. A module is a file consisting of Python code. A module can define functions, classes and variables. A module can also include runnable code.

4.3.9.1. Import Statement

When the interpreter encounters an import statement, it imports the module if the module is present in the search path. A search path is a list of directories that the interpreter searches before importing a module. The from…import Statement Python’s from statement lets you import specific attributes from a module into the current namespace. The from…import has the following syntax − from modname:

import name1[, name2[, … nameN]]

When the interpreter encounters an import statement, it imports the module if the module is present in the search path. A search path is a list of directories that the interpreter searches before importing a module.

4.3.9.2. The from … import Statement

Python’s from statement lets you import specific attributes from a module into the current namespace. The from … import has the following syntax:

::
from module1 import name1[, name2[, … nameN]]

4.3.10. Date Time in Python

The datetime module supplies classes for manipulating dates and times in both simple and complex ways. While date and time arithmetic is supported, the focus of the implementation is on efficient attribute extraction for output formatting and manipulation. For related functionality, see also the time and calendar modules.

The import Statement You can use any Python source file as a module by executing an import statement in some other Python source file.

>>>from datetime import datetime

This module offers a generic date/time string parser which is able to parse most known formats to represent a date and/or time.

>>>from dateutil.parser import parse

pandas is an open source Python library for data analysis that needs to be imported.

>>>import pandas as pd

Create a string variable with the class start time

>>>fall_start = '08-21-2017'

Convert the string to datetime format

>>>datetime.strptime(fall_start, '%m-%d-%Y')
datetime.datetime(2017, 8, 21, 0, 0)

Creating a list of strings as dates

>>>class_dates = ['8/25/2017', '9/1/2017', '9/8/2017', '9/15/2017', '9/22/2017', '9/29/2017']

Convert Class_dates strings into datetime format and save the list into variable a

>>>a = [datetime.strptime(x, '%m/%d/%Y') for x in class_dates]

Use parse() to attempt to auto-convert common string formats. Parser must be a string or character stream, not list.

>>>parse(fall_start)
datetime.datetime(2017, 8, 21, 0, 0)

Use parse() on every element of the Class_dates string.

>>>[parse(x) for x in class_dates]
[datetime.datetime(2017, 8, 25, 0, 0),
 datetime.datetime(2017, 9, 1, 0, 0),
 datetime.datetime(2017, 9, 8, 0, 0),
 datetime.datetime(2017, 9, 15, 0, 0),
 datetime.datetime(2017, 9, 22, 0, 0),
 datetime.datetime(2017, 9, 29, 0, 0)]

Use parse, but designate that the day is first.

>>>parse (fall_start, dayfirst=True)
datetime.datetime(2017, 8, 21, 0, 0)

Create a dataframe.A DataFrame is a tablular data structure comprised of rows and columns, akin to a spreadsheet, database table. DataFrame as a group of Series objects that share an index (the column names).

>>>import pandas as pd
>>>data = {'class_dates': ['8/25/2017 18:47:05.069722', '9/1/2017 18:47:05.119994',
                        '9/8/2017 18:47:05.178768', '9/15/2017 18:47:05.230071',
                        '9/22/2017 18:47:05.230071', '9/29/2017 18:47:05.280592'],
        'complete': [1, 0, 1, 1, 0, 1]}
>>>df = pd.DataFrame(data, columns = ['class_dates', 'complete'])
>>>print(df)
                     class_dates  complete
0  8/25/2017 18:47:05.069722         1
1   9/1/2017 18:47:05.119994         0
2   9/8/2017 18:47:05.178768         1
3  9/15/2017 18:47:05.230071         1
4  9/22/2017 18:47:05.230071         0
5  9/29/2017 18:47:05.280592         1

Convert df[‘date’] from string to datetime

>>>import pandas as pd
>>>pd.to_datetime(df['class_dates'])
0   2017-08-25 18:47:05.069722
1   2017-09-01 18:47:05.119994
2   2017-09-08 18:47:05.178768
3   2017-09-15 18:47:05.230071
4   2017-09-22 18:47:05.230071
5   2017-09-29 18:47:05.280592
Name: class_dates, dtype: datetime64[ns]

4.3.11. Control Statements

4.3.11.1. Comparision

Computer programs do not only execute instructions. Occasionally, a choice needs to be made. Such as a choice is based on a condition. Python has several conditional operators:

>   greater than
<   smaller than
==  equals
!=  is not

Conditions are always combined with variables. A program can make a choice using the if keyword. For example:

>>> x = int(input("Guess x:"))
>>> if x == 4:
...    print('You guessed correctly!')
...    <ENTER>

In this example, You guessed correctly! will only be printed if the variable x equals to four (see table above). Python can also execute multiple conditions using the elif and else keywords.

>>> x = int(input("Guess x:"))
>>> if x == 4:
...     print('You guessed correctly!')
... elif abs(4 - x) == 1:
...     print('Wrong guess, but you are close!')
... else:
...     print('Wrong guess')
... <ENTER>

4.3.11.2. Iteration

To repeat code, the for keyword can be used. For example, to display the numbers from 1 to 10, we could write something like this:

>>> for i in range(1, 11):
...    print('Hello!')

The second argument to range, 11, is not inclusive, meaning that the loop will only get to 10 before it finishes. Python itself starts counting from 0, so this code will also work:

>>> for i in range(0, 10):
...    print(i + 1)

In fact, the range function defaults to starting value of 0, so the above is equivalent to:

>>> for i in range(10):
...    print(i + 1)

We can also nest loops inside each other:

>>> for i in range(0,10):
...     for j in range(0,10):
...         print(i,' ',j)
... <ENTER>

In this case we have two nested loops. The code will iterate over the entire coordinate range (0,0) to (9,9)

4.3.12. Datatypes

4.3.12.1. Lists

see: https://www.tutorialspoint.com/python/python_lists.htm

Lists in Python are ordered sequences of elements, where each element can be accessed using a 0-based index.

To define a list, you simply list its elements between square brackest []:

>>> >>> names = ['Albert', 'Jane', 'Liz', 'John', 'Abby']
>>> names[0] # access the first element of the list
'Albert'
>>> names[2] # access the third element of the list
'Liz'

You can also use a negative index if you want to start counting elements from the end of the list. Thus, the last element has index -1, the second before last element has index -2 and so on:

>>> names[-1] # access the last element of the list
'Abby'
>>> names[-2] # access the second last element of the list
'John'

Python also allows you to take whole slices of the list by specifing a beginning and end of the slice separated by a colon ::

>>> names[1:-1] # the middle elements, excluding first and last
['Jane', 'Liz', 'John']

As you can see from the example above, the starting index in the slice is inclusive and the ending one, exclusive.

Python provides a variety of methods for manipulating the members of a list.

You can add elements with append:

>>> names.append('Liz')
>>> names
['Albert', 'Jane', 'Liz', 'John', 'Abby', 'Liz']

As you can see, the elements in a list need not be unique.

Merge two lists with extend:

>>> names.extend(['Lindsay', 'Connor'])
>>> names
['Albert', 'Jane', 'Liz', 'John', 'Abby', 'Liz', 'Lindsay', 'Connor']

Find the index of the first occurrence of an element with index:

>>> names.index('Liz')
2

Remove elements by value with remove:

>>> names.remove('Abby')
>>> names
['Albert', 'Jane', 'Liz', 'John', 'Liz', 'Lindsay', 'Connor']

Remove elements by index with pop:

>>> names.pop(1)
'Jane'
>>> names
['Albert', 'Liz', 'John', 'Liz', 'Lindsay', 'Connor']

Notice that pop returns the element being removed, while remove does not.

If you are familiar with stacks from other programming languages, you can use insert and pop:

>>> names.insert(0, 'Lincoln')
>>> names
['Lincoln', 'Albert', 'Liz', 'John', 'Liz', 'Lindsay', 'Connor']
>>> names.pop()
'Connor'
>>> names
['Lincoln', 'Albert', 'Liz', 'John', 'Liz', 'Lindsay']

The Python documentation contains a `full list of list operations <>`_.

To go back to the range function you used earlier, it simply creates a list of numbers:

>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(2, 10, 2)
[2, 4, 6, 8]

4.3.12.2. Sets

Python lists can contain duplicates as you saw above:

>>> names = ['Albert', 'Jane', 'Liz', 'John', 'Abby', 'Liz']

When we don’t want this to be the case, we can use a set:

>>> unique_names = set(names)
>>> unique_names
set(['Lincoln', 'John', 'Albert', 'Liz', 'Lindsay'])

Keep in mind that the set is an unordered collection of objects, thus we can not access them by index:

>>> unique_names[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  TypeError: 'set' object does not support indexing

However, we can convert a set to a list easily:

>>> unique_names = list(unique_names)
>>> unique_names
['Lincoln', 'John', 'Albert', 'Liz', 'Lindsay']
>>> unique_names[0]
'Lincoln'

Notice that in this case, the order of elements in the new list matches the order in which the elements were displayed when we create the set (we had set([‘Lincoln’, ‘John’, ‘Albert’, ‘Liz’, ‘Lindsay’]) and now we have [‘Lincoln’, ‘John’, ‘Albert’, ‘Liz’, ‘Lindsay’]). You should not assume this is the case in general. That is, don’t make any assumptions about the order of elements in a set when it is converted to any type of sequential data structure.

You can change a set’s contents using the add, remove and update methods which correspond to the append, remove and extend methods in a list. In addition to these, set objects support the operations you may be familiar with from mathematical sets: union, intersection, difference, as well as operations to check containment. You can read about this in the Python documentation for sets.

4.3.12.3. Removal and Testing for Membership in Sets

One important advantage of a set over a list is that access to elements is fast. If you are familiar with different data structures from a Computer Science class, the Python list is implemented by an array, while the set is implemented by a hash table.

We will demonstrate this with an example. Let’s say we have a list and a set of the same number of elements (approximately 100 thousand):

>>> import sys, random, timeit
>>> nums_set = set([random.randint(0, sys.maxint) for _ in range(10**5)])
>>> nums_list = list(nums_set)
>>> len(nums_set)
100000

We will use the timeit Python module to time 100 operations that test for the existence of a member in either the list or set:

>>> timeit.timeit('random.randint(0, sys.maxint) in nums', setup='import random; nums=%s' % str(nums_set), number=100)
0.0004038810729980469
>>> timeit.timeit('random.randint(0, sys.maxint) in nums', setup='import random; nums=%s' % str(nums_list), number=100)
0.3980541229248047

The exact duration of the operations on your system will be different, but the take away will be the same: searching for an element in a set is orders of magnitude faster than in a list. This is important to keep in mind when you work with large amounts of data.

4.3.12.4. Dictionaries

One of the very important data structures in python is a dictionary also referred to as dict.

A dictionary represents a key value store:

>>> person = {'Name': 'Albert', 'Age': 100, 'Class': 'Scientist'}
>>> print("person['Name']: ", person['Name'])
person['Name']:  Albert
>>> print("person['Age']: ", person['Age'])
person['Age']:  100

You can delete elements with the following commands:

>>> del person['Name'] # remove entry with key 'Name'
>>> person
{'Age': 100, 'Class': 'Scientist'}
>>> person.clear()     # remove all entries in dict
>>> person
{}
>>> del person         # delete entire dictionary
>>> person
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  NameError: name 'person' is not defined

You can iterate over a dict:

>>> person = {'Name': 'Albert', 'Age': 100, 'Class': 'Scientist'}
>>> for item in person:
...   print(item, person[item])
...   <ENTER>
Age 100
Name Albert
Class Scientist

4.3.12.5. Dictionary Keys and Values

You can retrieve both the keys and values of a dictionary using the keys() and values() methods of the dictionary, respectively:

>>> person.keys()
['Age', 'Name', 'Class']
>>> person.values()
[100, 'Albert', 'Scientist']

Both methods return lists. Notice, however, that the order in which the elements appear in the returned lists (Age, Name, Class) is different from the order in which we listed the elements when we declared the dictionary initially (Name, Age, Class). It is important to keep this in mind: you can’t make any assumptions about the order in which the elements of a dictionary will be returned by the `keys()` and `values()` methods.

However, you can assume that if you call keys() and values() in sequence, the order of elements will at least correspond in both methods. In the above example Age corresponds to 100, Name to ‘Albert, and Class to Scientist, and you will observe the same correspondence in general as long as `keys()` and `values()` are called one right after the other.

4.3.12.6. Counting with Dictionaries

One application of dictionaries that frequently comes up is counting the elements in a sequence. For example, say we have a sequence of coin flips:

>>> import random
>>> die_rolls = [random.choice(['heads', 'tails']) for _ in range(10)]
>>> die_rolls
['heads', 'tails', 'heads', 'tails', 'heads', 'heads', 'tails', 'heads', 'heads', 'heads']

The actual list die_rolls will likely be different when you execute this on your computer since the outcomes of the die rolls are random.

To compute the probabilities of heads and tails, we could count how many heads and tails we have in the list:

>>> counts = {'heads': 0, 'tails': 0}
>>> for outcome in coin_flips:
...   assert outcome in counts
...   counts[outcome] += 1
...   <ENTER>
>>> print('Probability of heads: %.2f' % (counts['heads'] / len(coin_flips)))
Probability of heads: 0.70
>>> print('Probability of tails: %.2f' % (counts['tails'] / sum(counts.values())))
Probability of tails: 0.30

In addition to how we use the dictionary counts to count the elements of coin_flips, notice a couple things about this example:

  1. We used the assert outcome in counts statement. The assert statement in Python allows you to easily insert debugging statements in your code to help you discover errors more quickly. assert statements are executed whenever the internal Python __debug__ variable is set to True, which is always the case unless you start Python with the -O option which allows you to run optimized Python.
  2. When we computed the probability of tails, we used the built-in sum function, which allowed us to quickly find the total number of coin flips. sum is one of many built-in function you can read about here.

4.3.13. Functions

You can reuse code by putting it inside a function that you can call in other parts of your programs. Functions are also a good way of grouping code that logically belongs together in one coherent whole. A function has a unique name in the program. Once you call a function, it will execute its body which consists of one or more lines of code:

def check_triangle(a, b, c):
    return \
            a < b + c and a > abs(b - c) and \
            b < a + c and b > abs(a - c) and \
            c < a + b and c > abs(a - b)

print(check_triangle(4, 5, 6))

The def keyword tells Python we are defining a function. As part of the definition, we have the function name, check_triangle, and the parameters of the function – variables that will be populated when the function is called.

We call the function with arguments 4, 5 and 6, which are passed in order into the parameters a, b and c. A function can be called several times with varying parameters. There is no limit to the number of function calls.

It is also possible to store the output of a function in a variable, so it can be reused.

def check_triangle(a, b, c):
     return \
             a < b + c and a > abs(b - c) and \
             b < a + c and b > abs(a - c) and \
             c < a + b and c > abs(a - b)

result = check_triangle(4, 5, 6)
print(result)

4.3.14. Classes

A class is an encapsulation of data and the processes that work on them. The data is represented in member variables, and the processes are defined in the methods of the class (methods are functions inside the class). For example, let’s see how to define a Triangle class:

class Triangle(object):

     def __init__(self, length, width, height, angle1, angle2, angle3):
             if not self._sides_ok(length, width, height):
                     print('The sides of the triangle are invalid.')
             elif not self._angles_ok(angle1, angle2, angle3):
                     print('The angles of the triangle are invalid.')

             self._length = length
             self._width = width
             self._height = height

             self._angle1 = angle1
             self._angle2 = angle2
             self._angle3 = angle3

     def _sides_ok(self, a, b, c):
             return \
                     a < b + c and a > abs(b - c) and \
                     b < a + c and b > abs(a - c) and \
                     c < a + b and c > abs(a - b)

     def _angles_ok(self, a, b, c):
             return a + b + c == 180

triangle = Triangle(4, 5, 6, 35, 65, 80)

Python has full Aobject-oriented programming (OOP) capabilities, however we can not cover all of them in a quick tutorial, so please refer to the Python docs on classes and OOP.

4.3.16. Modules

Make sure you are no longer in the interactive interpreter. If you are you can type quit() and press Enter to exit.

You can save your programs to files which the interpreter can then execute. This has the benefit of allowing you to track changes made to your programs and sharing them with other people.

Start by opening a new file hello.py in the Python editor of your choice. If you don’t have a preferred editor, we recommend PyCharm.

Now write this simple program and save it:

from __future__ import print_statement, division
print("Hello world!")

As a check, make sure the file contains the expected contents on the command line:

$ cat hello.py
from __future__ import print_statement, division
print("Hello world!")

To execute your program pass the file as a parameter to the python command:

$ python hello.py
Hello world!

Files in which Python code is stored are called modules. You can execute a Python module form the command line like you just did, or you can import it in other Python code using the import statement.

Let’s write a more involved Python program that will receive as input the lengths of the three sides of a triangle, and will output whether they define a valid triangle. A triangle is valid if the length of each side is less than the sum of the lengths of the other two sides and greater than the difference of the lengths of the other two sides.:

"""Usage: check_triangle.py [-h] LENGTH WIDTH HEIGHT

Check if a triangle is valid.

Arguments:
  LENGTH     The length of the triangle.
  WIDTH      The width of the traingle.
  HEIGHT     The height of the triangle.

Options:
-h --help
"""
from __future__ import print_function, division
from docopt import docopt

if __name__ == '__main__':
      args = docopt(__doc__)
      a, b, c = int(args['LENGTH']), int(args['WIDTH']), int(args['HEIGHT'])
      valid_triangle = \
              a < b + c and a > abs(b - c) and \
              b < a + c and b > abs(a - c) and \
              c < a + b and c > abs(a - b)
      print('Triangle with sides %d, %d and %d is valid: %r' % (
              a, b, c, valid_triangle
      ))

Assuming we save the program in a file called check_triangle.py, we can run it like so:

$ python check_triangle.py 4 5 6
Triangle with sides 4, 5 and 6 is valid: True

Let break this down a bit.

  1. We are importing the print_function and division modules from Python 3 like we did earlier in this tutorial. It’s a good idea to always include these in your programs.
  2. We’ve defined a boolean expression that tells us if the sides that were input define a valid triangle. The result of the expression is stored in the valid_triangle variable. inside are true, and False otherwise.
  3. We’ve used the backslash symbol `` to format are code nicely. The backslash simply indicates that the current line is being continued on the next line.
  4. When we run the program, we do the check if __name__ == ‘__main__’. __name__ is an internal Python variable that allows us to tell whether the current file is being run from the command line (value __name__), or is being imported by a module (the value will be the name of the module). Thus, with this statement we’re just making sure the program is being run by the command line.
  5. We are using the docopt module to handle command line arguments. The advantage of using this module is that it generates a usage help statement for the program and enforces command line arguments automatically. All of this is done by parsing the docstring at the top of the file.
  6. In the print function, we are using Python’s string formatting capabilities to insert values into the string we are displaying.

4.3.17. Installing Libraries

Often you may need functionality that is not present in Python’s standard library. In this case you have two option:

  • implement the features yourself
  • use a third-party library that has the desired features.

Often you can find a previous implementation of what you need. Since this is a common situation, there is a service supporting it: the Python Package Index (or PyPi for short).

Our task here is to install the `autopep8`_ tool from PyPi. This will allow us to illustrate the use if virtual environments using the pyenv or virtualenv command, and installing and uninstalling PyPi packages using pip.

4.3.18. Using pip to Install Packages

Let’s now look at another important tool for Python development: the Python Package Index, or PyPI for short. PyPI provides a large set of third-party python packages. If you want to do something in python, first check pypi, as odd are someone already ran into the problem and created a package solving it.

In order to install package from PyPI, use the pip command. We can search for PyPI for packages:

$ pip search --trusted-host pypi.python.org autopep8 pylint

It appears that the top two results are what we want so install them:

$ pip install --trusted-host pypi.python.org autopep8 pylint

This will cause pip to download the packages from PyPI, extract them, check their dependencies and install those as needed, then install the requested packages.

Note

You can skip ‘–trusted-host pypi.python.org’ option if you have patched urllib3 on Python 2.7.9.

4.3.19. GUI

4.3.19.1. GUIZero

Install guizero with the following command:

sudo pip3 install guizero

For a comprehensive tutorial on guizero, click here.

4.3.19.2. Kivy

You can install Kivy on OSX as followes:

brew install pkg-config sdl2 sdl2_image sdl2_ttf sdl2_mixer gstreamer
pip install -U Cython
pip install kivy
pip install pygame

A hello world program for kivy is included in the cloudmesh.robot repository. Which you can fine here

To run the program, please download it or execute it in cloudmesh.robot as follows:

cd cloudmesh.robot/projects/kivy
python swim.py

To create stand alone packages with kivy, please see:

-  https://kivy.org/docs/guide/packaging-osx.html

4.3.20. Formatting and Checking Python Code

First, get the bad code:

$ wget --no-check-certificate http://git.io/pXqb -O bad_code_example.py

Examine the code:

$ emacs bad_code_example.py

As you can see, this is very dense and hard to read. Cleaning it up by hand would be a time-consuming and error-prone process. Luckily, this is a common problem so there exist a couple packages to help in this situation.

4.3.21. Using autopep8

We can now run the bad code through autopep8 to fix formatting problems:

$ autopep8 bad_code_example.py >code_example_autopep8.py

Let us look at the result. This is considerably better than before. It is easy to tell what the example1 and example2 functions are doing.

It is a good idea to develop a habit of using autopep8 in your python-development workflow. For instance: use autopep8 to check a file, and if it passes, make any changes in place using the -i flag:

$ autopep8 file.py    # check output to see of passes
$ autopep8 -i file.py # update in place

If you use pyCharm you have the ability to use a similar function while p;ressing on Inspect Code.

4.3.22. Further Learning

There is much more to python than what we have covered here:

  • conditional expression (if, if…then,`if..elif..then`)
  • function definition(def)
  • class definition (class)
  • function positional arguments and keyword arguments
  • lambda expression
  • iterators
  • generators
  • loops
  • docopts
  • humanize

4.3.23. Writing Python 3 Compatible Code

To write python 2 and 3 compatib;e code we recommend that you take a look at: http://python-future.org/compatible_idioms.html

4.3.24. Using Python on FutureSystems

Warning

This is only important if you use Futuresystems resources.

In order to use Python you must log into your FutureSystems account. Then at the shell prompt execute the following command:

$ module load python

This will make the python and virtualenv commands available to you.

Tip

The details of what the module load command does are described in the future lesson modules.

4.3.25. Ecosystem

4.3.25.1. pypi

Link: pypi

The Python Package Index is a large repository of software for the Python programming language containing a large number of packages [link]. The nice think about pipy is that many packages can be installed with the program ‘pip’.

To do so you have to locate the <package_name> for example with the search function in pypi and say on the commandline:

pip install <package_name>

where pagage_name is the string name of the package. an example would be the package called cloudmesh_client which you can install with:

pip install cloudmesh_client

If all goes well the package will be installed.

4.3.25.2. Alternative Installations

The basic installation of python is provided by python.org. However others claim to have alternative environments that allow you to install python. This includes

Typically they include not only the python compiler but also several useful packages. It is fine to use such environments for the class, but it should be noted that in both cases not every python library may be available for install in the given environment. For example if you need to use cloudmesh client, it may not be available as conda or Canopy package. This is also the case for many other cloud related and useful python libraries. Hence, we do recommend that if you are new to python to use the distribution form python.org, and use pip and virtualenv.

Additionally some python version have platform specific libraries or dependencies. For example coca libraries, .NET or other frameworks are examples. For the assignments and the projects such platform dependent libraries are not to be used.

If however you can write a platform independent code that works on Linux, OSX and Windows while using the python.org version but develop it with any of the other tools that is just fine. However it is up to you to guarantee that this independence is maintained and implemented. You do have to write requirements.txt files that will install the necessary python libraries in a platform independent fashion. The homework assignment PRG1 has even a requirement to do so.

In order to provide platform independence we have given in the class a “minimal” python version that we have tested with hundreds of students: python.org. If you use any other version, that is your decision. Additionally some students not only use python.org but have used iPython which is fine too. However this class is not only about python, but also about how to have your code run on any platform. The homework is designed so that you can identify a setup that works for you.

However we have concerns if you for example wanted to use chameleon cloud which we require you to access with cloudmesh. cloudmesh is not available as conda, canopy, or other framework package. Cloudmesh client is available form pypi which is standard and should be supported by the frameworks. We have not tested cloudmesh on any other python version then python.org which is the open source community standard. None of the other versions are standard.

In fact we had students over the summer using canopy on their machines and they got confused as they now had multiple python versions and did not know how to switch between them and activate the correct version. Certainly if you know how to do that, than feel free to use canopy, and if you want to use canopy all this is up to you. However the homework and project requires you to make your program portable to python.org. If you know how to do that even if you use canopy, anaconda, or any other python version that is fine. Graders will test your programs on a python.org installation and not canpoy, anaconda, ironpython while using virtualenv. It is obvious why. If you do not know that answer you may want to think about that every time they test a program they need to do a new virtualenv and run vanilla python in it. If we were to run two instals in the same system, this will not work as we do not know if one student will cause a side effect for another. Thus we as instructors do not just have to look at your code but code of hundreds of students with different setups. This is a non scalable solution as every time we test out code from a student we would have to wipe out the OS, install it new, install an new version of whatever python you have elected, become familiar with that version and so on and on. This is the reason why the open source community is using python.org. We follow best practices. Using other versions is not a community best practice, but may work for an individual.

We have however in regards to using other python version additional bonus projects such as

  • deploy run and document cloudmesh on ironpython
  • deploy run and document cloudmesh on anaconde, develop script to generate a conda packge form github
  • deploy run and document cloudmesh on canopy, develop script to generate a conda packge form github
  • deploy run and document cloudmesh on ironpython
  • other documentation that would be useful

4.3.25.3. Autoenv: Directory-based Environments

Warning

We do not recommend that you use autoenv. Instead we recommend that you use pyenv.

Link: Autoenv <https://pypi.python.org/pypi/autoenv/0.2.0>

If a directory contains a .env file, it will automatically be executed when you cd into it. It’s easy to use and install.

This is useful for

  • auto-activating virtualenvs
  • project-specific environment variables

To use it add the ENV you created with virtualenv into .env file within your project directory:

$ echo "source ~/ENV/bin/activate" > yourproject/.env
$ echo "echo 'whoa'" > yourproject/.env
$ cd project
whoa

To install it on Mac OS X use Homebrew:

$ brew install autoenv
$ echo "source $(brew --prefix autoenv)/activate.sh" >> ~/.bash_profile

To install it using pip use:

$ pip install autoenv
$ echo "source `which activate.sh`" >> ~/.bashrc

To install it using git use:

$ git clone git://github.com/kennethreitz/autoenv.git ~/.autoenv
$ echo 'source ~/.autoenv/activate.sh' >> ~/.bashrc

Before sourcing activate.sh, you can set the following variables:

  • AUTOENV_AUTH_FILE: Authorized env files, defaults to ~/.autoenv_authorized
  • AUTOENV_ENV_FILENAME: Name of the .env file, defaults to .env
  • AUTOENV_LOWER_FIRST: Set this variable to flip the order of .env files executed

Autoenv overrides cd. If you already do this, invoke autoenv_init within your custom cd after sourcing activate.sh.

Autoenv can be disabled via unset cd if you experience I/O issues
with certain file systems, particularly those that are FUSE-based (such as smbnetfs).

4.3.26. Resources

If you are unfamiliar with programming in Python, we also refer you to some of the numerous online resources. You may wish to start with Learn Python or the book Learn Python the Hard Way. Other options include Tutorials Point or Code Academy, and the Python wiki page contains a long list of references for learning as well. Additional resources include:

A very long list of useful information are also available from

This list may be useful as it also contains links to data visualization and manipulation libraries, and AI tools and libraries. Please note that for this class you can reuse such libraries if not otherwise stated.

4.3.27. Jupyter Notebook Tutorials

A Short Introduction to Jupyter Notebooks and NumPy To view the notebook, open this link in a background tab <https://nbviewer.jupyter.org/> and copy and paste the following link in the URL input area <https://cloudmesh.github.io/classes/lesson/prg/Jupyter-NumPy-tutorial-I523-F2017.ipynb> Then hit Go!

4.3.28. Exercises

EPython.1:

Write a python program called iterate.py that accepts an integer n from the command line. Pass this integer to a function called iterate.

The iterate function should then iterate from 1 to n. If the ith number is a multiple of three, print “multiple of 3”, if a multiple of 5 print “multiple of 5”, if a multiple of both print “multiple of 3 and 5”, else print the value.

EPython.2:
  1. Create a pyenv or virtualenv ~/ENV
  2. Modify your ~/.bashrc shell file to activate your environment upon login.
  3. Install the docopt python package using pip
  4. Write a program that uses docopt to define a commandline program. Hint: modify the iterate program.
  5. Demonstrate the program works and submit the code and output.