Pytest

Pytest is a utility to unit test python code. We use pytest and not __main__ to test all functionality so they can be automatically run and reports can be generated. A project that does not have a sufficient number of tests to make sure the module works can not be accepted.

Installation

The pytest module can be installed with the help of pip utility:

$ pip install pytest

This will install the pytest module in the current Python distribution. All tests are included in the folder tests. For example for the cloudmesh-cloud module they are in

• https://github.com/cloudmesh/cloudmesh-cloud/tree/master/tests

If the order of the tests is important make sure the tests are specified in alphabetical order while introducing a numbering such as

+cm
  + cloudmesh
  + tests
    - test_01_topic1.py
    - test_02_topic2.py
    - test_03_topic2.py

Cloudmesh Debugging Information

To switch on debugging information for the tests you can do this with

cms set trace=True
cms set debug=True
cms set verbose=10

After you are done testing, you may want to set them back to

cms set trace=True
cms set debug=True
cms set verbose=10

You can also set them within a test with the command

from cloudmesh.common3.Benchmark import Benchmark

Benchmark.debug()

Test Specification and Execution

A simple example for a test is

• https://github.com/cloudmesh/cloudmesh-cloud/blob/master/tests/test_key.py

Note that all test python programs have specific function names of the form

def test_topic (self)

Through the class decorator, they are executed in order

A test may also have a method def setup(self) which is run prior to each

other method.

A test method includes one or multiple asserts that check if a particular test succeeds and reports this to pytest to expose the information if a test succeeds or fails, when running it

To have better visual separation of tests with the –capture=no option, all pytest functions in cloudmesh start with a HEADING(), which will print a banner with the function name. Invocation is simply done with the comment lines you see on top that you will include. In our case the test is called test_key.py so we include on the top:

###############################################################
# pytest -v --capture=no tests/test_key.py
# pytest -v  tests/test_key.py
# pytest -v --capture=no -v tests/test_key.py:Test_key.<METHODNAME>
###############################################################

Make sure that you place this comment in your tests. to remind the developers how to execute your test from the main directory. If you need special instructions, please add them here.

The following example is our simple pytest for key. The file is stored at tests/test_key.py

First, we import the needed classes and methods we like to test. We define a class, and then we define the methods. such as the setup and the actual tests.

You run it with

$ pytest -v --capture=no tests/test_key.py

###############################################################
# pytest -v --capture=no tests/test_key.py
# pytest -v  tests/test_key.py
# pytest -v --capture=no -v --nocapture tests/test_key.py:Test_key.<METHIDNAME>
###############################################################
from pprint import pprint
from cloudmesh.common.Printer import Printer
from cloudmesh.common.util import HEADING, banner
from cloudmesh.management.configuration.SSHkey import SSHkey
from cloudmesh.management.configuration.config import Config
from cloudmesh.common.StopWatch import StopWatch

@pytest.mark.incremental
class TestKey:

    def setup(self):
        self.sshkey = SSHkey()


    def test_find_key(self):
        HEADING()
        StopWatch.stop("key")
        pprint(self.sshkey)
        print(self.sshkey)
        print(type(self.sshkey))
        pprint(self.sshkey.__dict__)

        assert self.sshkey.__dict__  is not None


    def test__key_from_git(self):
        HEADING()
        config = Config()
        username = config["cloudmesh.profile.github"]
        print ("Username:", username)
        StopWatch.start("get from git")
        keys = self.sshkey.get_from_git(username)
        StopWatch.stop("get from git")
        pprint (keys)
        print(Printer.flatwrite(keys,
                            sort_keys=("name"),
                            order=["name", "fingerprint"],
                            header=["Name", "Fingerprint"])
              )

        assert len(keys) > 0


    def test_results(self):
        HEADING()
        banner(f"Benchmark results for Keys")
        StopWatch.benchmark()

The output with pytest tests/test_key.py does not provide any detail, but just reports if tests fail or succeed.

----------------------------------------------------------------------
Ran 2 tests in 0.457s

OK

The output with pytest -v tests/test_key.py results in

tests.test_key.TestName.test_01_key ... ok
tests.test_key.TestName.test_02_git ... ok

----------------------------------------------------------------------
Ran 2 tests in 1.072s

OK

During the development phase, you want to use

pytest -v --capture=no tests/test_key.py

which prints all print statements