Automation using OpenStack Tempest Plugin

Major challenge in implementing the OpenStack Cloud with different deployments and configurations is testing the Cloud Functionalities. How can we ensure that our OpenStack Cloud functionalities are working properly?

Testing using Tempest is the answer. Tempest, OpenStack’s official test suite, is a powerful tool for running a set of functional tests against an OpenStack cluster. Tempest also acts as gate in OpenStack CI, which lets us avoid merging the changes that break things.

Current OpenStack Tempest supports scenario cases which are run against the CI and for doing the Driver Qualification e.g. on Redhat OpenStack Suite as well. However problem is:-

• There are some more steps to do when we do the manual testing for our backend driver like verification of results on our storage backend and at various other places.
• Also testing an OpenStack deployment by running 100s of detailed manual test scripts is a very time consuming effort. 

This essentially means we cannot directly use the tempest for our regular testing activities. Luckily, the Tempest test framework has a plugin interface and scenario test framework that can be leveraged for automating detailed test scenarios.

So we can define our plugin to handle the needs as below:-

Add custom framework class which will be responsible for

• providing the required API’s for backend integration & validation
• providing API for CLI validation
• creating the custom cases per our need

Add Utils for REST Handling

• Create the respective REST request wrappers so that test engineer can use it without requiring the knowledge about the REST details of respective OpenStack Client

This will help in performing end-to-end testing for OpenStack framework like:-

• Integrate testing of OpenStack with third-party tools / APIs
• Complex test scenario testing
• Stress testing
• New API and CLI tests

Plugin structure

A Tempest plugin is basically a Python package that contains your custom test cases. It needs to be installed on the local or virtual environment (venv) where Tempest is located. All of your installed plugins will be discovered by Tempest when it runs without you needing to do anything extra.

Directory structure

setup.cfg

setup.py

README.rst

plugin_dir/

      __init__.py

      config.py

      plugin.py

      tests/

            __init__.py

           api/

                 __init__.py                                                                                     

           scenario/

                   __init__.py

                  test_my_scenario.py

setup.cfg: In this file the most important thing is to specify the entry point, which is the location of the project's main class that will allow Tempest to find the plugin.[entry_points]

tempest.test_plugins =    first-tempest-plugin = first_tempest_plugin.plugin:MyPlugin 

README.rst: Information about the project.

plugin_dir: Main folder, usually has the name of the plugin. Example: first_tempest_plugin.

• plugin.py: Plugin's main class, entry point that contains all the configuration that Tempests needs to run the plugin. You just have to implement the methods from the abstract class.
• tests: Folder where all the tests have to be.
• api: All test cases related to API testing.
• scenario: All test cases related to scenario testing.

Running the Plugin Cases:-

Once the above configuration is in place, Tempest framework will detect this new plugin. We can run all our normal tempest cmds then.

References:-

Refer OpenStack Tempest Plugin Docs