Dev or Ops oriented testing

Development vs. Operations testing - I’ve just realized, that there is such approaches.

The live-cycle of an actively developed software typically employs compilation, packaging, and installation.

When the compilation is needed to execute software and related tests, packaging and installation are not necessary.

Typically developers do not build packages and do not install them during development and test execution. It allows to iterate faster through an implement-test loop, and it does not require to install this software into the system (which typically requires dependencies of specific versions and configuration of the system).

Installation typically involves unpacking software, checking the target environment and dependencies there, potentially the installation of these dependencies, and configures the system for the intended use.

When writing test-automation scripts, we can follow the development workflow. It will test not only the software itself but also that the development workflow is still working. Along with this, such an approach typically allows

• building only the parts of software which are required for tests,
• split the test execution onto multiple working nodes
• run several tests in parallel, since such executions are naturally isolated and do not rely on the system-wide configuration or dependencies.

If we follow the Operations workflow we include not only software testing in the testing cycle but the packaging and installation process as well. Note that these scripts typically do not have tests, and are not executed often enough to be sure, that it is working properly. The problem with such an approach is that it typically requires the hardware or virtual environment close to production one. Often it allocates significant resources that may stay idle when no tests are performed. And we do not invest much into development-friendly infrastructure.

Luckily these approaches are not mutually exclusive, and you can implement them both. For example, focus on development-focused tests. But still, execute some limited amount of development-oriented tests.