An abstraction for verification of test outcome

While developing infrastructure for automated tests, one of the needs was to provide generic means for developers to verify outcome of a test step.

Context : After performing a test step, the verification logic will be called. This logic could be re-used in many other test cases. An example verification could be executing some query to collect data and verify if it matches the expected. The application under test serves both synchronous as well as asynchronous requests. Hence a test step may submit a synchronous request or an asynchronous one. Intern the verification may complete immediately in case of synchronous task or may have to wait for the completion of application activity initiated by the asynchronous request.

Now thinking on how to abstract this; the expectation at the outset is that it allows test developers to add new verification methods - i.e verification strategies. In programming terms we would define an interface Verifier. Implementations of this interface can decide on the inputs.

So what should be the interface definition? If we consider the state of the application being tested, then any action initiated by a test step, would change the state. Then the task of the verifier is to check if this changed state is as expected. So, the Verifier would execute some logic to gather and process the data needed. It would tell whether the verification is complete i.e. whether system has attained the desired state. And then it would tell if outcome of the step - state change - is as expected i.e whether verification is success. Optionally, due to the asynchrony of the application, we would want to timeout the verification.

With above thoughts the interface definition is as below -

interface Verifier {
  void execute(); 
  boolean isComplete();
  boolean isSuccess();
  void setTimeout();
}        

How do we execute the verifier ? A verifier can be executed in two ways - execute once in case of a synchronous task, execute repeatedly in case of asynchronous task till verification isComplete. After the verifier is executed, assert if verification isSuccess. The developer can decide to call either way.

Though this abstraction is devised during test automation in the development of an - automated tests development framework, I feel, it can find other use cases.