Structured Testing Framework

Structured Testing Framework

Long Running Tests
Author: Serjik Zargarian

Background
Test cases spanning over several days are inherently difficult to perform manually. Things like;

• Over night orders
• Option expiration
• Settlements

And basically events occurring at T+0…T+n…T+x.
There are usually several tasks involved on each step.

• System configuration and database setup.
• System start on specific business date (T+n).
• Initiating system load.
• Caring out test case(s) and verification(s).
• Persisting test data.
• Preparing system shut down.
• Moving to a new business day and starting all over again!

Failing on any step often results in repeating all the scenarios from the scratch.

Mission
To create a framework which will automate the involved procedures and minimize the amount of manual interference.
The framework should also utilize available resources and minimize the over head in terms of code mass by offering some basic functionalities.

The Framework
Is an xml-based Protocol in which the test cycle is described. For instance to

• Start/stop servers
• Add some load
• Run a jUnit test etc

The Parser
Interprets and executes the test script. It also offers some basic functionalities, e.g;

• Define a new business date (fixed-or floating).
• Back up/load DB.
• Stop/Start servers.
• Restore the system.
• or, tweak it to your needs.

Running the script
To run, execute the cmd:$./parser.sh –tc test.xml
For help: $./parser --help

A Simple Demonstration
Run a set of junit tests on specific days. Let ANT execute the tasks and store the results in an xml- or other format.
You could also let Jenkins run and display the results of the executions.

Test script example

<root name=”Project_Name" proj_test_path=”Integrationtest" >
    <testsuite name="Over Night Orders" >
        <newday name="Day0" restartServers="true" skipWeekend="false" weekends="Sat|Sun" date="2015-03-27">
            <jarfile name=”overnightOrders-src" dir="${HOME}/8.6.0-SNAPSHOT/lib" >
                <package name="com.example.testcases.longrunning" >
                    <exec name="overnightOrders Day0-1" class=" OvernightOrdersDay0" method=”Day0_1" arg="x y z" />
                </package>
                <package name="com.example.testcases.longrunning" >
                    <exec name="overnightOrders Day0-2" class=" OvernightOrdersDay0" method="Day0_2" arg="x y z" />
                </package>
            </jarfile>
        </newday>
    </testsuite >
</root>

XML Built-in tags
<root>
<server>
<cmd>: Bash command invocation.
<invoke>: Built-in function invocation.
<testsuite>
<newday>
    -restartServers: true/false
    -skipWeekend:true/false
    -Weekends: ”Sat|Sun” or ”Fri”
    -date: Fixed or incremental
<jarfile>
<package>
<exec>: Test to execute.

Example ANT
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:antcontrib="antlib:net.sf.antcontrib" default="${ant.target}" basedir=".">
    <target name="${ant.target}" depends="setTestClasspath">
        <junit fork="yes" forkmode="once" haltonerror="no" haltonfailure="no" printsummary="yes" showoutput="true" outputtoformatters="true">
            <formatter type="xml" />
            <test name="${test.package}.${test.class}" methods="${test.method}" outfile="reports/TEST-${test.class}-${test.method}-${NOW}">
                <formatter type="xml"/>
            </test>
        </junit>
    </target>
</project>