Non-Functional tests using Postman

In this article I am going to suggest solutions for two NFR related problems that in-sprint API testing team may face.

Problem Statement 1: Broken access control

When you are working on project where your APIs are consumed by both internal client app teams (B2C) and are exposed to third party for integrations (B2B), then any bug in JWT token roles will leads to cyber security issues. Failures typically lead to unauthorized information disclosure, modification, or destruction of all data or performing a business function outside the user's limits. This may not be caught in typical functional testing as both the teams testing with B2B and B2C token will see app working properly.

Problem Statement 2: Performance degradation

When there are multiple deployments per day in lower environment, there is chance of making mistakes that leads to delay in API response. Ideally infrastructure in lower environment will not be the same as Production and these delays (due to code changes) if ignored as infra related issue will lead to performance issues in Production. Assumption here is the delay is very marginal among the builds such that testing team is not realizing the delay in response or even your automation scripts are breaking.

Solution for Statement 1:

We can have a check in smoke test suite by parsing the JWT token and verifying the roles in payload whether the token is getting generated accordingly. Any JWT token will have header, payload and signature details. This needs thorough check.

This can be done by decoding the JWT token in https://jwt.io Refer the sample token decoded.

Also, this can be automated in postman by using CryptoJS module inside postman tests. Parsing and verification can be performed by leveraging any tests that are already automated in postman i.e., any token generation tests. Parsing can be done as shown below:

var data = JSON.parse(responseBody);

var tempvariable = data.token.split(".");

var rawHeader=CryptoJS.enc.Base64.parse(tempvariable[0]);

var decodedHeader = CryptoJS.enc.Utf8.stringify(rawHeader);

By stringifying the JWT token, header can be parsed as normal JSON thereafter. Similarly, payload and signature too can be stringyfied & validated. If this validation is added to the smoke suite, any change in the payload can be arrested in early stage of development.

Solution for Statement 2:

When there is delay in response time in lower environment & if it is very marginal i.e., it is not causing any failures in functional tests then there is chance it will be missed. Same can be assumed as environment specific slowness as in most of the cases Testing team will not have luxury to have Prod like infrastructure.

So, it is always better to have separate test automation suite to capture the response time of the critical APIs. This should not be added in the existing functional test cases (as hard assertions), because we do not want the test suites to stop execution when there is marginal delay in API response.

Instead, there must be a separate suite with hard assertions to check if the API response time is meeting the SLA. It is also good to have response time logged in the report when considering these kind of suites which can be used in future to check since when the slowness in response is happening.

Logging response time in postman can be easily done with the below command and assertion can be made accordingly.

console.log(pm.response.responseTime);