API Automation Testing Using Karate Framework

Background 

Microservices based architecture is the basis of fast paced development and delivery of services, which can independently be deployed on production without any downtime. Agile practice helps create a framework for fast development, which needs the mandatory support of automated testing to go in parallel with development. This parallelism can be called as “in sprint” automation.  

Many organizations have achieved “in sprint” automation successfully, including our team here at Frontdoor Inc. We’ve used Karate DSL to automate the REST APIs – in parallel to the actual development of the APIs. This article will help readers understand and learn about our implementation of Karate DSL and help achieve the “in-sprint” automation along with the answers to “Why”, “What” and “How”.  

Why did we decide to use Karate DSL? 

• We wanted to build an automation framework which allow us deliver “in-sprint” API automation, which can easily be used by Developers or Manual/Functional QAs. 
• We wanted to build an automation framework, which could be extended for API performance Testing. 
• We wanted to build an automation framework, which could be extended for use as an API mock server. This is especially useful if you use non JSON payloads. 

What is Karate DSL? 

• Karate DSL is an open-source framework that can be used for API testing, front-end UI testing, and performance testing as well.  
• Karate has a BDD syntax, which is language-neutral, and is easy to understand even for non-programmers. 
• Karate has built-in JSON and XML assertions, making it very easy to validate the response from the API. 
• Karate is fast and reliable as we don’t need to compile the code for executing the test cases. The test cases hit the endpoints directly and get the response. 

How to Setup Karate API Automation Framework ?

Setting up of Karate API Automation Framework is quite easy. We would just have to add few dependencies to our pom.xml and that’s it. Of course, you will have to setup a maven project through your IDE first and to do so you also need to install Java.  

Let’s have a look at the pom.xml, which we have used -

At this point, you have the project structure ready to start writing the tests. Let’s create one package folder and name it “features” under src/test/java. This is where all the feature files will reside. 

Folder structure 

The folder structure should look like- 

All the test cases are written under src/test/java directory and you can create the sub-directories according to your requirements. 

Let's go through each file one by one - 

.feature files 

•  All the test scripts must be written as .feature files, which contain the assertions and the request data. The data can also be provided from external files.  

•  Feature: This keyword is used to identify the feature in the file 
• Background: This keyword is used for defining the variables that will be used in the particular .feature file and will be used by all the requests in the feature file. 
• Scenario/Scenario Outline: It is the description of each test cases and helps us identify that what is happening in the test case. 
• Given: This keyword is like a pre-request script where we give the data which we are familiar with. For example, URL can be different for different types of request. So, it can be defined here. 
• When: This keyword is used to define the action to be performed. 
• Then: This keyword is used to verify the result of the request. It is used to check that the request executed correctly or not. 
• And: This keyword is used for giving multiple when and then in any requests and also for putting up assertions as well. 
• Example: This keyword follows the Scenario Outline keyword. This is used to execute a test with multiple set of data.  

Runner files 

• These are the .java files, which are used for executing the .feature files in the project. 
• We have to pass the name of our feature file and it will execute all the features from that file. 
• If we want to execute all our feature files from a directory, we will have to give the path of that directory. 

Test Report 

We have used cucumber-reporting open-source library for reporting the test result because Karate's integration includes the HTTP request and response logs in-line with the test report, which is extremely useful for troubleshooting the test failures.  

The report looks like-