Building Async Non-Blocking MicroServices Using Spring 4

In this blog, we will see how to develop non-blocking REST services using Spring MVC. We will also see the vast difference in scalability that non-blocking services provide compared to traditional blocking services.We will use Spring Boot to create a web app for deployment in a Servlet 3.0 compliant Tomcat server. Finally, we will use JMeter to load test the REST services.With an ever-increasing number of connected devices, e.g. mobile devices and Internet of Things, the requirement of handling a large number of concurrent requests in the application servers becomes more critical in many projects.The key technology for an application server to meet this requirement is the capability to handle requests in a non-blocking way, i.e. without allocating a thread to each request. This is equally important for both incoming and outgoing requests. Servlet 3.0 specification has standardized on how to perform non-blocking processing towards the underlying web servers and frameworks. Spring 4.0 was released with an unparalleled simplicity for developing non-blocking REST services using Spring MVC and deploying them on any Servlet 3.0 compliant web server using Spring Boot

Blocking REST service with Spring

@RequestMapping(method=RequestMethod.GET, value="/users/{id}", 
produces=MediaType.APPLICATION_JSON_VALUE)
public User getUser(@PathVariable("id") String id)
{
   //Logic to get customerreturn user;
}

The problem, from a scalability perspective, is that the request thread is blocked during the processing of this method. If the method needs to make a long-running call to an external resource, such as another REST or a database, the request thread will be blocked during the wait for the external resource to respond. The following picture illustrates this:

To avoid the blocking of the request thread the programming model is changed to a callback model. The REST service doesn’t return the actual result to the Servlet container but instead, an object called a DeferredResult, that will receive the result at some time in the future. The result will be filled in by some other thread, typically using a callback-object. Spring MVC will hand over the result to the Servlet container that sends it back to the client. In the REST service, we have to initiate this processing before we return the DeferredResult object to the Servlet container like:

@RequestMapping(method=RequestMethod.GET, value="/users/{id}", 
produces=MediaType.APPLICATION_JSON_VALUE)
public DeferredResult<User> getUser(@PathVariable("id") String id)
{
	DeferredResult<String> deferredResult = new DeferredResult<String>();
    //Application Logic 
	return deferredResult;
}

The callback object will be called sometime in the future and it will then set the response in the DeferredResultobject to complete the processing of the request. The actual processing of the logic can also be done in a non-blocking way

Performance Comparison between Blocking and Non-blocking implementation

We created 2 simple REST APIs for doing the comparison

package com.sidd.nbs;

import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.context.request.async.DeferredResult;

import java.util.Timer;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * Created by Siddharth on 4/3/18.
 */

@RestController
public class MyRestController {

    Timer timer;
    ScheduledExecutorService ses;
    public MyRestController(){
        timer = new Timer();
        ses = new ScheduledThreadPoolExecutor(10);
    }
    @RequestMapping("/process-blocking")
    public String blockingProcessing(@RequestParam(value="processingTime") long processingTime) throws InterruptedException
    {
        long sTime = System.currentTimeMillis();
        Thread.sleep(processingTime);
        String result = "SUCCESS";
        long eTime = System.currentTimeMillis();
        long timeTaken = eTime-sTime;
        return result + "   /process-blocking completed in " + timeTaken + " Ms";
    }
    @RequestMapping("/process-non-blocking")
    public DeferredResult<String> nonBlockingProcessing(@RequestParam(value="processingTime") long processingTime) throws InterruptedException
    {
        DeferredResult<String> deferredResult = new DeferredResult<String>();
        Job j = new Job(deferredResult, processingTime);
        ses.schedule(j,processingTime, TimeUnit.MILLISECONDS);
        return deferredResult;
    }
}
  

/**
 * Created by Siddharth on 4/4/18.
 */
public class Job implements Runnable{

    DeferredResult<String> deferredResult;
    long processingTime;

    public Job(DeferredResult<String> deferredResult, long processingTime){
        this.deferredResult = deferredResult;
        this.processingTime = processingTime;
    }
    @Overridepublic void run(){
        String result = "SUCCESS   /process-non-blocking completed in " + processingTime + " Ms";
        deferredResult.setResult(result);
    }
}



/**
 * Created by Siddharth on 4/3/18.
 */

@ComponentScan(basePackages = "com")
@EnableAutoConfiguration
public class Application {

    public static void main(String[] args){
        SpringApplication.run(Application.class, args);
    }
}

JMeter Script

I wrote a simple JMeter test to invoke these 2 services with 1000 threads for 5 minutes. See the huge difference in throughput of these services. Blocking service has around 600 TPS whereas the non-blocking service as around 3000 TPS! Latency is 1500 Ms for blocking whereas 320 for non-blocking!

Blocking service

Non-Blocking service

You can find the code here https://github.com/spatnaik77/async-nonblocking-service