Thought of the day: Batch processing is dead (again)

Every time a new technology shows up in the technology stack, its advocates are quick to dismiss the “old” ways of doing IT. They believe that the “new” shiny toy has a better solution for everything. Don’t believe me? Let’s look at some technologies:

When SAP released SAP R/2 and SAP R/3 in the 90s, the theory behind it was that processing would move from batch to real-time (in case you wondering – the “R” stands for real-time). When SOA architecture became the new buzz, people proclaimed that batch scheduling was dead, as the Service Bus would replace all those needs.

With Hadoop, people were quick to point out that all automation needs can be solved with Oozie, and even most Databases have an integrated Scheduler, but while all those automation technologies are fine for the silo they serve, none address the overarching need of an organization to visualize and execute end-to-end business processes. 

Just like a single server or application, SAP does not live in isolation, and neither does a single technology. Just like an SAP system has to connect to other applications to get its data or direct its output to another application, all new technologies need to be integrated into existing ecosystems and processes.

We can take this thought even further – think about DevOps today. Application deployment has the same traits we already know and love from batch processing: repeatable and predictable run-defined processes.

It’s no coincidence that the Operations side of DevOps uses automation tools like Automic to manage their end of the deployment deal. Yes, the addition of environment, source code repository and versioning information provide the base of the Continuous Delivery that everyone strives for, but at the end of the day it is the orchestration and automation of those tasks on a single platform that makes it repeatable and predictable. 

That’s why it is so important that modern automation houses such as Automic keep up with new technologies and industry standards in order to maintain an ability to bridge the islands of the “old” and “new” worlds. This minimizes the disruptions and risks that can occur when adopting new technologies or methods.

As you can see, batch automation is not dead. It has merely elevated itself to become part of the new ‘Just in Time’ or ‘Service-Oriented’ thinking that drives many businesses today.

Dave Kellermanns