Data Driven Decisions @ Speed of Thought

Your Data, Your Advantage

Everyone’s talking about companies being data driven. Great, but driven to do what? The typical answer I receive states something like; companies have done a good job of capturing and analyzing batches of data, which is then used for helping to shape their go forward strategies. But it’s 2017, so I’m asking the question; is that enough? It’s been 10 years since we all rallied around Big Data, so it must be time for the next thing, right? Yep! Big Data has led us to Fast Data, which is already mainstream and potentially being deployed by your competition.

Big Data is data at rest, the historical data businesses collect and learn from past output. Fast Data is live actionable data that provides real-time and predictive insights. Fast Data operationalizes the learning and insights that companies derive from their Big Data. What’s different today, in 2017, is the realization the Data has a shelf life.  I’m suggesting the next Data craze will be the need for companies to leverage their various data streams, augmented in real time with contextual information, so they can instantly act upon and predict.

To be clear, Fast Data isn’t a replacement for any of the previous concepts of Big Data, Data-Warehousing or Machine Learning. Rather, it’s the latest application. The objective is to analyze the various data sets in real time, when it is as actionable and relevant as possible. The speed at which insights are gained and ultimately how they are put-to-use, defines your business success strategies in 2017 and moving forward.  

This is Fast Data.

By definition -- “Fast data is the application of big data analytics to smaller data sets in near-real or real-time in order to solve a business problem or to create business decision. The goal of fast data is to quickly gather and mine structured and unstructured data so that action can be taken.”

Fueling the Demand for Managing Data Velocity

There are a couple of reasons why the appeal of Fast Data has grown. As increased volumes of data become commonplace across many industry sectors, the value of applying a Fast Data strategy, as an end-to-end solution, has become the norm.

Another factor which has fueled the demand for solutions that manage high velocity of data is the ‘Internet of  Everything. There are more devices than ever before, there are more people adapting technology then ever before, there are more softwares than ever before thus generating more data in real time then ever before. In fact, the number of devices over the next 5 years is expected to increase by 22X. Increased connectivity leads to higher volumes of data generation at a substantially faster rate. It also allows end users to leverage real-time information across work and personal interactions.

As more and more devices come on board we’re also seeing an uptake in solutions that require device to datacenter (D2D); Fast Data is an important element in connecting streams of data together from device to the datacenter. In short, data that exists beyond the companies four walls can also be leveraged. For example, an organizations data, plus the data their customers are creating, plus public sentiment (social media) data, plus public data via governmental channels -- results in the capabilities of leaders and managers of the organization making data driven decisions @ speed of thought.

Bottom line, touch points are increasing.  As more and more devices add to the conversation, so does the demand for instantaneous responsiveness and improved experiences. Fast Data delivers the most current and real-time information to customers, organization service representatives, business analysts, and others in a real-time operationalized manner.

Run your Business Smarter with Fast Data

Every business wants to be smarter about how they do business. Digital transformation is the current buzz word. Velocity is a critical component to achieving this end goal. By capturing data faster and moving it faster means being able to analyze it and act on it faster. For many of the most cutting-edge applications such as; demand forecasting, fraud detection, compliance reporting and failures in manufacturing process, data quickly loses its value if it can’t be analyzed and acted on immediately. For example:

-       Data scientists at Walmart were putting together the latest iteration of the supermarket giant’s data framework, a digital transformation related decision was made; that only the previous few weeks’ worth of transactional data would be streamed through their pipelines. Everything else was regarded as too untimely to impact any real value in their demand forecasting.

-       In banking and insurance, enterprises are finding that immediate access to the most relevant data is vastly more valuable than petabytes of historical data that has been stored in warehouses for years, gathering virtual dust (and incurring storage and compliance expense) because someone though that it may one day be useful.

-       It’s also vital for the complex fraud and error-prevention algorithms employed by banks, where the problems caused by errant transactions can be magnified if they are not detected and rectified immediately.

-       Another application is “smart” power grids, where demand can be forecast and resources allocated across the grid to ensure supply is available when and where it is needed. As an example, this technology is being applied in smart city projects around the world.

-       Fast Data has become essential for tasks such as recommendation and personalization engines, where information about a customer needs to be processed as soon as they visit a web page or walk into a store.

-       Increasingly, systems monitoring and responding to unstructured data such as posts made to social media, or audio data gathered from customer service calls, are also adding value when made available in real or near-real time. One example is video security systems, where predictive modelling can be used to raise alerts when suspicious or abnormal activity is picked up by surveillance devices.

The open source community has embraced the concept of Fast Data, with platforms such as Apache Apex, Flink, Spark, Kafka and Storm becoming popular in recent years due to their ability to process streams of data with lightning speed. To achieve this, data is often processed in-memory, cutting down the time needed to spin up physical hard disks and seek the information stored on them. An important differentiator is that “fast” Big Data is generally processed as a stream, while “slow” Big Data is processed in batches.

In summary, more and more applications of fast data are quickly turning the Big Data landscape into a racetrack.  The following are four key value drivers that I’m seeing from leveraging Fast Data solutions:

Building new services

Ability to process highly-dynamic data efficiently, provides broader insight into customers and allows companies to offer differentiated services or products that were not possible before.

Improving customer experience

By leveraging the latest customer information, including most recent contacts across all customer touch-points, support issues, and operational data, companies can offer real-time personalized service at every customer interaction.

Increasing operational efficiencies

Ability to process highly-dynamic data in a rapid fashion, brings advanced optimization opportunities across the business, such as in asset utilization, workforce management, or inventory management.

Developing higher quality in operations

Fast Data expands the breadth of data analysis and delivers great predictability and visibility which enable higher quality in business operations.

Summary

While the term Fast Data might be new; it’s a set of mature ideas and technologies around the rapid processing of high volumes and varieties of data delivering real-time analytical insights. It’s recently been impacted by the explosion in volume and variety of data; the ubiquitous nature of devices & users (internet of  enverything), and ultimately the heightened expectations of enterprises for personalization. No longer do we see Fast Data in only financial stock exchanges or limited to Hadoop based deployments; instead it’s becoming more and more pervasive across every industry as companies are recognizing the need to run their businesses in real-time.

Most companies use Big Data to learn from the past, however today you must leverage Your Data into Your Advantage, in the form of Fast Data.

Mactores partners with leading emerging technology and cloud vendors to deliver the best in breed technology and services, delivering innovative solutions to solve unique business challenges. Reply below in the comments field and “let’s talk Fast Data” to share ideas on ways to operationalize your Big Data.

 Founded in 2003, Mactores is a global technology consulting company with operations in The United States, India and Australia. Driven by technology, we focus on delivering solutions on Cloud, Intelligent Analytics, Fast Data, Internet of Everything & Machine Intelligence. With over 300 happy customers and winner of over 50 international awards and recognitions, Mactores has become a trusted technology partner for enterprises worldwide.