Breaking Down Buzzwords - 'Big Data and Analytics'

Hello and welcome to the second installment of ‘Breaking down Buzzwords’, BdB for short. BdB was a concept I came up with a little while back when I decided I’d had enough of people using buzzwords without really knowing what they mean, while expecting everyone else to know this crucial bit of information.

‘Big Data and Analytics’ are the buzzwords we’ll be breaking down this time around; as always let’s begin by splitting out and defining the two buzzwords.

Big Data – This is a catch-all term for sets of data that are too large and/or complex for traditional data processing applications to handle.

Analytics – Encompasses a number of methods that analyse data sets for patterns, trends, and correlations before providing valuable findings via a dashboard or otherwise easily-digestible interface.

It’s important to make the distinction between Big Data and Analytics. While most people speak about ‘Big Data’ they are in fact referring to Analytics and the way in which it provides value from Big Data.

Big data is what your earphones look like when they’ve been in your pocket for an hour – a tangled mess. Analytics is your brain and hand-eye coordination that untangles your earphones to give you a usable tool to listen to music with.

Types of Analytics 

Generally speaking there are four widely accepted forms of Analytics, - Descriptive, Diagnostic, Predictive, and Prescriptive. They work in a supply-chain style - Descriptive Analytics is the most basic, or raw, form of Analytics and at each following stage more value is added until you reach Prescriptive Analytics where there is theoretically a ‘final product’ delivering immense value. As is common with technology the reality is slightly different… Let’s explore the various Analytics out there.

Descriptive analytics – “What happened?”

Descriptive Analytics takes large volumes of historical big data and provides the results in an easily digestible dashboard. Social media platforms provide some of the more readily accessible sources of Descriptive Analytics. Their use of descriptive analytics is generally based on simple metrics -  for example once this article is published on Pulse my dashboard will show me how many users have viewed the post, the number of shares, daily rates of likes and comments; all useful insights into how well received the BdB series is. Twitter analytics is much the same - number of retweets per tweet, number of interactions, website views etc.

Think of descriptive analytics as performance measurements or KPIs that allow you to see what’s worked, and what hasn’t. Importantly, Descriptive Analytics when scaled up isn’t quite as good at telling you why something has or hasn’t worked. For that, you’d need to make use of Diagnostic Analytics.

Diagnostic analytics – “Why did it happen?”

Diagnostic analytics are used to determine why something happened, and are usually deployed on top of a Descriptive Analytics solution. Diagnostic analytics comes into its own by again taking masses of data and simplifying it to a few digestible outputs. In this example a diagnostic analytic solution may let us know there were 20 successful hacks last month (descriptive) before going a step further by identifying the fact that each of them came via an open port, or perhaps via that website we used to illegally stream Game of Thrones... We now know not only what happened, but why it happened. We can now more accurately take action to prevent the undesired occurrences.

While I’ve used a negative example of being hacked, the opposite is also true.

Perhaps mobile data usage across the South East was up 20% in the last week (descriptive). The why may be due to the fact that a major internet provider had a massive outage (diagnostic). While this seems like a bit of a leap it’s a viable example due to glorious Big Data. By data mining social media, news outlets and the like, diagnostic analytics will be able to marry the increased data usage with the thousands of tweets complaining of an internet outage as well as news reports.

Diagnostic analytics can save organisations millions as they will be able to find bugs and security breaches more quickly and won’t panic-upgrade infrastructure to cope with increased usage. This means more secure software and better, more reliable devices for us as consumers.

Predictive analytics – “What is likely to happen?”

Predictive Analytics at its simplest takes historical patterns and correlations before providing predictions based on the continuation of those trends. Additionally, variations of Predictive Analytics allow organisations to model different scenarios by tweaking specific variables (sales volumes, product colour, launch country, consumer age) via the Monte Carlo modelling method.

Think of Predictive Analytics as an organisation’s crystal ball that allows them to set realistic goals, manage expectation, realise more effective planning, and release more customer value-driven products. Predictive Analytics gives organisations insight into where to direct R&D funding, when a product may be ready for release, and the types of functionality that should be included depending on consumer sentiment.

What does this mean for us as consumers? Organisations should ideally know exactly what we value and desire in goods and services. This will allow them to develop and deploy higher quality products that gel with our wants and needs in future, which can only be a good thing.

Common examples of predictive analytics can be found in weather forecasts to financial market performance and even variations in sales volumes.

Prescriptive analytics – “What should I do about it?”

Prescriptive Analytics is Predictive Analytics on steroids. In short it performs predictive analytics but goes a step further than simple forecasting and uses the likes of cognitive machines and machine learning to provide recommendations as to the best course of action one should take to reach a desired outcome.

Banks and mortgage lenders are currently taking up this form of analytics when evaluating mortgage and loan requests. By taking data feeds from numerous sources, a lender can see a person has been on-time paying bills each and every time (descriptive), they’re in regular employment and their salary dwarfs their expenses (diagnostic), all things being equal it is likely this person will continue to pay their bills in full and on time (predictive).

By analysing the outputs from the descriptive, diagnostic, and prescriptive analytics it’s most likely the Prescriptive analytics will suggest the person be given the greenlight for a loan. Of course, the opposite is also true. Prescriptive Analytics is causing a shake-up in how credit scores are calculated as there is now the ability to use many more data sources and more nuanced algorithms to take advantage of.

If we’ve deployed an end-to-end Big Data value chain; we know what has happened (descriptive), we know why it’s happened (diagnostic), and we know what will happen if things continue along the same lines (predictive). Prescriptive Analytics takes the outputs of the three aforementioned methods and provides guidance as to decisions an organisation should take to reach a desired outcome. The outcome could be anything from becoming the next Apple, increasing number of sales by 10%, reducing costs, or pushing forward the release of an innovative product.

Wrap-up

If you’ve made it this far, congratulations and a big thank you! There’s more to Big Data and Analytics that meets the eye but I hope this post has helped make sense of the jargon so regularly used these days. Most of you working in IT know that the real value in Analytics is actually found in the Data Scientists as opposed to the software solution, but that’s a bit too far removed from the end users for this posting!

Any feedback is greatly appreciated as I’d like to shape the series to my readers’ needs since it’s here to help make sense of the nonsensical buzzwords invading our lives! Was this post too complicated, or perhaps not complicated enough? What other buzzwords would you like to see broken down?

Feel free to connect with me :)