Data trends in 2017

Let's play Nostradamus. And no, a costume is not needed.

What are the main drivers and trends for data going to be in 2017 ? I think there will be three main lines of thought: natural language, real-time and data protection.

Natural language

For decades we have asked data to tell us a story. Once upon a time a tabular report was enough, then understanding what happened became of the up most importance, subsequently we asked data to be elastic across segmentations, then we asked data to forecast predictions and now we ask data to tell us what to do next. All in a variety of formats and outputs ranging from lists of records, to cubes, to KPIs, to basic Yes or No. Also in this context I would like to bring in the fact that we are moving away from coding solutions around data, we now configure models or fine tune algorithms. In conjunction with the rampant investment in machine learning, deep learning, AI and robotics, we might find that natural language processing techniques will be at the centre of inputs and outputs management and interpretation. Communication with the fabrics of customer intelligence for example will inevitably happen in plain English, or whatever your language flavour is, where a chat bot will be able to tell us what to do for that customer right now or where we will find impractical to call our robot butler clicking a mouse and we will just shout at it to come pronto with a nice cold lemonade. I call this, data coming at full circle. We will move into an era where executives will receive mails or texts saying, 'Your company is doing fine today, do you care to know why ? Y/N' or in the worst scenario possible 'You are bankrupt now, you should have cared when I asked you to first', bringing the realm of possibilities of data crunching outputs to an all new level. It's evolution as well, whereas once upon a time reading, writing and speaking were the cornerstone of communication and progress, similarly data has become the new de facto way to communicate between events and ourselves.

Real-time

There are two separate data flows in every day's life and events. One that is generated by actual data events and the other that in parallel contributes to enhance the value of events with real-time predictive and prescriptive outputs. One cannot live without the other anymore and both have almost the same value. This is not IoT, let's not confuse apples with oranges here and nor is machine learning. This is the future dichotomy of data for any IoT event you might be able to think of. A grid of possibilities that will corroborate new events to a level we could have not imagined a decade ago, where life is laid out in front of us and where only one path will be an actual occurrence, still all other paths will help new events to be better evaluated and supported from beginning to end (that is machine learning and deep learning). But the data currents are flowing together, in parallel, across this grid of possibilities, one immutable, one ever changing until it happens. There are major players in today's real-time solutions, Kafka and Spark Streaming for example, but there are already 'competitors' that promise experiences and better products like Flink or Apex or more recently Acca. Independently of the solution we will inevitably find two technologies supporting those data parallel flows one that supports the immutable aspect and the other the ever changing until it occurs. The Kappa architecture will be perfect to support this new data world although every immutable event will need a counter ever changing partner, pushing the power of parallelism of these technologies to the extreme. It is probably something that is already happening to a certain degree but I foresee companies adopting this concept more and more in the next year.

Data protection

GDPR is going to be massive in 2017. Companies will need to align their data usage, governance and maintenance to this new regulation. I still believe that one day we will be able to monetize somehow on the data we generate as consumers or simply as people living their lives. Maybe this new regulation is the bridge that will make data self monetization or exploitation possible. In the meantime there are different and challenging new rules and options that everyone will need to comply to. The right to be deleted is certainly one of these big challenges. For companies that are fully digital (i.e. where data from paper has been fully scanned and no warehouses full of folders exist) the task is still a challenging one to roll out, but imagine those who will have to go through all that paper to make sure records for an ex customer are fully dealt with. Portability of collected data is also going to be an interesting one and if nothing else it will help to set people's minds on the fact that they carry with them a 'product' that eventually they might all be able to monetize on. There will be different implementations and solutions but to me the only possible and secure scenario for a successful roll out of the GDPR new regulations, is to have an holistic approach with the understanding that this cannot be dealt with an 'on top of everything' type of solution. GDPR is asking companies to change how they deal with their customers' data. It's about full transparency, it's about recognizing that data has evolved since the 1988 act and let's face it is also about stopping the big names from abusing people's generated information. It's also a challenge that will make companies understand the need to stop data proliferation within organizations where my name and data are yes stored in a centralized location to start with but also where my name and data can hand up in excel spreadsheets, PowerPoint presentations and in random folders on personal PCs that will be very hard to find in any organization.

Well, that was fun, wasn't it ? Whatever 2017 will bring to the table, it is a really interesting time to be involved in data within any organization.

Wishing you all a great new year !