The Data Ecosystem – How To Do Big Data successfully …

Building Data Systems

Big Data/Analytics are the word in boom today (like the stock market was till a couple of weeks back).  Everyone is talking about it and wants to be involved.  Lot of people have even realized this is becoming a buzz word.  So what is this for real and how does a user and how does the company doing it profits …

This is what I have seen and learnt.  It s true that it is all about Data.  But it is with a purpose.  You have a very specific goal in mind – you want your customers to reach a destination in the shortest time avoiding traffic, being aware of the traffic patterns (cars on the ramp, cops on the lookout) with the convenience of taking a small detour for coffee, lunch or dinner.  The goal starts small and increases as your first customers use it and you listen to them (either them specifically telling you or you identifying the need).  You then try to collect data from the different sources they are available to you.  You can get it as a data dump or using APIS.

Next figure out how to present the data to the user.  You need to present the right things at the right time.  Not flood the user (User Interface and the customer interactions) with everything you have.  Show things at different levels (hierarchy).  At each level record what the customer is doing with the data – how is it being used.  Provide insights to customers – what you have learnt so far – how they can use the data presented to their maximum potential.  Incentivize the customer – to take actions.  Finally, to complete the circle – provide a mechanism by which they can be part of the system.  The users of your system need to be able to provide data inputs (new data, correction, curation, …) to fine tune the data.  The last 2 points are crucial and are mostly not taken into account in designing any data systems.  Why should the customer use it and why should they act on the data presented?  And how can they input data – remember they would know a lot about it – since they are the users of the data.  For example, the traffic app that we talked about earlier … the app ‘Waze’ – has basic set of data on the maps and the POI (food, gas, …).  It presents it nicely and the right information at the right time and lets the users add information to the system.  It incentivizes the user to use the system – by giving you points/rewards.  It’s a very small example but shows the whole ecosystem and the importance of data collection, presentation and user feedback. 

I think this is the core of a data system and we at Market Bridge are keeping this design philosophy in mind in developing the next generation of the Sales/Marketing application for our business customers.