Disconnected: What happens when the data doesn't flow
As I have shadowed people in their jobs to create data strategies and assess what information people collect and use, I have noticed two things which are seemingly endemic in the corporate world:
1. Systems are under-utilised
2. Everyone has some aspect of their job which is completely off the system.
More often than not, these things are allowed to happen with little more than a grumble on the part of the system user on the basis that they are still able to complete their tasks without issue and that it is more effort than it is worth to change the system or process. The problems occur because this kind of attitude inhibits the flow of data through an organisation, promotes poor working practices and throws everyone into a blind panic when the inevitable request for more data and insight comes in from a senior manager and/or client.
How often have you been trained on a system and at some point been told, “We don’t fill that part in, it doesn’t do anything”? Undoubtedly there will be circumstances where there is genuinely no reason to use part of a system, especially if it is an off-the-shelf solution that doesn’t exactly match the needs of the firm. I would argue, however, that there will be modules to systems that remain unused for no reason other than that they have never been used. It may be that these fields are ancillary to the core function of the system or that they don’t drive processes so are forgotten about completely or worse still are updated inconsistently. These are exactly the sort of fields that whilst they don’t help with the day-to-day workload and perhaps add no value when taken individually, but help to compliment the core data and give insight when taken in aggregate or help to show hidden trends over time.
By contrast, the opposite can be true where there is nowhere to record the data that we actually need and so we create a trusty Excel spreadsheet to make a note of all of things we need to manage that our systems cannot handle. Not only does this induce all of the well-documented risks of storing and managing data in a spreadsheet but it also again inhibits the flow of data. Given that this information is held outside of the core system, it does not often make its way into the hands of the analysts or BI teams and knowledge of this data's existence usually remains with the user(s) of the spreadsheet. This puts the data outside the scope of any reporting or worse still makes it such that, the end-user is tasked with producing the MI, thus stealing away their time from doing their actual role.
This is not necessarily meant as a slight on the end-users’ operation of a system but rather a call for collaboration! As data analysts and BI specialists we should be engaging with the wider business in hunting down all of the corporate data that there is out there and working towards understanding it, cleansing it, validating it and then using it. There is also a call for end-users to work alongside system developers and challenge them to make the useless fields do something useful (drive a process, automate a manual input, calculate a metric) and to add the fields that are missing. If they don't know how the system needs to be used, they will never be able to create a solution. My ethos in all of this is that if we create systems that make people's jobs easier, automate the laborious, capture the data that is required, then the data will flow to the analysts and we can all stop panicking when that daunting ad-hoc data request lands on our desks!