Data Science - When there is less data

Data and Data Scientists! There is a certain appeal in looking at new datasets. A new business problem to analyse and discover the answer through data excites a data enthusiast. But a data enthusiast may walk into a project and find not enough data is available. Since many organisations are still in the early stages of embracing a data-driven culture, a scenario like this may be more common than not.

After experiencing the same in a few projects, I started categorising organisations based on its position in a data journey. The use of the tools depends on the current stage an organisation is in. The following quadrant is based on the data and the related infrastructure in organisations.

In this blog, I will share my experience and learning from participating in projects for Q1 organisations. A Q1 organisation is an organisation that has no data repository for its operational data, and a minimal amount of data may be available in many different pockets. The Q2 organisations are organisations with available data but not having the data technologies. The Q4 and Q3 organisations are prepared with good infrastructure but Q3 has more data than Q4 organisations. The Q1 organisation is in a very early stage of adopting a data culture. The possibilities are unknown, and the organisation requires tangible evidence to understand the value of its data. In a context like this, addressing a business problem using data is a stretched goal, and a project always has a time limit.

The application of qualitative methods is handy in projects for Q1 organisations. At the beginning of a project, a problem-solving framework is decided and applied. A structure that allows creative problem solving is better suited for data projects in the Q1 organisations. Design Principles and Osborne Parnes CPS models are examples of creative problem-solving frameworks. These models focus on clarifying the business requirement and emphasises on understanding the problem to be addressed in detail. The models employ ideation stages to creatively think about a solution.

After a problem-solving framework is decided, the project team requires to live in the client eco-system to develop a feel of the business and the problem it is trying to address. To achieve this the “Ethnography” method is used, where individual members of the project team are broken into groups and placed in the organisation environment to experience the operations and the challenges faced by customers and team members. It is a process of observation and sharing the individual observation by the project team members. It is essential to ensure the project team members do not disrupt anything in the ecosystem (organisation) they are placed. The observations then provide direction on what data and information are required to address the project problem. The observations also assisted in interpreting the data according to the business context and lay out how the data can be used to approach the problem.

In Q1 organisations, it is also important to motivate the management to embed a culture of data. A project to solve a business problem is an opportunity to persuade the management to appreciate the use of data. The project team members also play a role of the consultants. To include the business in the activities related to the solving of project problem is a way to put forward the case for data science. In a very recent project, our team worked with the client and included the team members from the client side in our problem-solving exercises. We wanted the client to understand how we used the observations to request data from their 3rd party service providers and developed a data model that can become an asset to the organisation in future if from now on the data is captured and stored centrally by the organisation. The client accepted the recommendations provided. But I consider the main achievement for my team was to able to inspire the client's teams to use data and consider building a central repository for its data.

References:

Anderson, K. (2018, October 21). Ethnographic Research: A Key to Strategy. Retrieved from Harvard Business Review: https://hbr.org/2009/03/ethnographic-research-a-key-to-strategy

Creative Education Foundation. (2018, October 20). The Creative Problem Solving Process. Retrieved from The CPS Process: http://www.creativeeducationfoundation.org/creative-problem-solving/the-cps-process/

Dykes , B. (2018, October 20). Creating A Data-Driven Culture: Why Leading By Example Is Essential. Retrieved from Forbes: https://www.forbes.com/sites/brentdykes/2017/10/26/creating-a-data-driven-culture-why-leading-by-example-is-essential/#643de19d6737

Singh, V., & Dickson, J. (2002). Ethnographic Approaches. In D. Partington (Ed.), Essential Skills for Management Research (pp. 117 - 134). London: Sage Publications.