Data Science Job Interview

Data Scientist is a relative new position for many organizations, it enables the discovery of Signal in the apparent Noise of their data.

What do I mean by this? Imagine the scenario where you are in a crowded restaurant. There are several groups of people sharing a meal and are deep in conversation. Each conversation is Signal being sent and received, and understood by the participants. But standing back and listening, you perceive all the conversations together as noise.

The same is true for data in an organization, it will consist of: business transactions, website page visits, IoT data from door sensors, thermostats, elevators, etc; security video, IT related events such as login, logout, forgotten password, etc. All of this data can be viewed naively as Noise, but it is actually Signal and it is the job of a Data Scientist to discover that Signal.

At some point a leader in the organization realizes that this data is valuable and could be used productively. In business the insights from the data could provide new revenue streams, in government it could assist in forming new policies. The leader than gets approval to hire a Data Science team.

Brilliant minds are hired, HDFS clusters are established, Spark, Storm, TensorFlow are installed, Jupyter notebooks are setup, super columnar databases like Vertica and SQreamDB are loaded with 100s of TB of data. And then the Data Scientists have their first meeting with the business.... Acronyms are thrown around, business domain jargon abound, and the Data Scientists are requirements with the consistency of diarrhea. The Data Scientists soldier on and attempt to guess and infer what the business want. At some point it is realized that nothing has been delivered and the leader, in a fit of rage, fires the Data Science team.

So what went wrong? Obviously there was an issue with the results that the business wanted or expected coupled with the usual problem of poor communication, poor planning and poor product/project management. Numerous articles, books, blog posts cover Product/Project management and I won't attempt to duplicate their work but I have some advice for the Data Scientist in the interview.

Uncle Pete's advice

Consider asking the following questions:

1. What is the business purpose for Data Science? A solid answer with good direction is what you want. Vague, nebulous, buzz word afflicted answers should be a warning bell.
2. Does the organization already have a Data Science team? - If the answer is no, you have an opportunity to create on from scratch, but the risk that the organization has no clue what insights they want from their data.
3. How big is the team? 0 means you are the first one with the same risk as above
4. What has been its success so far? Success indicates that their is an established infrastructure that you can add your genius to. No success could indicate lack of business or technical leadership.
5. Why did previous Data Scientists leave? Data Science is a smallish community and you may already know why, here is a chance to discover if the organization is honest or using "alternate facts"

Note: the opinions expressed here are entirely my own and based on anecdotal experience -- don't kill the messenger (smile)