When is Data Right For You

In short always, with limitations. Let me explain, I have worked at a couple of companies that jump at Buzz Words like "Distributed Systems", "Micro Services" and now the latest is "Data Science". Following trends is a great way to spend a lot of money, it's like buying the latest shoes every month, eventually, you are going to have a pile of shoes in your cupboard that serve no use but to take up space. The same with technology, before throwing money at it (Unless you're a huge corporation with the budget), rather see what is the right fit for you. Take an example:

I was working with a client on some Django micro-services, now most people that work with Django know that they are optimised for relational databases. One day I walked into a meeting where we sat down with one of the big guys. While I know I am not the most business savvy I take pride in knowing the ins and outs of my system to the finest details, with this in mind imagine my face when this man stood in front of me with no knowledge of what technology stack we were running and proudly told me we where moving our database system over to Cassandra DB....

.....

.....

My shocked silence must have invoked a explanation as I was told that the reason this had been decided was that the head office in the UK (Bearing in mind we were in South Africa) had started using it for some of their systems and were boasting about the speeds (Yes we all know that NoSQL is faster than relational) they where getting. Needless to say, I had to write up a huge email about the use cases of NoSQL vs Relational Databases and explain why moving a framework that was optimised for Relational was a bad idea etc...

Moving back to data. The above is a pretty good example of using the technology that is best for you. Don't pour millions into AI when a simple ML algorithm could be the perfect start to push your profits through the roof. Also, remember that in order for data analysis to be relevant your data must be reliable. No more should you allow those free text fields that 10000 users have access to change. Rather build Validation into your pipeline making sure that you don't fall into the "Garbage In Garbage Out" bracket. Once you are confident that your data is clean and you have a relevant business use case for that Spiffy AI bot you have been dreaming about then its time to start funding your AI R&D.

NB: A word of warning, in the coming years you will definitely need to be somewhere in the data space, so while I suggest above not to jump in with millions, you should most probably be dipping a toe into the waters