Lessons from Cloud

Two aspects of Information Technology is changing rapidly,

1. Network Speeds are increasing due to fiber optics, 5G, etc.

2. Processing Power and Storage cost are dropping

What does this mean for an organization - "Let's move to Cloud!"

And of course Gartner Said it here.

Well it all makes sense, we pay based on usage, performance is better and "world peace". Well just like any other change in human history this comes at cost and cost is substantially higher if these initiatives aren't well planned and carefully executed. Well I (oops We were!) was closely involved (read neck deep) in one such initiative where we were (still are) replacing an On Premise Document Management / Workflow Application with an improved Cloud Version. For someone who has played both the roles a. End User and b. Product Management the opportunity gave me a unique vantage point to witness its success and failures and therefore an article consolidating the learnings was in order.

So here are the key take ways from the two year marathon which includes large scale deployments with unique perspectives from players with skin in the game.

1. Quality of Data

Well I am sure this is not a surprise for anyone who has been part of a data migration project, if the source data is not clean (I can't use 's' word for known reasons) data in target system will be same and surprising bit is solution architecture may take a hit. Legacy Applications which have been around for decades have lot of skeletons, in normal day to day work they stay buried but they do come to haunt you during migration.

Thus, if you are dealing with legacy application makes sure the Data Model and Reference Values are aligned to industry standards.

For us in Oil and Gas while we rely on in-house standards industry standard in form of CFIHOS.

2. Automation and Feature Enhancements

Well, upgrading a product improves its usability, We live in the age where every investment demands quick returns. If a product claims to automate a process expect a call from the end user on day of go live - "Hi, good job with migration, Can we automate email triggers as it saves us XX hours // XX $$"

Naturally, you would like to enable the automation but here is the catch, most automations impacts someone's job and trust me human beings are around for 200000 years and they have mastered the Art of of Survival, yes pat on the back.

Rajat Singh who operates as Service Delivery lead and focuses on Resourcing says “Automation applied to inefficient processes makes that process more inefficient. So before any automation effort, the process should be strengthened.”

It is important for Product Team to get agreements prior to the migration, especially if these automation are part of the Business Case and ROI as it gives opportunity for various stakeholders to push back if the features are not ready and more importantly find themselves new jobs (surprise, surprise, automation = reduction of headcount). You certainly don't want to get in a discussion where you have convince someone enable automation and in the process find themselves a new job. Again there are number of proof points suggesting this doesn't reduce number of jobs, on the contrary it will create more.

3. Migration and Onboarding

Let's get into action, Just like every other project, success of an initiative depends on the level of details of Project Plan because honestly the competency of players is undoubtedly at par with the game, yes we have some super strong professionals in the team.

Nick Wesdorp who leads a pack of Workflow Consultant has Designed and Implemented End to End Migrations has this to say - “A plan always changes as per client requirements and findings along the way, which can only be properly outlined after a detailed source assessment. Without proper source assessment, the risk of delay, or worse, failures, increases”

Chart the clear route of Data Movement from Source->Target: On Premise Servers -> Intermediate Storage: FTP, Extraction Servers, Local Storage -> Target Storage -> Cloud Servers (Azure, AWS)

Apologies for stating the obvious but preparation is everything - Make sure the tools to enable the journey are configured and tested prior to the migration. Virtual Machines hosted in US and Europe came handy for us in India where internet speeds/connectivity are mediocre, Before I get attacked internet in India has way better than before and improving as we speak. Uff dodged a bullet there.

Vishwas and Akansha are in driving seat for most of the data extractions. Here is what they have to say -

Vishwas Singh who is in middle of lot of these migrations at the moment has this to share - Data analysis before initiating any migration activity helps to identify possible risks which if not handled properly can push project timelines. In case of large migrations always make use of virtual machines (while migrating from on-premise infrastructure) and make sure your tools/scripts are fully tested and you've enough compute, network and storage resources. Once you've made sure that these prerequisites are met, this will help you to make sure that your migration process is smooth, and my personal suggestion is to always keep few additional days as buffer in project timeline to tackle any issues.

Akansha Singh who has participated in lot of deployment projects has this approach to share - We use the cycle of Extract, Transform and Load (ETL) for gathering data from various available sources, changing/modifying it based on agreed business needs and then gathering the meaningful data into an objective dataset. Finally, we run a pilot/test project to test the migration setup. It's wise to spend some time in analyzing the requirements, agreeing on the method first then dealing with the issues (arousing from incorrect planning) once the actual migration starts.

If you are dealing with business critical data a single missing file or data point could have major implications and lot of angry users, this made us realize that we need tools to identify mismatch between source and target systems. We ended up spending substantial amount of time analyzing complex reports to hunt the missing data. Lesson - Plan for the worst case scenarios and build tools to identify missing pieces in Target System.

Akanksh B S who was involved in building a tool to extract legacy data has this to say - The concept of legacy project data migrations involving thousands of documents is a challenge. In addition to the sheer volume, the different formats of file metadata are a challenge. An analysis of different types of file metadata beforehand helped us to sort them into different automation buckets and increased efficiency tremendously.

Jim Stuckey our Technical Guru has mastered to art of data manipulation has this to say - Utilizing a Type I AI approach with RAD we can substantially reduce both cost and time while improving quality associated with migrations. This can include tasks like reading all the final information, performing various quality checks/repairs, producing error lists for manual intervention (for specific items), generating new metadata, organizing/renaming files and finally, dynamically generating the final import data in vendor-sized chunks.

4. Value Realization

In the world of Cash Crunch and Cost Saving organizations are getting super serious about - Funds Allocated v/s Value Realized. Now if you are neck deep in resolving punch points post migrations the effort to calculate value might never see day light. So add Value Realization as a task or project backlog and make sure someone is on it. This will lead to the most exciting task of the initiative - Decommissioning the Old System.

Nick Wesdorp who also drives value realization for the team and has first hand experience with Challenges of getting to a $$ value says -

“A migration is best to be done when it generates benefits and added value for the client. A proper value proposition is to be understood by client, including all transformation and change management steps to accomplish success. Any migration should also be followed up by a maturation execution action, to guarantee the benefits and value is met”

5. Maturation

Oil and Gas industry is going through a remarkable transition of moving to Digital Twin Space. Simply put, Digital Twin is a Digital Model of your Physical Asset. Advantages include improved data insights to enable better decisions, improved operation efficiency, compliance to heath and safety etc. Target tool we are dealing with is a critical piece in the Strategic Enterprise Architecture of Digital Twin and therefore expectation that is should compliment organization's vision for future is at center of affairs. Well it sounds good in documents and slides but if you are dealing with a SAAS application with its own product development cycles and customer with its independent set of requirements reality is both will rarely be in sync, not at least when the deployment journey has just begun. For the delivery team the key is to capture this reality as a risk and manage expectations.

Nick Wesdorp who works with lot of SaaS vendors has this say (yes he is one man army)-

“With the current challenges, and ahead, it is a must to understand consequence of migration from ‘on-premise’ to ‘cloud’. Relying on SAAS vendors requires a trustworthy companionship, in where product management, continues improvement and support are jointly shared, understood and executed”

I second Nick's recommendation relationship with the Vendor is key to success, In other words Long Term Engagement takes precedence over Short Term Financial Savings.

Hope you get takes ways from the note, if not I am sure you have few laughs. Until next time.