A tale of two cloud experiences

Over the last 12 months, I have been working heavily in Azure. The organization I work for has made the decision to go all in with the cloud and take a PaaS first approach for a few reasons:

1. Drive innovation
2. Speed to market
3. Reduced IT overhead in engineering / systems maintenance

As I look back at my experience with Azure and some GCP (Google Cloud Platform) at my previous company, I started to reflect on organizational approaches. This post isn't about Azure vs. GCP vs. AWS, it is about the strategy behind leveraging cloud and how organizations choose to leverage it (mind you, my Azure experience is quite positive...so there will be a bias towards it)

Cloud - Take #1

My first cloud experience was after we had made a large investment in a new data center. The intent was to dabble / get our feet wet with simple workloads. The focus was on non-production workloads while keeping critical workloads on-premise. The infrastructure team was leading the efforts with some level of executive support. There was still a constant concern / rhetoric about security, data sovereignty, and other "reasons" to not go cloud.

As an IT professional, I was excited to use cloud as an enabler and drive innovation. The applications I use are typically multi-server and resource intensive (Hadoop, DataStage, etc). Unfortunately, the tenancy was tightly guarded and everything had to go through a central team; even for POC workloads. There was extreme reluctance to grant any resource pools to other teams (control, cost, and many other "reasons"). The outcome...very little value, missed opportunities, and mostly disenfranchisement, at least from my perspective.

Cloud - Take #2

In my second experience, the move to the cloud was a top-down mandate. It was all-in, let's make it happen. Further, we have a lean infrastructure team which is willing to delegate/decentralize to subject matter experts. I did not have to jump through hoops to get portion of the subscription carved out and getting elevated rights where required. I was entrusted with the ability to make things happen. Further, the barrier to learning Azure was very low once I understood the subscription and resource group model, access management, and other aspects that make Azure tick. Once understood, use the portal or PowerShell to get building.

As we looked to implement our future data platform using Azure services, we completed a 4 month proof-of-concept to evaluate multiple technologies. Post-POC, the majority of the technologies were selected to proceed, some removed, and some replaced. The entire POC required very little involvement from the infrastructure team. The biggest interaction points were about Active Directory (AAD) and network connectivity; the rest, we were enabled to stand-up and test.

Looking back over the last 8 months, we accomplished a lot with very little friction. We stood up infrastructure that I could not imagine doing at the same speed in an on-premise scenario (try to procure and stand-up a Hadoop environment for the first time in less than 6 months). Over 4 weeks, we built the automation to stand up dev/test/staging/production environments for the following technologies while establishing naming standards + enforcing them through infrastructure as code:

• Resource Groups
• Virtual Networks & Network security groups
• Azure Blob
• Azure Data Lake Store
• Azure Data Factory
• HDInsight (Hortonworks Data Platform as a service)
• Azure Data Lake Analytics
• Azure Key Vault
• Azure Analysis Services & Power BI (still underway)

Now we are well underway to port one of our application from a standalone SQL server running in a multi-tenant scenario over to the new platform. The team is making great progress and we'll soon be looking at doing CI/CD using VSTS.

Parting Thoughts

1. Trust your SME's to do the right thing. Enable, enable, and enable.
2. See #1
3. Really think about your motives behind cloud. Doing non-production system, disaster recovery, or performing backups/archiving in the cloud are missed opportunities. Look for business enablers and speed drivers.
4. Establish cloud standards quickly to prevent chaos and rework (Subscription, resource groups, network topology, connectivity to on-premise, security, naming standards)
5. Look for opportunities to leverage PaaS. It drives speed to market and reduces overhead
6. If cost is a concern, there are many options to delegate billing to groups. Agility and speed to market are worth more than money. Stop making excuses.
7. Stop fighting the cloud on security grounds. If you think you can do PCI / HIPAA / "excuse regulation" better than cloud providers; think again. Most services come with encryption at rest and in transit and the management components surrounding those solutions. Most security challenges are not technology based, they are process related.

What have been your cloud experiences? Additional thoughts? Concerns?