Software Infrastructure as a Freeway...

As an architect, I am often confronted with multiple tensions to manage. 9 months ago, I moved from being an Application Architect to an Infrastructure Architect. In that move, I went from being an Architect that consumed infrastructure, to one that now produces infrastructure. A struggle that I had while being an Application Architect was sometimes I could only move as fast as the enterprise infrastructure would allow me, which sometimes frustrated me as a developer and oftentimes frustrated my business partners and product owners. Now that I am in the Infrastructure Architect spot, I am very aware of those challenges and also can see a different view. However, not being one to stay with “status quo” (and perhaps just being a naive new Infrastructure Architect), I have been pondering how to reconcile the two (and why perhaps they seemingly conflict). I’ve also noticed that while some teams are nearly driven mad by having to use the enterprise infrastructure, other teams are just as happy to consume the architecture and help move it along for the enterprise as a whole. In the following analogy I hope to bring out both sides of the argument, and perhaps even more important, propose a solution that, if agreed upon by the two parties, may be an equitable solution. At the very least, will help each side see the value in both options.

I will utilize an analogy that was definitely not my idea originally and has morphed in my head and through others I have talked to over this past 9 months. The analogy is that infrastructure is like a freeway. There are a few things about a freeway. For instance, once a freeway is built, it allows its motorists to go as fast as they want up to the speed limit allowed by the authorities. Now, people aren’t forced to go on the freeway. If they decide to, they can take side streets. It may take longer, but there is nothing stopping them from taking that path. Let’s also say this freeway has something a lot of us in Texas (and especially San Antonio) don’t like, and that is a toll. You have to pay to use the freeway. Again, you don’t have to take the freeway, but if you do, you can go super fast and you can get to other big cities and destinations quickly. But let’s take a step back. I mentioned at the beginning, something about “once the freeway is built.” Let’s dig into that idea a bit deeper. Another thing us Texans hate is construction on the freeway. As cities grow, it becomes necessary to widen the roads and also do general maintenance. It can be painful at times, but the alternative is to go down side streets and through the middle of towns to get around it. Sometimes necessary, but usually only in extreme circumstances when traffic is at a standstill. Now let’s get back to my original idea about Infrastructure. I’m guessing you may already have picked up on a few ideas of how a tolled freeway is much like an enterprise infrastructure. Let me pull out a few comparisons that I find important. Much like a freeway takes a lot of effort up front, so too, can an infrastructure component. For instance, my domain is the Testing domain. We use a cloud vendor to do some of our automated testing. We have built infrastructure that abstracts out the vendor specific “configurations” so our teams don’t have to know the guts of the implementation. It also helps for ease of migration if we decide to move to another vendor for competitive purposes or even if we want to bring another vendor in for environment availability/backup reasons. We also have infrastructure built around this for reporting. As in, if you use our infrastructure, you get a standardized reporting solution as a part of the infrastructure. Our team also provides the training documentation, such as training labs and where possible will point you to proven practices from articles/posts on the internet that align with the open source components we use. The “toll” portion of the infrastructure is primarily some of the overhead of using the infrastructure, which could take the form of importing a few extra dependencies or also having to conform your design to align with our infrastructure. However, by using the freeway (infrastructure), you are prevented from having to build your own components and required integrations.

Now let’s take the alternate approach. Say you don’t want to use the freeway to drive from point A to point B. Let’s say you abhor paying a “toll” and want to orienteer your way to point B. We’ll call this approach “not using enterprise infrastructure.” I’ll even throw a bone out and give you some roads by which to travel on, which would be open source software that you can use (but different open source software than what is provided under the covers by our infrastructure). Perhaps the software the “infrastructure” supports is still meeting the need, but the software you want to use has a few whiz-bang features that you really want to take advantage of. Back to the road analogy. By taking your own road, you can go as fast as you are able to go. However, now you have to go through stop lights, watch for school zones, and take a little more care knowing where to go on the map, because there isn’t as straight as a way to get there. You can even go off-roading where maybe there are no rules… Sounds great, right? Let’s go back to the things that the freeway provided that you no longer have. The freeway has paved roads, no stop signs, or school zones. In my company, we have a lot of regulators that “help” us. Let’s call those regulator expectations the school zones. On the freeway, we’ve already ensured that the school zones are taken care of by by building the “regulatory” needs (such as reporting or audit controls) into the infrastructure. If you choose to build your own components, then you still must abide by the regulations, but now you are on your own to explain why you built the components the way you did and must ensure you are still following regulations and audit requirements, but you must build and maintain your own. Also, back to the vendor “configurations” that we abstracted out for the cloud vendor testing. If you call the cloud vendor directly and we move to another vendor, then you are expected to rewrite all the test code that integrated (including the vendor specific implementation configuration code) with that vendor, while if you used the infrastructure, it would be seamless. Also, you’d be expected to fulfill the reporting requirements with your new components (because those requirements didn’t go away).

I think there are times when, for urgency/deadline sake is is necessary to circumvent infrastructure, but the full impact of not using infrastructure entails some of the items discussed above. A few other things to consider are the implications of performance engineering your components, as well as training, what happens when developers switch teams, etc. Another thing to consider as a large enterprise is, if you think of every developer in the company as driving a car, using a freeway to get to point B can be very useful, versus each car having to find their own way. I also understand, there are times when the infrastructure has so many potholes in it, that new infrastructure (and thus a new freeway) needs to be built.

Now the solution that I am proposing is to have the conversation with the teams that want to (for whatever valid reason) circumvent the enterprise infrastructure to lay out what the differences are and agree upon building out whatever controls (i.e. regulatory/audit) are necessary. Also, have a full understanding that when the underlying components change that impact them (as in my vendor configuration example), they are required to migrate, without any effort spent by the infrastructure team. I feel like sometimes it is hard for each side to see the value in what the other is thinking (and the implications), so I hope that whichever type of team you are on, this might provide a good conversation starter.

Please feel free to add your thoughts and comments to continue the conversation.