Complexity is the Enemy
An existing dynamic exists within most technology focused workforces today - a linkage between the notions of "enterprise grade", "advanced capabilities" and "complexity". It goes beyond IT, of course: even when one enters a modern aircraft and looks towards the pilots a dizzying array of controls are presented. Or if one casually looks at a creative professional using an Adobe product such as Photoshop - one is equally overwhelmed at the tool choices. Surely, in order to accomplish great things one must implement solutions that require mastery? Sadly, nothing could be further from the truth. Worse, this perception creates behaviors that drive negative outcomes. Complexity itself is the enemy.
As organizations began to move assets to the cloud, many technology shops were often blockers, citing the lack of granular controls that they claimed (often erroneously) to be using on-premises. The decrease in downtime, consistent security and built-in industry standards gained through a migration of services to the cloud would be held up because of a single piece of software that performed a task deemed critical, often by monitoring an underlying service that in the cloud, was irrelevant. "But we have to be able to get disk I/O queue length on the Exchange server - that's how we know when email is going to be slow," is a great example of the type of objection.
By focusing on a subcomponent of the overall solution, these folks were missing the larger benefits - they didn't understand the trade-off involved. Another easy place folks are still misled is in the area of security - right now the average enterprise has over sixty different security solutions in place. That increase in complexity is not only costly from a resource perspective, but impossible to wrap ones arms around: no one can master so many different toolsets covering so many orthogonal areas.
Of course, this increase in complexity isn't inevitable - one need only look at the automotive industry - cars used to be complex pieces of machinery requiring years of training to operate, mechanical knowledge for the inevitable breakdowns and plenty of physical controls and gauges to interact with on even a short trip. Todays vehicles, by contrast, only have two primary controls: a steering wheel, and a lever to move between "drive","reverse" and "park". Keys in the ignition? Gone. Lights? Fully automated. Windshield wipers? Auto as well. Even door locks are increasingly vanishing - if one walks away from the car most simply lock automatically. Even without the coming autonomous revolution, today's vehicles are far simpler, and safer, than those twenty years ago.
Inside organizations, this simplification began at the most commoditized level - in the rooms dedicated to backing up machines. As tapes shifted to disks, and disks shifted to cloud-based options, the folks responsible for moving those tapes trained into new positions. Next came modern applications such as Exchange - in the past, deploying these services required fancy storage arrays installed at great cost - but several years ago, Microsoft said "just use some cheap disks, a lot of them, and assume some will fail". That idea - that commodity hardware designed to fail was a better solution than complex, expensive storage, continued the trend: instead of focusing on "backup" or "failover" or "recovery" or "high availability" - all of which were designed around a concept of operating success - by focusing on resilient solutions designed to fail gracefully - one could have services that never truly went down.
Although many organizations have made progress, there are still large gaps. Having over 600 people sign off on a desktop upgrade process, for example. Even groups that are "all digital" often find that critical business information is stored within a huge group of Excel documents scattered between accounting, an executive team and various consultants, instead of being in a single system of record. If a core part of any critical business process is a human moving data to and from an Excel document, that process is both too complex, and too brittle. Often, these complex systems aren't even discovered until someone goes on vacation, and they stop working as folks expect. That is the hidden danger of digital complexity - if 99% of the folks working in a business assume the system of record contains all the relevant information, but 1% of the people are performing a necessary task outside of that system, it isn't immediately obvious to the rest what is going on. Complexity can be difficult to rule out.
To be able to discover these hidden pockets of complexity, I often ask customers a few simple questions:
- How much planned downtime (with services being unavailable as a result) does your organization experience?
- Are there any systems that behave different when certain folks go on vacation? Or certain folks that "know" how things work better than others, and are therefore irreplaceable?
- What processes require "sign-off" by a human? Are there any that require more than two people to sign off on?
By honing in on these, we can root out where complexity lives. But that's only half the battle: swapping out a complex system for a simpler, more robust one is often both a technical and political challenge at many organizations. The concept of "technical debt" is broadly understood within many software development folks, but I'm always surprised by how many IT shops instantly jump to "let's go purchase solution X which solves this" rather than trying to explore how existing systems can deal with an issue. And beyond the IT team - on the business side, many systems are often implemented with little thought as to the amount of debt implementing a new solution entails, versus using an existing, simpler solution already deployed and understood.
If your customers, or worse, your staff, are demanding complex solutions to their problems, it may be time to challenge them on their underlying assumptions. Simpler isn't always better, but over the long-run, often the most elegant solutions are ones that don't try to cover every single scenario, but merely cover most outcomes in as efficient a manner as possible, as securely as possible. That's a win for everyone.