You’re missing something, and it may kill your project.
Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones. – Donald Rumsfeld
A few months ago, we began to replace software some of our staff use on a daily basis. It’s the first step in a process that will completely overhaul our system, but we wanted (and needed) to start small. We picked a small-ish piece of functionality that 3 or 4 people may use during a given workday, and set about re-creating the functionality to fit into our new browser-based architecture. Interviews were done with the users to collect feedback on…usability (of course). “Hey user, you’ll notice the steps in the process are the same, but the flow has improved, right?” We worked and tweaked and followed our style guide.
The end result? Nailed it. Not only was the new product (initially) met with adoration from the users, but it was a tight, textbook example of how to deliver using Scrum. From design to integration, this shiny new piece of software replicated existing functionality, improved the user experience, and moved us one step further away from an aging legacy system (which, all things considered, has performed more than admirably). Once we moved the product into production, it was smooth sailing.
Except that it wasn’t.
We missed something. Well, actually we missed a few somethings.
In our diligence in examining the product we were replacing, we forgot to ask some questions. The backlog was there, with plenty of well-defined stories fitting into a handful of epics. We built what we were supposed to build. But we failed anyway. We recognized our mistakes and fixed them quickly and with relatively little disruption, but still, we didn’t hit the mark.
Why? This is why:
This thing right here? It’s nasty. It’s an absolute mess. Not only that, it’s dangerous. Seriously. People die on a regular basis in poorer cities across the world where you find things like this. When I say “things”, I mean “solutions”. Yes, it’s dangerous, but it was done because there was a need that was not being met any other way. In the case of this mass of wires, people needed electricity and they weren’t able to get it by calling up the electric company. And the designers of the original system certainly didn’t plan in a way that accommodated the needs of the people in the neighborhood. So the people took matters into their own hands and created this monstrosity.
But if it’s a choice between creating this beast or going without electricity, you create the beast.
That’s what we missed. We are replacing functionality that has existed for over a decade. Let’s assume the people designing the original solution got everything right 11 years ago. No, really. Not a single misstep in design or creation. Even in that scenario, changes to business, or changes to process flow have the capacity to destroy the viability of the solution. In our case, we had a well-designed product that adapted well to the changes we’ve faced over the years, but there were still problems from time to time that simply couldn’t be addressed by the product. So our employees did what most people tend to do. They created hacks. Workarounds. Solutions.
We learned something valuable. When replacing an existing system or piece of functionality, it’s not sufficient to replicate what was already there and add in the new features. You need to account for the ways people are working around a solution that no longer meets all their needs. The workarounds won’t always be obvious. They’ll rarely be documented. But trust me. There’s a giant ball of something hiding from you. It may not kill dozens of people each year, but ignore it and I can promise it will make a few lives very miserable.
Dana, thanks for sharing!
Excellent post Dana Pace on the diligence needed, and being able to learn from the past, and present on complex solutions that you are working on.
Nice post, Dana. You've hit upon one of the ultimate truths of change management. There will always be a dip in productivity when a change is implemented. The depth of that dip depends on communication with the affected targets and their engagement in helping you. And remember, affected targets ≠ intended targets. Your post also highlights the need for constant upkeep of the Value Stream Map. People put a lot of work into documenting their processes, only to have VSMs become shelfware. Workarounds are generally undocumented, value-added activities. They are things that people do to "expedite" downstream value, ordinarily on a one-time or temporary basis. Once the workaround becomes standard operating procedure, however, it needs to become a "known" activity.
Rumsfield didn't hit the mark, either. And information was fabricated and ignored under his watch (granted, it wasn't only him, but he was a major part of it.) My guess is that people didn't die using your new software, but they did die when NASA "normalized deviance" as has been amply discussed and promoted by astronaut Michael Mullane. You didn't miss the mark because the mark is usually deceivingly elusive, even at the supposed end of the project. You made design choices and had omissions that you ultimately had to remedy. Modern product development is loaded with such examples from automobiles to airplanes to space shuttles. The menagerie of wires in the photo is the result of cheap, fast, and simple. It is also an example of what happens when you view work in the context of the Simple domain of the Cynefin framework. Chaos results from complacency. Rumsfield was correct, ironically. But, depending upon the context and domain, the unknown unknowns can have far ranging impacts when discovered, and sometimes discovery of them happens in the final testing of products, which is always with the user. That's why limiting the "blast radius" in Spotify speak is a wise strategy. You're usually not as done as you believe your are.