Past-Present-Future... Software Engineering Management

"From whence we came not seeking fortune or fame. The thrill of discovery become our pursuit to push those boundaries beyond what was due. We had to conclude at some point the search and conclude with real work for which payment deserved." - Jerry Durant

Many have passed on that were those early adopters of technologies of the 1960s and 1970s. But for those that remain it was a time of intrigue that was driven my interest and pursuit of discoveries. Experimentation pushed the boundaries of machines and cultivated unabashed creativity and innovation thinking relative to the art of system sciences. From this rudimentary beginnings adoption and the commanding demand for productive value results came forth. This resulted in a new birth, one involving the formalization of process. Undoubtedly the reason for such lied in consistency, expediency and advancement in thinking. The birth of formalization, later known as 'methodologies', became the backbone of movements up to this point in time.

Where did methodologies come from? Many of those who engaged during these formative years had scientific and engineering backgrounds. It was from these disciplines that practice embrace methods, to which groups of methods became methodologies. These early roadmaps were later used to guide others in what is referred to as 'best practices'. Many focus on the 'best' as ideal when in fact it comes from the origins of common or routine conduct of practioners. Adoption of such directives took on a commercial bent and became a shelf commodity that any company could buy into. Commercial methodologies were sequential, leading to the analogy of a cascading 'waterfall', and were accompanied with exhaustive documentary artifacts to reflect the journey, serve as a reference tool and to justify the process being followed. It was at this time that we started to see issues, not so much in the processes as it was in the adoption of these processes. The concept of adaption to applicable use was overlooked. Largely out of fear of failing or being remiss when not following the methodologies to the letter of the law. This was further hindered by decision makers forcing compliance especially when solutions failed when adoption wasn't followed precisely. To overcome this occurrence many variations took place to attend to the issue of adaption, adoption and adding other superlatives that served to justify a shift such as more expedient and cost effective. As we railed through top-down, bottom-up, modularity, unified models, spiracle/conical, iterative, rapid-application-development (RAD) and others we seemed to be unable to reach a suitable solution.

February 11-13, 2001 thought leaders in systems engineering met at Snowbird, Utah USA to ponder the question of how could present conditions be resolved. Known today as the birth of agility and the formation of the Agile Manifesto a new start emerged. On the outside looking in many had reservations whether this was yet another 'here we go again' moment or whether we were actually onto a more durable answer. Admittedly, I too had my doubts and after more than two years I was able to endorse, utilize, adapt and apply the major paradigm shift that the Agile movement framed. First and foremost it was a new invention but the culmination of practices that had been used in earlier models to overcome issues. Secondly it legitimately became an exercise that applied lean engineering to existing frameworks that were founded on comprehensiveness and not value as the foremost purpose. As interest grew so were struggles in the adoption and transitioning. While there was much dissension about current practices, giving up those elements of security were hard coming. Trust needed to be forged and this was at a personal level, not the success story of some other institution. Slowly organizations around the world started to replace practices with those endearing agility. At the same time companies retained past practices in those cases where the applicability of agility was less than desirable, as is the case for epics and sagas. Attempts have been made to apply agility to these but often struggling with contextual loss, those more aptly treated as a legacy engineering solution becomes more advantageous. Always remember, we use methods to guide our work and not to take a level of purpose great than that of the mission we are attempting to achieve.

In present day agility we see a quite similar parallel to pre-agility methods in terms of variations. Similarly they have appeared for the very same reason, that of issues. This has resulted in questioning whether agility is a responsible methodological approach or has too much leniency been exercised. Are we placing too much trust in that element of 'people over process' and less focus on the pragmatism of software engineering? Decision makers are concerned because of the growing sentiment over soft science elements of collaboration, teamwork, passive leadership, augmented coaching and a renewed concern over lagging sprint goals. If this is a case of history repeating itself have we reached a point where agility must also be reshaped as was done back in 2001?

What challenges we facing moving forward will undoubtedly influence any methodologies that exist and are used in any amount. The advent of a new industrial revolution 4.0 (IR4.0), characterized by physical, data and technologies. While herald by such technological disruptions such as artificial intelligence (AI) and robotics, its less about the means than it is about the inter-cohesion between them. These are not interfaces but event permissive gateways that afford cross sharing in varying ways and purposes. To achieve this scale of cohesion creates a pressing need for trust based processes and disciplines. It will not be simply a matter of cooperation but one in which the way we do work becomes consistently applied with maximum flexibility to endure the ongoing dynamics of disruption. To achieve these conditions we need a different framework, discussions and formation of solutions that addresses the much wider demands of IR4.0. We aren't there yet and to date discussions have focused only on specific technologies and not the overshadowing future demand for cohesion. A look into the future suggests that it will involve the renewed importance of design, the creation of docking connectors but most importantly a re-emphasis on strong internal software engineering.

In conclusion, what we know is what we have. What we might have leaves room for speculation. But as we all know change has a responding effect. Normality will not persist in a world of disruption, regardless of the speed to which it occurs. The time to get our engineering processes in good working order and to contemplate the impact of change is now. It is no longer a case of whether it 'might' occur as much as it is a matter of 'when'.