The Wicked Problem

Wicked problems in software engineering projects are situations in which the configuration of technical and non-technical components that form the solution cannot be aligned to satisfy all your stakeholders simultaneously no matter how hard you try to configure the solution or how many iterations you perform.

The situation is often due to reasons beyond your control. Characteristically the definition of done is not clear and failure to achieve done is interpreted as project failure to align the components of the solution correctly.

A wicked problem may just be indicating the Product concept of the project was never bought into fully by all the stakeholders. Also the lack of understanding of the project purpose and outcome by some of those involved, or due to the length of the project realisation the problem has be redefined by some of the stakeholders or that definition was not possible at the start but becomes clearer and now the stakeholder engagement plan changes fundamentally.

Wicked problems can also be about the misalignment of the process of project implementation of the business architecture with the solution architecture. The point being without that alignment no matter how good your business architect hones your business processes, or your solution architect fashions your software product there will be a gap in the enterprise architecture which leads to the failure.

1) Misalignment of the Business Process with the Business Process Model

2) Misalignment of the Business Process Model with the Solution Architecture

3) Misalignment of the Solution Architecture with the System Design

4) Misalignment of the System Design with the Business Process Model

5) Misalignment of the System Design with the Business Process

6) Misalignment of the Business Process with the Solution Architecture

If we then said that our Solution Architect and Business Architect were themselves internal stakeholders of the solution and the above alignment were their requirement we could then define a wicked problem in purely stakeholder relationship terms.

Just like a Rubic cubic where the colours of each facing square is changed as a solution looks to be achieved, so a wicked problem the requirements change with each iteration of the product. If the requirements don't change the interpretation of those requirements by certain stakeholders change.

Source: Stakeholder Analysis | BebasBanjir2015

So those of you who are fascinated with this seemingly impossibly complex situation will be wondering how we can solve it. The point is you cannot, "solving it" is part of the problem. The way forward is re-framing your wicked problem. It is not how can we get Product XYZ to suit Stakeholders ABC, how can the product ever suit those different stakeholders in the first place simultaneously? How can we make the product flexible to meet the varying requirements, perspectives and contextual landscapes where each stakeholder lives. Imagine a space traveller who has to design a lunar space buggy to work on different planets with the following variables.

1.Gravitational Pull

2. Heat Intensity and surface temperature

3. Oxygen availability

4.Terrain

5. Hostility to Life

No professional space traveller would expect the same lunar buggy to work with no alteration on each planet. Only a lunatic would insist of the same lunar buggy being used regardless of its suitability on each planetary landscape and expect the same performance. Then why would one expect the same software product to suit a swathe of stakeholders who have different agendas and expectations and reasons for wanting the product's outputs and sometimes competing and rival business logic. Yet we frame most software engineering projects in these terms and are surprised when these are not aligned for each stakeholder.

The Lunar Buggy can be designed to work in different conditions after re-configuration if the design is made with this feature and variable factors are suitable for each use case. How flexible you make the Lunar Buggy and how smart you are about arranging the re-configuration is not about technical wizardry or functional mastery it is simply dependent entirely on the professionalism, intellectual honesty and willingness of your stakeholders to solve their own problems, engage positively and work with you and with each other to achieve a flexible product.

Now let us talk about the Stakeholder Matrix. I once worked with a very experienced Consulting Practice Lead who would ask for a stakeholder matrix in the Project Charter while it was being drafted and then just before the charter was sent to the stakeholders he'd take out the "real" stakeholder matrix and swap with a more "user friendly" one. He even had a "Human Factor" report for really "difficult" clients.

Well looking at it there is any wonder, it is quite insulting to a large number of stakeholders to be tagged and bagged in this way. However it is their disconnect with their influence, interest and impact on the project which does cause a lot of the social problems which we frame as the wicked problem. Even a simple project status report or project steering committee can make a Project Team juggle all sorts of ethical and political machinations.

Ultimately a stakeholder matrix is a conversation about the project roles and the impact on the project which is implementation focused. It is also about the customers (or users of the system) and delivery and after care focused. What is never discussed is that a stakeholder matrix is never static and not often the same view seen by the implementation team let alone the project sponsor or project steering committee. Stakeholders can change their matrix category or taken on multiple categories at once.

Source: Salience Model to Analyze Project Stakeholders - PM Study Circle

The Stakeholder Venn Diagram gives a more realistic view of the stakeholder categories with Urgency, Power and Legitimacy being the factors this time. I that Practice Lead mentioned earlier would be even less likely to want to share this analysis in the project charter.

The discussion of stakeholder relationships and engagement turns into a discussion about each person's self, task, interpersonal motivation and capability in context of the project delivery not just the pantomime villain categories of the classic Stakeholder Matrix and even scarier Human Factor report.

However the Stakeholder engagement has to be grown up, honest and professional one. It is more about managing behaviours, in other words socialising your project with those who care about it and want it to succeed and convincing those who don't to support it or at least not to hamper it or block it.

You must always remember that the Stakeholders are ultimately who you work for and Stakeholders need to positively engage with the Developers for flexible solutions to be achieved. Clients engage for their benefit not your own, respect that. If the project is to be successful then both stakeholders and developers must both engage positively and remember that all stakeholders need to be consulted and involved in the project as a whole.

Any Product Design is perfectly implemented is only completed when it serves all the stated requirements of ALL your stakeholders and that interpretation is theirs but remember they have a stake in the success of that product.

For a great article on stakeholder management: Stakeholder Management and the Law of Inertia by Olalekan Ayelabola

References

1) Dilemmas in a General Theory of Planning by Horst Rittel and Melvin Webber