The Right and Wrong Way to Approach Integration Patterns & Standards

I can’t say how many times I’ve been asked to create integration patterns and standards as the priority when I join a newish Enterprise Architecture (EA) team, and how many times I have seen an EA team spend the majority of its efforts on developing and documenting patterns/standards which end up on the shelf that nobody references. Subsequently, the EA team is seen as an ivory tower - an isolated entity that fails to prove its value and engage with the rest of the organization effectively. 

In my opinion, developing patterns and standards is the most overrated,and sometimes a total time-wasting activity delivered by Enterprise Architects if we approach it in the wrong way, especially for an organization just starting out its Enterprise Integration Journey. 

Reason #1: Don’t reinvent the wheel

Many technology platforms, such as SAP, Salesforce, AWS, etc., already have their own defined integration patterns to guide common integration scenarios. Each of these platform owners in your organization should already reference these best practices and establish patterns and standards in their team to control quality and help onboard new team members to implement integration faster. The challenge is that these patterns are not described consistently across different platforms and are applied in a different context. Hence, from an enterprise perspective, it’s crucial to position the use of any existing patterns and standards first before introducing any new or self-defined patterns and standards that might cause confusion.

Reason #2: Don’t attempt to capture all integration scenarios upfront

It is not possible to capture all integration scenarios upfront, especially in areas where it’s a new venture and expected to evolve over time. An EA could develop the patterns that are either too loose or too restrictive, which simply informs what most project teams already knew or results in many anti-patterns later. Both scenarios could make integration patterns meaningless. Moreover, your project teams can get very frustrated if they are forced to follow these patterns, and every time they have to explain why it’s not applicable to them; consequently, it increases friction between the EA team and project teams.

Reason #3: Don’t think it is a quick job

Defining patterns and standards could take several months for a newish EA team in any medium-to-large cap company to just gather the information and establish a good overall picture. If an EA or a consultant has to do it quickly, he/she has to take integration patterns and standards done somewhere else plus many guesswork in order to show deliverables. I believe integration patterns can be replicated at the platform level but not at an organizational level because almost no organizations hold identical technology stacks and business models. Hence, your team may struggle to follow these newly introduced patterns and standards, and it will also take more time/effort to rectify them later on.

For these reasons, I would say that developing integration patterns and standards can be a wasted effort for a newish EA team as you can spend a huge amount of time on it if not careful. Here are my suggestions on what we should do while developing enterprise integration patterns and standards that truly serve their purposes. 

Tip #1: Clarify your WHY

Don’t be afraid to clarify with your leadership team - the motives for having patterns and standards in place. For example, is it driven by the leadership team to put control in place? Do your team members need guidance so that they can do their jobs better and faster? Do you have any in-flight projects that urgently need to make critical design decisions, or otherwise it will be too costly to undo down the road? All these are very good reasons for having integration patterns and standards in place. However, don’t do it simply for the sake of doing it or proving your authority. 

Asking these basic questions helps you to prioritize, draw the scope, and choose the starting point that can bring immediate value to your organization, as well as build a trusted relationship with your project teams and avoid developing patterns and standards in isolation.

Tip #2: Make it a by-product, not the end goal

Technology is changing faster than ever. We also need to be practical about it in terms of spending months developing patterns because you soon realize they are out of date or not applicable anymore in a year or two. I’m not saying it's not worth spending time on it. I simply want to point out that patterns and standards can be documented or updated along the way as a by-product wherever a recurring scenario is identified or when it is required by an in-flight project.  

The point here is that patterns/standards should be seen as a living-library to be updated regularly and used for team reference.  

Tip #3: Leverage, Leverage, and Leverage

Not many organizations can afford a big EA team. Often developing, maintaining, and updating patterns and standards can be a full-time job in itself. This shouldn’t be the sole responsibility of your EA team; rather, it’s a joint effort from everyone in your IT department to maintain and govern their own patterns and standards at the right level that’s applicable to them. 

The EA should leverage what’s already out there, focus on connecting the dots, and define where these different sets of patterns are used and how they are related to each other, so that project teams can find what’s applicable and how to adopt them. 

Remember that an EA is an expensive resource, so use their time wisely, think where they can add the most value and have the most impact, and consider the things only they can do or to which only they have access.

Let’s Connect

Whether you are an architect, developer or technology manager, what are your biggest challenges in developing or adopting integration patterns & standards, where do you find integration patterns & standards add the most value in your organization, how easy or realistic is it to apply them? I would love to hear your observations and thoughts in the comments below.