A Modest Proposal In Solution Deployment Strategy Part 1: A Managed Refresher

Recently I have been on a couple of large development teams that incorporated a managed ALM DevOps approach. These 2 teams have approached the managed solution ALM pipeline in a similar fashion but with a few key differences. This article is my attempt to develop a reasonable managed solution ALM proposal for development teams of similar sizes. I still see MVPs and other sharp, experienced professionals that I look up to raising valid questions and asking how others approach the managed solution question. I'm all in favor of this openness as we all attempt to discover potential best practices. So please feel free to offer any feedback and fill me in on any issues you have encountered with your managed solution approach. There is no longer any debate these days. Whether unmanaged production solutions fit better for your situation or not, Microsoft has been strongly advocating managed production solutions as the way and unmanaged production solutions as unrecommended or unorthodoxy at best. So managed solutions are now gospel on production. Great, this being said, managed solutions are much more complicated than unmanaged solutions. And there are so many ways you can go wrong, and you can spend just as much time troubleshooting managed solutions as your bugs. So since managed solutions are the way forward, how can we minimize the downsides of managed solutions and enhance their benefits? My two-part series will present a few ideas, but first, let's go through a primer on solution layering and how managed solutions work first.

Managed and Unmanaged Solutions: An Introduction

Some of us, even those who are very experienced, may have been skewed to 100% unmanaged solutions, while others have been taking the managed approach for years. So to have a common understanding, let's do a refresher course on solution layering. I will try to explain this, but please refer to official Microsoft documentation for the most correct and up-to-date information.

Solutions exist in two basic layers in Dataverse. Managed and unmanaged. Think of a managed solution as a carefully curated set of components that are versioned together explicitly as a group contained in a metaphorical container. Think of an unmanaged solution as a group of components and a figurative list or lists that point to those same components. In the first analogy, a component can only exist in 1 container. In the second analogy, a component can exist on many lists. The container in the first example refers to a managed solution, and the lists refer to unmanaged solutions. Think about those analogies and take a look at the image below.

Seeing the above images do the following solution statements make sense?

1. When a managed solution is deleted or removed, its components are also removed.
2. When unmanaged solutions are deleted or removed its components are not removed

Now I will get into the versioning differences between managed and unmanaged solutions. Some of you older grizzled Dynamics veteran's eyes may widen at my following analogy but bear with me here. I promise it will get better.

Let's say we have two files named Insect.html. One of these files exists in a managed solution environment. The other file exists in an unmanaged solution environment. The versioning of both text files happens in this order:

1. Insect.html - 1st version contents is "larva"
2. Insect.html - 2nd version contents is "caterpillar"
3. Insect.html - 3rd version contents is "chrysalis"
4. Insect.html - 4th version contents is "butterfly"

How do the versioning capabilities compare between the managed and unmanaged deployment models? Well, let's take a look below.

As you can see, managed solutions can give a single file a history (or a layer, as you will see later). Unmanaged solutions do not. In our image below, the managed solution butterfly can remember when he was a caterpillar. The unmanaged solution butterfly has no memory or history. It's almost as if he appeared into existence as a full-fledged adult. Poor butterfly (Or carefree butterfly who isn't burdened by his past 😊). This was a super-simplified introduction of a single solution with a single file that changes over time. Let's amp it up a notch. How do multiple managed and unmanaged solutions interact in a single environment?

Solution Layering: A Primer

There is simple logical precedence between managed and unmanaged solution files, plus a few nuances. Let us discuss. Take two separate and different solution files. Managed Solution A (Insect.html = larva) and unmanaged Solution B (Insect.html = caterpillar).

1. An admin imports managed solution A to our target environment 1st. (Insect.html = larvae)
2. The admin imports unmanaged solution B to our target environment 2nd. (Insect.html = caterpillar)

What will our insect be?

You are correct if you answered a caterpillar. But why? The simple overriding reason is that the unmanaged solution import came after the managed import. Now, some of you grizzled Dynamics veterans may say hold up; the unmanaged solution is unmanaged and would take precedence over a managed solution no matter the sequence. Not in this case. The order or sequence matters! Let us explore that statement in a little bit. Before we do that, look at the image below, which shows how a combination of different solutions is evaluated. Everything being equal, the dynamics import logic will evaluate as follows:

Ok, let's turn back to the previous question about the unmanaged solution taking precedence over a managed solution despite the sequence

1. The admin imports unmanaged solution B to our target environment 1st. (Insect.html= caterpillar)
2. An admin imports managed solution A to our target environment 2nd. (Insect.html= larvae)

What will our insect be? Is it that the unmanaged Insect.html already exists in the target environment, so the managed solution will not be able to overwrite the caterpillar? Or will the order of imports take precedence and the managed Insect.html overwrite the unmanaged Insect.html into a larva?

You are correct if you guessed a larva. First, when we import our managed solution, we are given the following warning.

This tells us a managed solution will convert an unmanaged component into a managed component. I went with the default managed import (I wasn't given any special options). The insect became managed and eventually became a larva. To the brand new Dynamics consultant, this is probably the expected behavior and seems intuitive.

So let's denote the four basic import behaviors for components. Solution imports will either:

• Add a component
• Replace a component
• Merge a component
• Remove a component.

When it comes to solution imports, all components can be added to an environment when they are newly placed in a solution.

When it comes to solution imports, every component is replaced by a newer version of itself except for FormXml files, RibbonXml files, and SiteMapXml files.

The only files that can be merged during a solution import are the files listed above. FormXml files, RibbonXml files and SiteMapXml files.

Any solution component can be removed from an environment.

There is a qualifier to the above statement. Any component can be removed, but only the managed solution has this capability. Not the unmanaged solution. We are not accounting for someone manually removing components in the target environment (In normal ALM practice, this is frowned upon). So we can say that unmanaged solutions are purely additive. See the comparison below.

So you say to yourself, what is so great about unmanaged solutions and why was there this contentious debate for over a decade? Managed solutions have everything unmanaged solutions have plus one extra feature, the ability to remove components. Well, here's the catch. Since unmanaged solutions are purely additive and cannot remove components in the target environment, it allows for smaller solutions in the future. For instance, in my first release to production, I have 250 components, but in my second release, I only have 100 components. I can remove the previous 250 components from my solution and add the 2nd 100 components (The previous 250 components already exist in my target environment). This minimizes any issues because the component footprint is much smaller. Anything wrong in the deployment will be from the 100 new components. Remnants of the 250 previous components have no chance of causing deployment issues or solution bloat.

On the other hand, a managed (upgrade) solution will HAVE to include the 250 original components plus the additional 100 components, leading to solution bloat that only gets worse on each release. I need to add there is a QUALIFIER to the previous statement. If you choose a managed "UPDATE" instead of a managed upgrade, you can only include the delta components in your solution (More on this below). This isn't the recommended practice as environments can become out of sync. Back to the managed upgrade discussion, if you decide to remove the original 250 components from the solution because it already exists in the target, then the import process will attempt to delete these components and either be successful or fail because of dependencies. Either way, I highly doubt the result would be what you intended.

You could create a second managed solution that would contain your new 100 components leading to 2 managed solutions on the target. But this would lead to other issues that I will get into later, such as inconsistent ALM Dev Ops management. Not to mention the complexity of accurately recreating an environment via solution installations. So hence, this is why we had the decades-old debate. The argument for unmanaged solutions has been on the side of simplicity. The main counter-argument from the managed solution side is the DevOps ability to rollback releases without doing an environment restoration and simple solution-based recreation of environments. These days the debate is over, and managed solutions have become the favored and best practice method. One of the remaining questions is what are the standard patterns of streamlining and simplifying the managed pipeline for multiple streams of development? I hope to offer an idea or two in part two of the series.

Ok, so as you can see above, both managed and unmanaged solutions have their pluses and minuses. Now that we've described some of the properties and behaviors of managed and unmanaged solutions let's do a couple of mixed solution environment exercises to drive home this information to see how managed, and unmanaged solutions play together.

Mixed Solution Exercises

Lets say we have 4 solutions. 2 managed solutions and 2 unmanaged solutions. The solutions are deployed in the following order:

1. Managed 1
2. Managed 2
3. Unmanaged 1
4. Unmanaged 2

Each of the solutions contain the following where the right column represents the value inside the file:

What should be in the resulting environment table above? Give yourself a little bit of time. *Spoiler Alert! The results are displayed below.

• Doc1 - v2
• Doc2 - v1
• Doc3 - v1
• Doc4 - v2
• Doc5 - v2
• Doc6 - v1
• Doc7 - v1

If you didn't get these results do you understand why? Lets switch the order. What would be the final values of the 7 documents in the target entity?

*Spoiler Alert! The results are displayed below.

• Doc1 - v2
• Doc2 - v1
• Doc3 - v1
• Doc4 - v1
• Doc5 - v2
• Doc6 - v1
• Doc7 - v1

Almost the same results except Doc4 would still be v1 because the Managed 2 solution came after Unmanaged 1 in sequence. Did you do ok in the previous exercises? Awesome. Let's move on.

Managed Solution Import Options

There are three main managed import options when you import the same managed solution into a target environment after the first time. Upgrade, Update and Stage for Upgrade.

Upgrade (default)✔ will be the proper choice in most situations. It is also the Microsoft recommended choice in classic deployment situations. An upgraded solution will add new components and delete components that are no longer in the managed solution. It also does the standard merging of the XML components (Forms, Ribbons, Site Map). This is the classic managed behavior I showed above.

Update has similar behavior to the classic unmanaged solution in that it is additive. Suppose a component no longer exists in the current importing solution but exists in the target environment. It will NOT remove the component (It stays in the managed solution on the target environment). Because of this added feature, there are now fewer arguments against using managed solutions versus unmanaged solutions in higher environments. The difference from unmanaged solutions is that there IS a history. So you can say that the update option has some of the benefits of the managed upgrade and the most crucial benefit of the unmanaged option. There really isn't a great case for unmanaged solutions in upper environments.

Stage for Upgrade is an edge case where the importing solution does not delete the previous version solution or any patches on the target environment. This is usually done as a temporary measure to keep components around in the near future until after a specific event has been completed. For instance, this may be a production data migration process on particular fields or a production event that needs to occur unimpeded in the near future. This should be the least picked option of the three choices.

Patches

The final option I'll cover in this article is patches. To me, patches weren't the most intuitive to understand as there were some new rollup processes and steps that you don't deal with in the standard software patching that you deploy. These extra steps were also brand new to the solution deployment process. That being said, patches enable you to group and separate changes simultaneously. This allows you to somewhat control the chaos by providing a quick and separate software patch for your production environment. Patching certainly doesn't take away the complexity of partitioning and stabilizing future development, but it is there for those willing to invest in the process. Ok, let's dive into the process.

Let's simplify the explanation of patching with a hypothetical scenario. Let's say you ship version 1 of managed Solution A to production. Immediately you find out that a custom button you built on a page holds up and freezes the UI for 3 seconds. Let's pretend this didn't happen in the lower environments, but you have triaged the issue and know how to fix the issue. First, select your Solution A solution and "clone a patch" in your development environment. This will, in effect, create another Solution A patch solution. You would then add the component(s) you are working on for the fix from the original Solution A solution to this Solution A patch. The rest of the components in the original Solution A are now locked, and no one can edit these components. This will cause 1 of 2 things. Either your subsequent sprint development work will be on HOLD until your patch is validated. Or you will create another related Solution A patch by once again cloning a patch and adding the components you will need to work on for the next sprint. These are the two options you have at your disposal to maintain a single managed solution in your production environment. Now some of you may ask how you will only have 1 Solution A managed solution in the upper environments when you have 2 Solution A solutions and potentially three separate Solution A solutions in circulation? The answer is that when you are ready, for instance, your patch is validated. In dev, you can then choose to clone a solution from your original Solution A solution. This will cause all patches to be rolled back into the same solution in your development environment. When you deploy the solution as a managed upgrade, the solution will then roll all previous patches back into 1 FLATTENED managed solution in your upper environments. The patches will cease to have a history in these target environments. This may be a bit difficult to visualize at first so take a look at the diagrams below.

As you can see above, the patch will reside in the target environment as a separately managed solution with a similar name. In the diagram below, the patch is then cloned as a solution and then rolled back into the original Solution A solution.

So this is just one way to handle patches. Other methods could be creating another solution or creating a separate fix environment and rolling the fix back into the original dev environment at the appropriate time. These are all just different options, and the one you choose will depend on your organization's specific needs and preferences (One of them being DevOps). All of these options are plausible.

Conclusion

Microsoft will be adding some managed solution enhancements that will incorporate some unmanaged features to simplify and enhance managed solution imports. When will this finally happen? I'm not sure, but let us try to help each other come up with some best practices that make managed solutions less complex and more predictable. Please tune in for part 2 of this series, where we look at some managed deployment options that would be friendly for incorporating a repeatable DevOps process. 😎