Explaining different infrastructure management methodologies

They are different ways to manage infrastructure each method has it own pros and cons. You should pick a method that matches the need, but you should also consider the skill level of team and other item the teams are responsible for. Choosing a new method can be used to skill up, but this only works if the team has the team and motivation to learn the new method. Teams can use multiple methods, but it will increase the amount of work. This is because they will need to repeat work for each of the methods. The different methods to manage infrastructure are:

1. Manual Management
2. Automation via scripting
3. Configuration as code
4. Immutable infrastructure

A key way to investigate a method is to take a look at how the systems are set up or make a change.

Manual Management

This method is where tasks are done manual, with few scripts or none at all. In this method to set up a new system, an admin will need to log in and type the command or use a GUI to set it up. The admin will also make runbooks out of the common tasks in most cases, which will be followed later on. A runbook, is basically a set of instructions on how to do a task, they include commands or screenshots. Often this paired with a manual change management process. Where an admin needs to open up a change request describing what to be done. Then someone will approve it. The work typically take place in a maintenance window. Before a change is made it is sometime tested out on a staging environment. This method requires little setup to get going. Items can be figured out a long the way. Someone does not need to spend a lot of at the beginning coding the setup of everything.

Pros:

1. This process require little up front cost
2. This tends to be good for one off scenarios
3. Doesn't require scripting
4. The barrier to entry is low
5. Doesn't require API for this to work

Cons:

1. This process is very labor-intensive. Each server will require additional administration resources, more than other methods
2. This is very hard scale
3. This method is prone to human error
4. The complex configuration are harder to achieve reliable, due to the increase chance of human error
5. Process can be very hard to repeat reliable
6. The cost does not decrease over time

Automation via scripting

This method addresses some repeatable issues with manual management method. It can also reduce the cost at scale and speed up the deployment process. This method achieves this by taking common tasks and automating them via scripting or automation. Scripting can be done in many ways, this may include bash scripts, web browser automation frameworks or configuration management languages like ansible. The automation will do a task, but there is still an admin behind scene figuring out when to run the task. They are some key difference with this method and configuration as code, which I will describe in the next section. When a change needs to be made or a server needs to be setup, the admin will run the correct script. The automation may still include some manual steps, but significantly less than manual management method. This process will require runbooks and similar change management process as the manual method. Admins will typically try out the change on a staging environment first. The runbooks tends to be simpler then the manual management process.

Pros

1. Doesn't have significant up front cost, the automation work is typically done throughout the project lifespan. Instead of all at the start
2. Scales a little better than the manual management
3. Less error-prone than the manual management

Cons:

1. Still relatively labor-intensive, even though it is less than manual management
2. Hard to scale
3. This method is prone to human error
4. The cost does not decrease over time

Configuration as code

This method is similar to automation, but there is one key difference which has a big impact. The difference is instead of writing automation you are defining the state of the server in code. Automation is used to achieve the state. Let's take install a service for example. The automation method will have the admin write a script to install the service. The configuration as code method will define the state that service should be installed, and if it is not installed then it will install it. This means the automation can run at any point of time, and it doesn't require the admin to figure out what to do. The code contains all that information. This is typically done via configuration management languages like ansible, chef, and puppet. This method changes the conversation from what tasks to run, to what should the server look like. All the knowledge of what to do when is capture is code. This captures the expertise in code. This significantly improve repeatability. Due to the majority of expertise is in code, runbooks are not needed as much. You may still see them because it is hard to achieve this for 100% of the work. To make a change you need to update the code to describe a new configuration. This leads to the code being stored in a source code management system, which introduces development practices into the mix. To set up a server, the code just needs to be run.

Pros:

1. Easy to scale this method to different servers
2. Adding a new server doesn't cost a lot, since it is the same configuration code running on each
3. After the initial setup the cost is lower
4. The method is not as prone to human error, as automation method

Cons:

1. Upfront cost to build the automation
2. Require skilled engineers. They will need to know the configuration management language like ansible. They also need to know how to use the configuration management language to manage as well as to set up the server.
3. At large scales may have machine variance, which can cause random bugs

Immutable infrastructure

This concept takes a different look at hosting infrastructure. The previous methods are about building a server from scratch with various different levels and type of automation. Immutable infrastructure is about building a master image and cloning it many times. It is like stamping something out of a mold in a factory. This way each copy is identical to the original. This reduces variations and basically make it non-existent. At high level of scalability variations can cause problems. It is typically the cause of the one off or weird behavior. One server is a little different which causes an issue. This make scaling very simple, but building the master can be challenging. This task is especially challenging for legacy applications, who are not designed for this method. Recently this method is increasing in popularity, with the introduction of containers and container management services like kubernetes. With the maturing and increase popularity of the tools, the cost to building the master, and finding people who have the skills execute are easier. The new technology makes it possible for one team to build off the work of another. Since the infrastructure is immutable, to make a change requires a rebuild to master template with the change included. This is because you can't change the deployed instance. A common practice is to have a devops pipeline run some automation to build the master template. This method tends to utilize development practices. A pattern which can be achieved with containers, is having the development provide the container to the operation team. This way the operation team doesn't need to know how to install the application the development team is writing.

Pros:

1. Very easy to scale
2. After the initial setup the cost is relatively low
3. The method is not as prone to human error
4. Works well at large scale, because you don't have machine variance

Cons:

1. Upfront cost to build the automation
2. Require skilled engineers. They will need to know how to build the immutable assets like container. They will also need to know the immutable orchestration engine like Kubernetes.

Conclusion

One trade off you see with the different method is upfront cost vs ongoing cost. The more manual methods the less upfront cost you have, but the more ongoing you have. The more advance methods you have the opposite. If you are doing one off installation, the more manual methods can be easier. If you have an application that needs to scale, the more advance methods will be better. The bigger the scale the more important and better the more advance techniques are. Using the advance techniques will cost more to start, but will save in the long run.

Something interesting is happening in the industry today, the rise of containers. This trend is starting to driving everybody to the immutable infrastructure via containers. A key benefit which is driving the adoption is reusability. You can get a container from the community or the vendor, instead of building it yourself. Instead of everyone figuring out to set up the items up, only one person need to figure it out. Since containers are standardized, an admin can reuse the same platform for all the container, even if they come from different places. This make deploy significantly simpler.