Testing Ansible Scripts locally using an HTTPS Traffic Recorder

 Introduction of Article Series

A very important requirement of nowadays projects is time to market facilitating developing better products than other competitors do. We as developer can contribute to the time to market race by optimizing our development processes. In this article series, we focus on developing local tests for Ansible Scripts that will be later run on EC2 Instances via AWS System Manager, so that we can reduce the cycle times between detecting bugs and the fixing them. System Manager is the central place for managing your EC2 instances and integrates quite well with Ansible for having a standardized way of changing your EC2 Instances declarative. Anyhow, the way of mocking AWS Services for tests is not limited to the System Manager/Ansible Use Case.

Let’s make an example: we have EC2 Instances that runs our applications that are deployed from archives which are stored in S3. It is very critical that these artifacts are not tampered and thus, we employ an artifact signing process. Therefore, we implement our process by running an Ansible Script under control of AWS System Manager that downloads the artifacts and validates its checksum stored in our metadata store.

For testing the scenario we mock the AWS Services and simulate the EC2 Instances via docker containers.

On one hand, the exact testing with building up real infrastructure is less error prone. On the other hand, it might be difficult to be managed (e.g., guarantee decommissioning of infrastructure) and it takes usually more time than unit testing. Local testing with simulation the EC2 Instance with Docker Containers might be helpful here depending on your use case.

Testing with an HTTPS Proxy Recorder

In the final article of this series, we use the HTTP Proxy (i.e., mitmproxy) for recording responses from real infrastructure temporary build up, running the tests against it while recording request and responses. This recordings will be used again for daily tests and these recordings can be refreshed whenever needed. Therefore, the HTTPS Proxy defines services to change different modes of the recording (Mitmproxy-Addon). The recording or playback is simply controlled by and environment switch (Source) and the handling of the switch is done via a decorator (Source).

Due to the fact that you use responses from real requests recorded earlier, you are close to calling real services, except that the answers might be outdated. According to your requirements, you can refresh this recording depending on your freshness needs. Further, you can mock with this approach any HTTP Service or HTTPS Service. For simplification of the recording and playing back of AWS Request, we use the same tests with a flag that activates different behavior in the recorder.

Still the is the option of using Localstack for mocking AWS Services (see previous article). So this is another option, which can be considered on a use case basis and shows how you can use mitmproxy for doing such kinds of recordings.

Please find the complete example code here: Source.

Thank you for reading this article. I am happy to read other opinions in the comments below and other helpful input. Please comment and lets learn from each other.

Thank you also to Stefan Kühnel for his valuable input on this article series.