Transparently Local Testing of Ansible Scripts running on EC2 Instance using AWS Mocks and HTTPS Proxy

Introduction of Article Series

A very important requirement of nowadays projects is time to market facilitating developing better products than other competitors do. We as developer can contribute to the time to market race by optimizing our development processes. In this article series, we focus on developing local tests for Ansible Scripts that will be later run on EC2 Instances via AWS System Manager, so that we can reduce the cycle times between detecting bugs and the fixing them. System Manager is the central place for managing your EC2 instances and integrates quite well with Ansible for having a standardized way of changing your EC2 Instances declarative. Anyhow, the way of mocking AWS Services for tests is not limited to the System Manager/Ansible Use Case.

Let’s make an example: we have EC2 Instances that runs our applications that are deployed from archives which are stored in S3. It is very critical that these artifacts are not tampered and thus, we employ an artifact signing process. Therefore, we implement our process by running an Ansible Script under control of AWS System Manager that downloads the artifacts and validates its checksum stored in our metadata store.

For testing the scenario we mock the AWS Services and simulate the EC2 Instances via docker containers.

On one hand, the exact testing with building up real infrastructure is less error prone. On the other hand, it might be difficult to be managed (e.g., guarantee decommissioning of infrastructure) and it takes usually more time than unit testing. Local testing with simulation the EC2 Instance with Docker Containers might be helpful here depending on your use case.

Testing with Localstack and HTTPS Proxy

In the second approach, we use an https proxy for transparently rerouting AWS Requests to the Local Stack Container. Therefore, you have to set the HTTPS Proxy Environment to our HTTP Proxy (i.e., mitmproxy, Source). We extend mitmproxy with a small python script for implementing the rerouting (Source). Additionially, we need to extend our EC2 Simulation Container with the mitmproxy root certificate so that the SSL Certifcates of mitmproxy are accepted by our tests (Source (line no.:10,11,14,17)):. All this settings, happens outside of the Ansible Scripts and our Ansible Tests.

Here you can find the complete code: example code.

Thank you for reading this article. I am happy to read other opinions in the comments below and other helpful input. Please comment and let’s learn from each other.

Thank you also to Stefan Kühnel for his valuable input on this article series.

Looking forward, you continuing reading this small series.