CloudFormation - Building with unknowns in AWS
Hello, and welcome to an in-depth exploration of Amazon Web Services. This is the third article in a series of posts focused on leveraging AWS services to make your life easier, while providing the most scalable, stable and secure platform possible. The first article touched on AWS CloudFormation and Nested stack outputs, and is available here. The second addressed passing parameters into a Nested Stack and is available here (FIX THIS LINK!) This article focuses on expanding usage of CloudFormation templates across AWS Regions and working with unknowns, specifically, which Availability Zones (AZs) are available in which region. If you are unfamiliar with how AWS defines a Region or AZ, I suggest checking out this post over at the wonderful CloudAcademy blog.
My goal this time around is to write my CloudFormation templates in such away that they can be deployed across as many AWS regions as possible, without needing to be updated based on a specific region. In order to provide the most stable application platform possible, I recommend using AWS Regions with a minimum of three AZs, ensuring that each region is served by at least three, distinct, separate data centers to minimize the risk of downtime.
Looking at AWS Global Infrastructure page, I am able to determine that the regions shown in the table below meet my requirements:
Now that I know that these regions meet my requirements, how can these specific AZs be used without hardcoding the specific name for each? AWS provides solution via their Command Line Interface through commands such as “aws ec2 describe-availability-zones” which returns a nicely formatted JSON output:
{
"AvailabilityZones": [
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1b"
},
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1c"
},
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1d"
}
]
}
This output is great, if you are interacting via the CLI, API or an application running on AWS Lambda. Sadly this doesn’t help us when working with CF templates directly... Or does it? JSON is structured in a consistent manner allowing us to reference items within it. This allows us the ability to leverage the CF Function “GetAZs” along with a the “Select” function to get what we need, and streamline our use of multiple AZs without hard-coding each specifically.
This has been particularly helpful to me as an engineer at my current company. I need to roll out a specific VPC/Subnet layout to multiple regions, quickly and seamlessly. Specifically, I need those subnets to be a CIDR defined by the Region and AZ within which they exist. Below, you can see how I tackled this in my own template. As with my previous article, please note these are only template snippets and not full, functional stacks.
I use mappings to define the initial state by region, based on our own internal methodology. This is done primarily to allow us to identify the location of something by IP in addition to the AWS tags. These can be passed into the CF stack or setup as I have done as they are reused in multiple places making the.
CF Mappings:
"Mappings" : {
"VPCipStart" : {
"us-west-1" : {"Octet" : "172.254"},
"us-west-2" : {"Octet" : "172.252"},
"us-east-1" : {"Octet" : "172.244"},
"sa-east-1" : {"Octet" : "172.240"},
"ap-northeast-1" : {"Octet" : "172.218"},
"ap-southeast-2" : {"Octet" : "172.206"}
}},
Resource Stack:
"Resources": {
"DMZSubnet1" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"AvailabilityZone" : { "Fn::Select" : ["0",
{ "Fn::GetAZs" : { "Ref" : "AWS::Region" } }] },
"VpcId" : { "Ref" : "VPC" },
"CidrBlock" : { "Fn::Join" : [ ".", [ { "Fn::FindInMap" : [ "VPCipStart", { "Ref" : "AWS::Region"}, "Octet" ] }, "100.0/24" ] ] },
"Tags" : [{
"Key" : "Name",
"Value" : {"Fn::Join" : [ "-", [ "DMZ", { "Fn::Select" : ["0",
{ "Fn::GetAZs" : { "Ref" : "AWS::Region" } }
]} ]] } }] } },
"DMZSubnet2" : {
"Type" : "AWS::EC2::Subnet",
"Condition" : "Create",
"Properties" : {
"AvailabilityZone" : { "Fn::Select" : ["1",
{ "Fn::GetAZs" : { "Ref" : "AWS::Region" } }
] },
"VpcId" : { "Ref" : "VPC" },
"CidrBlock" : { "Fn::Join" : [ ".", [ { "Fn::FindInMap" : [ "VPCipStart", { "Ref" : "AWS::Region"}, "Octet" ] }, "100.0/24" ] ] },
"Tags" : [ {
"Key" : "Name",
"Value" : {"Fn::Join" : [ "-", [ "DMZ", { "Fn::Select" : [
"1",
{ "Fn::GetAZs" : { "Ref" : "AWS::Region" } }
]} ]] } }] } },
Here is a breakdown, or these Join, Select, and GetAZs statements. This is the one we are most interested in:
"AvailabilityZone" : { "Fn::Select" : [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" } } ] },
In this code, I am defining the AvailabilityZone I want this particular DMZSubnet1 to exist within, the { "Fn::GetAZs" : { "Ref" : "AWS::Region" } calls the CLI tool behind the scenes and returns that JSON document as seen above, all transparently to you as the user. Once we have that JSON list in the background the { "Fn::Select" : [ "0", { leverages the CF Select Function to retrieve the first item in the ordered list, which is “0.” This gives us our first availability zone, regardless of it being 1A, 2C, or 1B.
Later on, under the “Tags” section, I use this same select function to set the “Name” of this particular Subnet.Assuming the we are in the US-East-1 region, the output would be “DMZ-us-east-1b”.
The next resource, “DMZSubnet2”, simply replaces the select “0” with a “1” to get the next AZ for that region, increment that number to “2” in order to get the third and final AZ.
Of course, this method does have its limitations. If the select function was expanded out to the 4th item (which doesn’t exist in most regions), the stack will fail to build, same as if it was run out to the third iteration in a region with only two AZs. However, within most deployments, these are all easy-to-live with limitations, and I appreciate the ease of replicating the stack in most regions over the alternative. Hope you do too!
Questions or comments? Please leave them below and I will do my best to answer them as quickly and efficiently as I can.