Mastering Terraform's Flatten Function: From Nested Chaos to Clean Infrastructure Code

I apologize for the long break - I've been extremely busy.

In the previous article, we created the azure_rg_config variable that describes all the necessary infrastructure. Now we face the question: how do we effectively work with such a nested data structure?

The Problem with Nested Structures.

When dealing with nested map objects (resource groups → vnets → subnets), Terraform cannot directly use them in for_each. We need to transform this multi-level structure into a flat list, where each element contains all the necessary information. This is where the flatten function comes to the rescue.

What Does the Flatten Function Do? The flatten function takes a list that may contain nested lists and transforms it into a single-level (flat) list. A simple example:

flatten([["a", "b"], ["c", "d"]])
Result: ["a", "b", "c", "d"]        

But its true power is revealed when working with complex data structures in combination with for loops.

Practical Implementation for Our Infrastructure

Let's examine how to apply flatten to the azure_rg_config structure from the previous article.

Step 1: Creating a Flat List of Subnets

locals {
  # Transform nested structure into a flat list
  subnets_flatlist = flatten([
    for rg_key, rg_val in var.azure_rg_config : [
      for vnet_key, vnet_val in rg_val.vnets : [
        for subnet_key, subnet_val in vnet_val.subnets : {
          # Key for resource identification
          unique_key = "${rg_key}_${vnet_key}_${subnet_key}"
          # Resource Group information
          rg_name  = rg_key
          location = rg_val.location
          # VNet information
          vnet_name = vnet_key
          # Subnet information
          subnet_name = subnet_val.name
          subnet_ip   = subnet_val.iprange
        }
      ]
    ]
  ])
# Convert list to map for use in for_each
  subnets = { for item in local.subnets_flatlist : item.unique_key => item }
}        

How Does This Work?

Here's the step-by-step breakdown:

1. The first for loop iterates through all resource groups

2. The second for loop iterates through all vnets within each resource group

3. The third for loop iterates through all subnets within each vnet

4. Each iteration creates an object with all necessary information

5. flatten transforms nested lists into one flat list

6. The final transformation creates a map with unique keys for for_each

Step 2: Creating Resource Groups

resource "azurerm_resource_group" "rg" {
  for_each = var.azure_rg_config
  name     = each.key
  location = each.value.location
}        

Simple and clean - we can use the variable directly here.

Step 3: Creating Virtual Networks

locals {
  vnets_flatlist = flatten([
    for rg_key, rg_val in var.azure_rg_config : [
      for vnet_key, vnet_val in rg_val.vnets : {
        unique_key = "${rg_key}_${vnet_key}"
        rg_name    = rg_key
        vnet_name  = vnet_key
        location   = rg_val.location
      }
    ]
  ])
  vnets = { for item in local.vnets_flatlist : item.unique_key => item }
}

resource "azurerm_virtual_network" "vnet" {
  for_each = local.vnets
  name                = each.value.vnet_name
  address_space       = ["10.0.0.0/16"]
  location            = each.value.location
  resource_group_name = azurerm_resource_group.rg[each.value.rg_name].name
}        

Step 4: Creating Subnets

resource "azurerm_subnet" "subnet" {
  for_each = local.subnets
  name                 = each.value.subnet_name
  resource_group_name  = azurerm_resource_group.rg[each.value.rg_name].name
  virtual_network_name = azurerm_virtual_network.vnet["${each.value.rg_name}_${each.value.vnet_name}"].name
  address_prefixes     = [each.value.subnet_ip]
}        

Now let's put it all together:

variable "azure_rg_config" {
  type = map(object({
    location = string
    vnets = map(object({
      subnets = map(object({
        name    = string
        iprange = string
      }))
    }))
  }))

  default = {
    "rg1_office_production" = {
      location = "East US",
      vnets = {
        "office_vnet" = {
          subnets = {
            "management_subnet" = {
              name    = "management"
              iprange = "10.0.1.0/24"
            },
            "sales_subnet" = {
              name    = "sales"
              iprange = "10.0.2.0/24"
            }
          }
        },
        "production_vnet" = {
          subnets = {
            "prod_subnet" = {
              name    = "prod"
              iprange = "10.0.3.0/24"
            }
          }
        }
      }
    },
    "rg2_lab_data_support" = {
      location = "West Europe",
      vnets = {
        "lab_vnet" = {
          subnets = {
            "lab_subnet" = {
              name    = "lab"
              iprange = "10.0.5.0/24"
            }
          }
        }
      }
    }
  }
}

locals {
  # Flatten for VNets
  vnets_flatlist = flatten([
    for rg_key, rg_val in var.azure_rg_config : [
      for vnet_key, vnet_val in rg_val.vnets : {
        unique_key = "${rg_key}_${vnet_key}"
        rg_name    = rg_key
        vnet_name  = vnet_key
        location   = rg_val.location
      }
    ]
  ])
  vnets = { for item in local.vnets_flatlist : item.unique_key => item }

  # Flatten for Subnets
  subnets_flatlist = flatten([
    for rg_key, rg_val in var.azure_rg_config : [
      for vnet_key, vnet_val in rg_val.vnets : [
        for subnet_key, subnet_val in vnet_val.subnets : {
          unique_key  = "${rg_key}_${vnet_key}_${subnet_key}"
          rg_name     = rg_key
          location    = rg_val.location
          vnet_name   = vnet_key
          subnet_name = subnet_val.name
          subnet_ip   = subnet_val.iprange
        }
      ]
    ]
  ])
  subnets = { for item in local.subnets_flatlist : item.unique_key => item }
}

# Resource Groups
resource "azurerm_resource_group" "rg" {
  for_each = var.azure_rg_config

  name     = each.key
  location = each.value.location
}

# Virtual Networks
resource "azurerm_virtual_network" "vnet" {
  for_each = local.vnets

  name                = each.value.vnet_name
  address_space       = ["10.0.0.0/16"]
  location            = each.value.location
  resource_group_name = azurerm_resource_group.rg[each.value.rg_name].name
}

# Subnets
resource "azurerm_subnet" "subnet" {
  for_each = local.subnets

  name                 = each.value.subnet_name
  resource_group_name  = azurerm_resource_group.rg[each.value.rg_name].name
  virtual_network_name = azurerm_virtual_network.vnet["${each.value.rg_name}_${each.value.vnet_name}"].name
  address_prefixes     = [each.value.subnet_ip]
}        

Key Advantages of This Approach:

Scalability - Adding a new subnet, VNet, or Resource Group only requires changing the variable DRY Principle - No code duplication for resources Readability - All infrastructure configuration is in one place Flexibility - Easy to adapt the structure to new requirements Maintainability - Changes are made in a single location

What's Next?

Now, when infrastructure requirements change, you simply update the azure_rg_config variable, and Terraform will automatically create or update all necessary resources.

This significantly simplifies management of rapidly evolving projects. In future articles, we can explore: - Using dynamic blocks for even greater flexibility - Creating reusable modules - Managing Terraform state in a team environment

What challenges have you faced when managing complex Terraform configurations? Share your experiences in the comments!

#Terraform #Azure #InfrastructureAsCode #DevOps #CloudComputing #IaC