Module Parameter Defaults with the Terraform Object Type

In this post I will walk you through the challenges I’ve faced when adopting the object type in my Terraform 0.12 modules, and the solutions I came up with to work around the caveats. As an example I will use the google_storage_bucket resource, as this is part of one of the modules I’ve built.

The Terraform Object Type

With the launch of Terraform 0.12 back in May 2019, a lot of new cool features have been introduced. Compared with Terraform 0.11, where you would find yourself repeating a lot of code, you can now utilize the new for_each functionality and object type to write cleaner code.

If you haven’t discovered the new object type yet, you may be surprised by its potential. It is one of the two complex types Terraform provides and gives you the possibility to describe object structures. While this new type has a lot of advantages, such as the ability to mix types and define multi layered structures, it also has a few caveats which I will explain further down.

The content below is taken from the Terraform docs itself:
object(...): a collection of named attributes that each have their own type.
The schema for object types is { = , = , ... } — a pair of curly braces containing a comma-separated series of = pairs. Values that match the object type must contain all of the specified keys, and the value for each key must match its specified type. (Values with additional keys can still match an object type, but the extra attributes are discarded during type conversion.)

Example object structure:

my_object = {
  a_string = "example",
  a_number = 1,
  a_boolean = true,
  a_map = {
    type1 = 1,
    type2 = 2,
    type3 = 3
  }
}

Provide Module Default Params with Object

As promised, in this blog post I will explain how I used the object type in my custom Terraform module.
In the example below I’ve used the object type to define the supported settings of the GCP storage bucket.

variable "bucket_settings" {
  type = object({
    location           = string
    storage_class      = string
    versioning_enabled = bool
    bucket_policy_only = bool
    lifecycle_rules = map(object({
      action = map(string)
      condition = object({
        age                   = number
        with_state            = string
        created_before        = string
        matches_storage_class = list(string)
        num_newer_versions    = number
      })
    }))
  })
}

Previously, when writing your Terraform module, you would need to create a variable for each setting you want to pass to your resource.
Sure, you would have maps and lists, but a map could only contain values of the same type, limiting the use of it greatly.
When using the object type, we can actually combine these settings in a complex structure.

I can now use this to populate my bucket, by providing all these settings in a variable (bucket_settings in the listing below).

bucket_settings = {
  location           = "europe-west4"
  storage_class      = "REGIONAL"
  versioning_enabled = true
  bucket_policy_only = true
  lifecycle_rules    = {}
}

Since I have certain settings that I want to have applied to all of my buckets, I want to have most of these settings to be set for me by default.
To do this, we can provide a default value for the bucket_settings object in the variable definition itself:

variable "bucket_settings" {
  type = object({
    location           = string
    storage_class      = string
    versioning_enabled = bool
    bucket_policy_only = bool
    lifecycle_rules = map(object({
      action = map(string)
      condition = object({
        age                   = number
        with_state            = string
        created_before        = string
        matches_storage_class = list(string)
        num_newer_versions    = number
      })
    }))
  })

  default = {
    location           = "europe-west4"
    storage_class      = "REGIONAL"
    versioning_enabled = true
    bucket_policy_only = true
    lifecycle_rules    = {}
  }
}

Now, if I create my bucket, I can just import the module like this:

module "my_bucket" {
  source = "path.to/my/module
  name   = "bucket-with-defaults"
}

And since I’ve set the defaults, I don’t need to provide the bucket_settings variable at all.

Change One Setting and The Defaults Disappear

But what if I want to overwrite just one of the defaults and keep the rest? In the next example, I try to set a lifecycle policy for the bucket using my module. Notice how I am passing the bucket_settings as a parameter.

module "my_bucket" {
  source = "./modules/bucket"
  name   = "bucket-with-defaults"

  bucket_settings = {
    lifecycle_rules = {
      "delete rule" = {
        action = { type = "Delete" }
        condition = {
          age        = 30
          with_state = "ANY"
        }
      }
    }
  }
}

Unfortunately, this will result in multiple errors:

Missing attributes

The given value is not suitable for child module variable "bucket_settings"
defined at modules/bucket/main.tf:6,1-27: attributes "bucket_policy_only",
"location", "storage_class", and "versioning_enabled" are required.

Since I provided the bucket_settings for this module, it will overwrite the entire defaults I’ve set for the variable. I will need to provide the keys like location, storage_class, versioning_enabled and bucket_policy_only as well.

But even after providing these settings, I’m getting a new error:

The given value is not suitable for child module variable "bucket_settings"
defined at modules/bucket/main.tf:6,1-27: attribute "lifecycle_rules": element
"delete rule": attribute "condition": attributes "created_before",
"matches_storage_class", and "num_newer_versions" are required.

For the lifecycle_rule condition, I’ve only provided the age and with_state since I don’t care about created_before, matches_storage_class or num_newer_versions. But since I’ve defined them in the object definition, Terraform will complain and tell me I need to provide them as well.

So to make this work with the module setup described above, I will need to provide all of the settings again.
As you can see, especially when you want to create multiple buckets with only a few differences in the bucket configuration, this will become quite cumbersome.

Solution, using a defaults variable and merge

Because I only want to provide the settings that differ from the defaults, I’ve come up with a solution for this. When I create a separate variable for the defaults, I can merge the provided bucket_settings with this bucket_defaults variable, before calling the actual bucket provider.

The end result looks like this:

Module code snippet

variable "name" {
  type = string
  content = "The name of the bucket"
}

variable "bucket_defaults" {
  type = object({
    location           = string
    storage_class      = string
    versioning_enabled = bool
    bucket_policy_only = bool
    lifecycle_rules = map(object({
      action = map(string)
      condition = object({
        age                   = number
        with_state            = string
        created_before        = string
        matches_storage_class = list(string)
        num_newer_versions    = number
      })
    }))
  })

  default = {
    location           = "europe-west4"
    storage_class      = "REGIONAL"
    versioning_enabled = true
    bucket_policy_only = true
    lifecycle_rules    = {}
  }
}

variable "bucket_settings" {
  content = "Map of bucket settings to be applied which will be merged with the bucket_defaults. Allowed keys are the same as for bucket_defaults."
}

locals {
  merged_bucket_settings = merge(var.bucket_defaults, var.bucket_settings)
}

I can now use the module by providing only the lifecycle rule that I want to set for my bucket, and all other defaults will still be applied.

Using the module

module "my_bucket" {
  source = "path.to/my/module
  name   = "bucket-with-defaults"

  bucket_settings = {
    lifecycle_rules = {
    "delete rule" = {
        action = { type = "Delete" }
        condition = {
          age          = 30
          with_state = "ANY"
        }
      }
    }
  }
}

Conclusion

I like the fact that I can now define objects with mixed types to create more complex structures. There are a couple of things I think that could be improved:

  • The way variable defaults are being handled. It would be great if these would be merged with the provided value, or at least have a setting to allow for this behaviour.
  • You should be able to define optional keys for your objects. Although the need for this will be come less urgent when the first point is taken care of.

Hopefully this blog post will help you understand the object type and it’s limitations, while giving you an idea how to utilize it to it’s best potential.
In my next blog post, I will tell you more about the for_each functionality, and how to utilize this in your Terraform modules as well.

Share this article: Tweet this post / Post on LinkedIn