AWS Cloudformation Validation in CI/CD Pipelines

on
jul 07, 2018
in
AWS

Providing autonomy to DevOps teams, faster feedback loops, and ensuring compliant use of the Cloud.

More and more organizations transform to a DevOps organization. This involves giving DevOps teams the responsibility for their cloud infrastructure. I’ll describe how to give DevOps teams the freedom to write their own infrastructure code with Cloudformation, and validate their source code against policies we have set.
Read about how to force the use of tags, use a white list or black list for service types, and check the templates for prohibited property values. You’re up and running in just a few minutes with this blog post and the examples.
Before I start, I’d like to thank Coolblue for creating this open source project together with AWS. The tool is “cool”, but the “how to” seems to be missing.
First we start with a sample cloudformation template. I’ll use S3, because it’s easy and fast to deploy and it’s a resource where all rules may apply. The typo in this example is on purpose.

Resources:
  S3Bucket:
    Type: AWS::S3:Bucket

Now let’s validate this template according the default specs:

$ pip install cfn-lint
$ cfn-lint --template template.yaml
E3001 Invalid or unsupported Type AWS::S3:Bucket for resource S3Bucket in us-east-1
template.yaml:3:9

Enforce Tags Property Values

In large organizations we often see Tagging strategies. We can now make them mandatory in Cloudformation. Some examples you might consider:

  • Team. Name of the team who is responsible for this resource.
  • Application. Name of the application, application stack, or microservice.
  • Environment. The name of the environment this resource belongs to, for example: sandbox/test/prod/live.
  • Repository. Name or ID of the code repository, to easily find the related source code and related pipelines.
  • Commit. Commit ID, to easily find the commit to blame.
  • Expiry. In case this resource is temporary, add a date so everyone knows it could be deleted.
  • CIA. Confidentiality, Integrity and Availability rating. For example: 333 for a high, and 111 for a low classified resource.

Find more about tagging on the AWS website.
Enforcing tags is currently a module you have to write yourself in python. An example can be found here. For our use case here, create a folder “append_rules” and create a new file PropertiesTagsRequired.py and copy the source code from the example. Update the required_tags to:

required_tags = ['Team', 'Application', 'Environment', 'Repository', 'Commit', 'Expiry', 'CIA']

And then run the following command. It should list the missing tags in your Cloudformation template.

$ cfn-lint --template template.yaml \
           --append-rules ./append_rules

E9000 Missing Tag Application at Resources/S3Bucket/Properties/Tags
template.yaml:5:11

E9000 Missing Tag Environment at Resources/S3Bucket/Properties/Tags
template.yaml:5:11

E9000 Missing Tag Repository at Resources/S3Bucket/Properties/Tags
template.yaml:5:11

E9000 Missing Tag Commit at Resources/S3Bucket/Properties/Tags
template.yaml:5:11

E9000 Missing Tag Expiry at Resources/S3Bucket/Properties/Tags
template.yaml:5:11

E9000 Missing Tag CIA at Resources/S3Bucket/Properties/Tags
template.yaml:5:11

A Black List for Resource Types

Our team has decided to white list all services, except for the ones we explicitly exclude. In real use cases, this is to prevent users from deploying expensive clusters, for demo purposes I’ve chosen to exclude SQS. Create a file spec.json and add the following code:

{
  "IncludeResourceTypes": [
    "AWS::*"
  ],
  "ExcludeResourceTypes": [
    "AWS::SQS::Queue"
  ]
}

Now try to create an SQS queue, and run the following command (we skip the previous --append-rules parameter for now).

Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
  SQS:
    Type: AWS::SQS::Queue

$ cfn-lint --template template.yaml \
           --override-spec spec.json

E3001 Invalid or unsupported Type AWS::SQS::Queue for resource SQS in us-east-1
template.yaml:9:9

Enforce a Property

Our teams are allowed to create buckets, but we want to enforce the buckets use Encryption by default. Overwrite the spec.json using the following code:

{
  "IncludeResourceTypes": [
    "AWS::*"
  ],
  "ExcludeResourceTypes": [
    "AWS::SQS::Queue"
  ],
  "ResourceTypes": {
    "AWS::S3::Bucket": {
      "Properties": {
        "BucketEncryption": {
          "Required": true
        }
      }
    }
  }
}

Now run the command again and we see the BucketEncryption property is now required.

$ cfn-lint --template template.yaml \
           --override-specs spec.json

E3001 Invalid or unsupported Type AWS::SQS::Queue for resource SQS in us-east-1
template.yaml:12:9
E3003 Property BucketEncryption missing from resource S3Bucket
template.yaml:4:9

A Forbidden Property Value

The resource AWS::S3::Bucket has an optional property AccessControl. By default the bucket is private, but we want to prevent the teams of creating a public accessible bucket. I need to create my own rule file: append_rules/S3BucketsNotPublic.py.

from cfnlint import CloudFormationLintRule
from cfnlint import RuleMatch


class S3BucketsNotPublic(CloudFormationLintRule):
  """Check if S3 Bucket is Not Public"""
  id = 'E9020'
  shortdesc = 'S3 Buckets must never be public'

  def match(self, cfn):
    matches = list()
    recordsets = cfn.get_resources(['AWS::S3::Bucket'])
    for name, recordset in recordsets.items():
      path = ['Resources', name, 'Properties']
      full_path = ('/'.join(str(x) for x in path))
      if isinstance(recordset, dict):
        props = recordset.get('Properties')
        if props:
          access_control = props.get('AccessControl')
          if access_control:
            forbidden_values = ['PublicRead','PublicReadWrite']
          if access_control in forbidden_values:
            message =  "Property AccessControl set to {0} is forbidden in {1}"
            matches.append(RuleMatch(
              path,
              message.format(access_control, full_path)
            ))
    return matches

Now, when I change my template to force all custom rules and specs checks to fail:

Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead
      Tags:
        - Key: Team
          Value: Binx
  SQS:
    Type: AWS::SQS::Queue
  WrongSQS:
    Type: AWS::SQS:Queue

And run the full check against a template full of problems:

$ cfn-lint --template template.yaml \
           --append-rules ./append_rules \
           --override-spec spec.json

E3003 Property BucketEncryption missing from resource S3Bucket
template.yaml:4:9

E9020 Property AccessControl set to PublicRead is forbidden in Resources/S3Bucket/Properties
template.yaml:4:9

E9000 Missing Tag Application at Resources/S3Bucket/Properties/Tags
template.yaml:6:11

E9000 Missing Tag Environment at Resources/S3Bucket/Properties/Tags
template.yaml:6:11

E9000 Missing Tag Repository at Resources/S3Bucket/Properties/Tags
template.yaml:6:11

E9000 Missing Tag Commit at Resources/S3Bucket/Properties/Tags
template.yaml:6:11

E9000 Missing Tag Expiry at Resources/S3Bucket/Properties/Tags
template.yaml:6:11

E9000 Missing Tag CIA at Resources/S3Bucket/Properties/Tags
template.yaml:6:11

E3001 Invalid or unsupported Type AWS::SQS:Queue for resource WrongSQS in us-east-1
template.yaml:10:9

E3001 Invalid or unsupported Type AWS::SQS::Queue for resource SQS in us-east-1
template.yaml:12:9

Conclusion

If you want to implement this in your CI/CD pipeline, do not forget to make the property Tags required on every resource which supports tagging. Either by adding this to your spec file, or by creating a custom rule.
Often teams are responsbile for their own CI/CD pipeline and could easily block this cfn-lint command, spec files or custom rules. In this case a peer review is an option. The platform team verifies if the DevOps team hasn’t been messing around with the cfn-lint, spec files or custom rules. It improves our security measures.
Some things I describe in this blog post can be secured using AWS IAM policies. I think it’s complementary to each other. Because the policy is applied when you execute a deployment, while this cfn-lint can be used in your IDE and in your CI/CD pipelines. It will save a lot of time and the feedback is much quicker.
Lots of templates are written trial and error. You write some Cloudformation, deploy it in a sandbox and it should work in production. With this cfn-lint we can use current validation and extend with our own rules, so we don’t make mistakes over and over again. We won’t have to create long documents on what the rules are, it became code. While testing in sandbox, with just one command we can check: will it run in production?
Checkout: https://github.com/awslabs/cfn-python-lint for more information and updates!

Share this article: Tweet this post / Post on LinkedIn