Blog

Building CloudFormation Custom Resources is Plain and Simple

25 Aug, 2018
Xebia Background Header Wave

I discovered most blog posts and documentation about Custom Resources for CloudFormation, are very complicated. It’s perfect for experienced users, but it’s pretty hard to use it for the first time. This blog post is really easy to use as your first CloudFormation Custom Resource project, and generally a good fit for most use cases. Spoiler: the whole Custom Resource Stack is a single file. It doesn’t have to be packaged and uploaded to S3, and it’s deployed using a single command.

Intro

Consider the following CloudFormation template. BucketName is not a required property, but let’s assume it is and you would like it to be randomly generated. Some other resources enforce you to enter a fixed name. You can’t deploy the same template again, or you need to specify a random name as a parameter. I like all my resources to get a random suffix, so I need some kind of RandomString function in my CloudFormation template.

Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: "myfixedbucketname"

Deploy a Custom Resource Lambda

It’s time to deploy the Lambda function for the Custom Resource. In many cases a Lambda function consists of multiple files and build and deployment steps. In this example it’s just a single CloudFormation file with inline Python code.

Resources:
  RandomString:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          import base64
          import json
          import logging
          import string
          import random
          import boto3
          from botocore.vendored import requests
          import cfnresponse

          logger = logging.getLogger()
          logger.setLevel(logging.INFO)

          def random_string(size=6):
            return ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(size))

          def lambda_handler(event, context):
            logger.info('got event {}'.format(event))
            responseData = {}

            if event['RequestType'] == 'Create':
              number = int(event['ResourceProperties'].get('Number', 6))
              rs = random_string(number)
              responseData['upper'] = rs.upper()
              responseData['lower'] = rs.lower()

            else: # delete / update
              rs = event['PhysicalResourceId'] 
              responseData['upper'] = rs.upper()
              responseData['lower'] = rs.lower()

            logger.info('responseData {}'.format(responseData))
            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, responseData['lower'])

      FunctionName: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:random-string'
      Handler: "index.lambda_handler"
      Timeout: 30
      Role: !GetAtt 'LambdaRole.Arn'
      Runtime: python3.6
  # The LambdaRole is very simple for this use case, because it only need to have access to write logs
  # If the lambda is going to access AWS services using boto3, this role must be
  # extended to give lambda the appropriate permissions.
  LambdaRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          -
            Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: "lambda-logs"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource:
                  - "arn:aws:logs:*:*:*"

Now deploy this template using the following command.

aws cloudformation create-stack \
    --stack-name cfn-custom-resources \
    --template-body file://template.yaml \
    --capabilities CAPABILITY_IAM

Use the Custom Resource

Now the Custom Resource Lambda function seems to work, it’s time to change our CloudFormation template to use the Custom Resource to generate a random string for our S3 Bucket.

Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub "mybucket-${RandomString.lower}"
  # returns attributes: lower and upper
  RandomString:
    Type: Custom::RandomString
    Properties:
      ServiceToken:
        !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:random-string"
      Number: 8

By providing the Number: 8 you can change the default of 6 digits to a longer random string. We use the “lower” string in this example, because an S3 bucket must be lower case.
Try it out and see what happens. You should now have a bucket with a bucketname starting with “mybucket-” followed by a random string. You can enforce an update of the RandomString resource by adding tags to your CloudFormation stack. The random string will not be regenerated, because the PhysicalResourceId is the random number, we can easily use this as an input.

Next Steps & Conclusion

Some conclusions:

  • It is easy to create a CloudFormation Custom Resource in a single CloudFormation template
  • It is easy to write, deploy and test the Lambda Function, including the Role and IAM Policy to access the AWS resources
  • Updating the stack and running the test can easily be done in a single bash command, or by adding the deployment in the test script
  • With this RandomString Custom Resource it becomes easy to expiriment multiple templates at the same time, knowing every resource has a unique name

Single Template

Somebody asked me if it’s also possible to add a custom resource together with the rest of your stack. The answer is yes, and this is the example template:

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  InputBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub 'yourbucket-${RandomName}'

  RandomName:
    Type: Custom::RandomNameGenerator
    Properties:
      ServiceToken: !GetAtt 'RandomNameGenerator.Arn'

  RandomNameGenerator:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.lambda_handler
      Timeout: 30
      Role: !GetAtt 'LambdaBasicExecutionRole.Arn'
      Runtime: python3.6
      Code:
        ZipFile: |
          import base64
          import json
          import logging
          import string
          import random
          import boto3
          from botocore.vendored import requests
          import cfnresponse

          logger = logging.getLogger()
          logger.setLevel(logging.INFO)

          def random_string(size=6):
            return ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(size))

          def lambda_handler(event, context):
            logger.info('got event {}'.format(event))
            responseData = {}

            if event['RequestType'] == 'Create':
              number = int(event['ResourceProperties'].get('Number', 6))
              rs = random_string(number)
              responseData['upper'] = rs.upper()
              responseData['lower'] = rs.lower()

            else: # delete / update
              rs = event['PhysicalResourceId'] 
              responseData['upper'] = rs.upper()
              responseData['lower'] = rs.lower()

            logger.info('responseData {}'.format(responseData))
            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, responseData['lower'])

  LambdaBasicExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
          Action: sts:AssumeRole
          Condition: {}
      Path: /
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts