AWS Lambda - Python 3.7 Support

At re:Invent 2018, AWS announced Python 3.7 runtime support for AWS Lambda. AWS Lambda has supported for Python 2.7 and 3.6 for some time. Python 3.7 is a great addition. Python 3.7 provides features like data classes, and Postponed Evaluation of Annotations that make working with ‘data records’ simpler. Lets take a look!

Data as a record

Data can be represented in a lot of ways. When data is being transported, a common format is used like binary ones and zeroes, JSON, XML, CSV. When data is being processed, a typed representation of data supports the type checker validate the data processing logic. To materialize data in a typed form, Python 3.7 supports data classes. When data classes are combined with mashumaro, a fast and well tested serialization framework on top of dataclasses, it gets very easy to commit data to a fixed shape. Lets see what that looks like.

JSON package

Python traditionally represents data as a dictionary of key-value pairs that can be converted from and to JSON with the json package.

import json


def handler(event: dict, context) -> dict:
    return {
        'statusCode': 200,
        'body': json.dumps('Hello World')
        }

Abstracting the Lambda Response

With data classes, a high level Response object can be created that abstracts the details of responding with a JSON dictionary to the API Gateway Proxy. In the example we use the new Python 3.7 features ‘postponed evaluation’ and ‘data classes’. Data classes provide the function asdict that converts the dataclass instance to a dict.

from __future__ import annotations
import json
from dataclasses import dataclass, asdict


@dataclass
class Response:
    statusCode: int = 200
    body: str = ''

    @classmethod
    def of(cls, status_code: int, body: dict) -> Response:
        return Response(status_code, json.dumps(body))

    def respond(self) -> dict:
        return asdict(self)


def handler(event: dict, context) -> dict:
    return Response.of(200, { 'msg': 'Hello World' }).respond()

Typed Responses

It is possible to return typed responses in the processing logic of the lambda. The example below uses mashumaro that provides a typed representation of data. In the example below, the Response class accepts a Message, that will be used when the lambda responds.

from __future__ import annotations
import requests
from requests.auth import HTTPBasicAuth
from dataclasses import dataclass, asdict
from mashumaro import DataClassJSONMixin


@dataclass
class Response(DataClassJSONMixin):
    statusCode: int = 200
    body: str = ''

    @classmethod
    def of(cls, status_code: int, msg: Message) -> Response:
        return Response(status_code, msg.to_json())

    def respond(self) -> dict:
        return asdict(self)


@dataclass
class Message(DataClassJSONMixin):
    message: str


def say_hello(msg: Message) -> dict:
    resp = requests.post(
        'https://httpbin.org/post',
        json=msg.to_dict(),
        auth=HTTPBasicAuth('username', 'password'),
        verify=False,
        timeout=2)
    try:
        return resp.json()['json']
    except Exception as e :
        return { 'msg': f'No body in response {e} -> {resp.text}' }


def handler(event: dict, context) -> dict:
    try:
        payload: dict = say_hello(Message("Hello World"))
        payload.update({'message': f"Received from httpbin: {payload['message']}"})
        msg: Message = Message.from_dict(payload)
        return Response.of(200, msg).respond()
    except Exception as e:
        return Response.of(500, Message(str(e))).respond()

Python 3.7 runtime configuration

CloudFormation supports configuring lambdas to use Python 3.7, by setting the Runtime field to python3.7.

  HelloLambda:
    Type: AWS::Lambda::Function
    Properties:
      Handler: helloworld.handler
      Runtime: python3.7
      Role: !GetAtt 'LambdaBasicExecutionRole.Arn'
      MemorySize: 128
      Timeout: 30
      Code:
        S3Bucket: !Ref S3Bucket
        S3Key: !Ref S3Key
        S3ObjectVersion: !Ref S3Version

Example

The example project contains an example project containing the lambdas and CloudFormation configuration that uses the Python 3.7 runtime. The example can be deployed by typing make deploy and removed with make delete.

Conclusion

It is great that Lambda supports Python 3.7. In this blog we have seen some of the exciting new features of Python and how we can apply them in lambda code in order to create more safe code. We have looked at a simple lambda, how you could normally serialize data. We have seen how we can abstract the respond logic using Python 3.7 features. Finally we saw how we can make a service call to httpbin and respond, in a fully type-safe way.

AWS Lambda – Python 3.7 Support