Training Tasters: One Week of Free Data, AI, and Cloud Training
Join us in the week of 18 – 22 January for five different training tasters: Python, Analytics Translation, AWS Architecting, Google Cloud Run.
Cloud Migration Scenarios
Four scenarios to migrate to AWS – from infrastructure to ML
When you develop an AWS Lambda function in Python, you may require packages which include binary libraries. In this blog we will show you how to use the official Docker Python image to make sure you have a working Lambda.
If you are developing your code on a Mac or Windows operating system, it is pretty hard to install the correct Python packages that your code depends on. There are even github repositories with pre-compiled binaries, which blindly copy into your zip file. This is an unwanted process, as it deviates from the standard way of developing and installing Python software.
Luckily Docker provides us with an official Python image based upon 64-bit Linux, which matches the AWS Lambda runtime. By running the installation and compilation process inside Docker, we have a standard workflow that delivers an artifact matching our target runtime environment!
In this example, we have the following source directory structure:
├── Dockerfile ├── requirements.txt ├── src │ ├── my_postgres_function.py
in which we want to include the Python Postgresql package psycopg2. This dependency is declared in the file
We can obtain the ready-to-deploy zip file for AWS Lambda, using the following standard Dockerfile:
FROM python:3.6 RUN apt-get update && apt-get install -y zip WORKDIR /lambda # Add the requiremts ADD requirements.txt /tmp RUN pip install --quiet -t /lambda -r /tmp/requirements.txt && find /lambda -type d | xargs chmod ugo+rx && find /lambda -type f | xargs chmod ugo+r # Add your source code ADD src/ /lambda/ RUN find /lambda -type d | xargs chmod ugo+rx && find /lambda -type f | xargs chmod ugo+r # compile the lot. RUN python -m compileall -q /lambda RUN zip --quiet -9r /lambda.zip . FROM scratch COPY --from=0 /lambda.zip /
To build your zip file, just type:
docker build -t my-lambda .
Getting the zip file out, takes a little bit of Docker foo:
ID=$(docker create my-lambda /bin/true) # create a container from the image docker cp $ID:/ . # copy the file from the /
By using the official Docker Python image and the multi-stage build functionality, we have a standard Dockerfile that we can use to build any AWS Lambda written in Python that includes the property binary dependencies.