When you develop an AWS Lambda function in Python, you may require packages which include binary libraries. In this blog we will show you how to use the official Docker Python image to make sure you have a working Lambda.
If you are developing your code on a Mac or Windows operating system, it is pretty hard to install the correct Python packages that your code depends on. There are even github repositories with pre-compiled binaries, which blindly copy into your zip file. This is an unwanted process, as it deviates from the standard way of developing and installing Python software.
Luckily Docker provides us with an official Python image based upon 64-bit Linux, which matches the AWS Lambda runtime. By running the installation and compilation process inside Docker, we have a standard workflow that delivers an artifact matching our target runtime environment!
In this example, we have the following source directory structure:
├── requirements.txt ├── src │ ├── my_postgres_function.py ├── target │ └── content
in which we want to include the Python Postgresql package psycopg2.
This dependency is declared in the file
To create a ready-to-run zip file for AWS Lambda, we first clear the target directory:
rm -rf target/content mkdir -p target/content
Then run a
pip install inside the container based upon the official Python 2.7 image:
cp requirements.txt target/content docker run --volume $PWD/target/content:/venv \ --workdir /venv python:2.7 \ pip install --quiet -t /venv -r requirements.txt
Next, we can add and compile all the sources:
cp -r src/* target/content docker run --volume $PWD/target/content:/venv \ --workdir /venv python:2.7 \ python -m compileall -q -f .
Now all the sources are added and dependencies are installed, we need to ensure that all sources and directories are readable:
find target/content -type d | xargs chmod ugo+rx find target/content -type f | xargs chmod ugo+r
Finally, we zip the result to upload to an s3 bucket for deployment.
cd target/content && zip --quiet -9r ../../target/$NAME-$VERSION.zip * chmod ugo+r target/$NAME-$VERSION.zip
By using the official Docker Python image, we can create functions which include the proper binaries for the AWS Lambda run-time.