Using AWS Lambda with Docker
July 21, 2021
Problem:
My goal is to use Docker to deploy my AWS Lambda function. The reason I need to use Docker is because I need to import additional python libraries, I’m working on a Mac and I’ve received “invalid ELF header” log files at some point. This article written 4 years ago in 2017 by Liz Rice on freeCodeCamp was really helpful.
Solution:
In order to use that article as a resource, I needed the following knowledge: 1) have a working AWS Lambda function, 2) how to use the command line on a Mac and 3) have Docker installed.
Initial Docker Setup; Installing Dependencies
Here were the commands I ran to get this working within Docker. My project requires PyGithub and because I’m zipping files and using the AWS CLI, I had to install a few packages:
# 1) Create a Docker container
$ docker create -t -i ubuntu
# 2) Start the new Docker container, passing in the container ID
$ docker start -a -i <CONTAINER ID>
# 3) Install dependencies within the container
$ apt-get update
$ apt-get install zip -y
$ apt-get install python3-pip -y
$ apt-get install awscli -y
# 4) Create a working directory, to store all of my AWS Lambda files and dependencies:
$ mkdir working
$ cd working
# 5) Install my Python dependency here
$ pip install -t . PyGithub
Great! So at this point, I’ve installed all of my required dependencies within my new Docker container.
Deploy updated code from local Github Repository
Next I ran these commands to copy my files from my local Github repo (containing the AWS Lambda code) to my Docker container:
# Access directory where all of the AWS files are loaded; copy files to Docker container
$ cd <LOCAL GITHUB REPO>
$ docker cp . <CONTAINER ID>:/working
# Start Docker container; zip files
$ docker start -a -i <CONTAINER ID>
$ cd working
$ zip -r linux-lambda.zip .
# Run this command, passing in my function name and the actual file
$ aws lambda update-function-code --function-name <AWS LAMBDA FUNCTION NAME> --zip-file fileb://linux-lambda.zip
To remove the manual steps, I ended up created a local bash script (calling it “update-aws.sh”) to execute the commands from my Desktop. In turn, anytime I make changes to my repo, I can execute this script and it’ll deploy for me. Here are the commands within the bash script:
# Access local repository
cd <LOCAL GITHUB REPO>
# Copy files from local repository to designated Docker container
docker cp . <CONTAINER ID>:/working
# Start Docker Container; execute script to upload files to AWS
docker start <CONTAINER ID>
docker exec -it <CONTAINER ID> bash -c "cd working && zip -r linux-lambda.zip . && aws lambda update-function-code --function-name <AWS LAMBDA FUNCTION NAME> --zip-file fileb://linux-lambda.zip"
In short, if I make changes to my local repo, I can simply run this bash script:
./update-aws.sh
And that’s how I have setup my environment, to make it easier to deploy changes to my AWS Lambda function on a Mac (which uses additional Python packages).
Sources
- Introduction - PyGithub 1.55 (n.d.) https://pygithub.readthedocs.io/en/latest/introduction.html
- Deploy Python Lambda functions with .zip file archives (n.d.). https://docs.aws.amazon.com/lambda/latest/dg/python-package.html
- Tutorial: Creating a Lambda function in Python 3.8. (n.d.). https://docs.aws.amazon.com/lambda/latest/dg/python-package-create.html
- Using AWS Lambda environment variables. (n.d.). https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html
- Rice, L. (2017, February 17) How to claw your way out of AWS Lambda function hell using the power of Docker. freeCodeCamp. https://www.freecodecamp.org/news/escaping-lambda-function-hell-using-docker-40b187ec1e48/
- Sagar (2012, September 4) How do I create a folder in a GitHub repository StackOverflow https://stackoverflow.com/questions/12258399/how-do-i-create-a-folder-in-a-github-repository
- Williams, J. (2020, December 25) Nightly GitHub Contribution Stats with PyGitHub and AWS Lambda. Dev.to https://dev.to/jameson/nightly-github-contribution-stats-with-pygithub-and-aws-lambda-8k