Creating An AWS Lambda Package for Python
If you would like to see the TLDR; version of this article please click TLDR;
AWS Lambda is arguably one of the most flexible and versatile services offered by AWS. Lambda is a managed containerize service that will allow you to execute your custom application code for tasks as simple as automating daily routines or being part of larger and more complex application infrastructure servicing thousands of requests per second.
A Lambda function default configuration is designed to be very light weight. While this can be a plus for many reasons, it is not always helpful when a required library for the application is not available at run time. Thankfully AWS provides a method to include extra packages with our application code by building a Lambda package.
What Is A Lambda Package?
A lambda package is simply a compressed file that will include all the necessary files to execute your application correctly. This can include third party libraries installed by the pip package manager. The Lambda package must built so the root of the application files will extract directly to the root of the execution folder. Once your application is unpacked in Lambda, the application files will reside and be executed from /var/task.
While it is possible to build a some lambda package in Windows, many packages will require the use of third party dependencies that may not be compatible with windows. It is recommended to use the AWS Linux AMI or a Cloud 9 environment to build your lambda packages.
It is recommended to use the AWS Linux AMI or a Cloud 9 environment to build your lambda packages.
Building The Lambda Package.
For this example, we are going to use the third party Python library called pymysql. This package is used as a MySQL client used to connect to MySQL databases.
To continue you will Python and pip installed on your local machine. For Linux you will need the zip command line utility installed. To compresse the package in Windows you will need 7zip or another third party compression utility. While using the default compression utility in Windows may work, I often hear from users that this utility caused issues.
Let’s start by taking a look at the following commands one line at a time.
We start by creating a new working folder for our lambda package.
mkdir lambda_package cd lambda_package
Next, we use the pip package manager to install the package into our working folder. The “-t” argument passed to pip,instructs pip to install the package in the “Target” location instead of at a system location
pip install pymysql -t ./
Now we can copy the application code into the root of this working directory. If the custom application file was named lambda_function.py, you would place this directly in the lambda_package directory. The directory structure should look like this.
lambda_package pymysql pyMySQL-0.9.3.dist-info lambda_function.py
Now it is time to compress these files into our Lambda package. This is where things will become a little different for Windows and Linux/Mac users.
For Linux/Mac, the zip command can be used. Please note the “.” at the end of this command. The “.” is used as a wild card for the zip command.
zip -r /tmp/lambda_package.zip .
For Windows, navigate file explorer to the root of the lambda_package folder. Select the directories pymysql, pyMySQL-0.9.3.dist-info as well as the file lambda_function.py. Right click this selection and choose “7-zip -> add to lambda_function.zip”. A new zip file will be created in the lambda_function directory.
Updating the Lambda Function Code
Now that our Lambda function package has been created, all that is left is to update the lambda function code and provide AWS with the new package to execute. Navigate to the lambda function in the Lambda portion of the AWS Web console. Below the function configuration section, you will find the Function code section. Below the heading “Code Entry Type”, select “upload a zip file”.
Follow the prompts to select the lambda_function.zip file previously created. Once you have selected this file, press the save button at the top of the screen.
The lambda function page will fresh and the application function code will display an updated copy of code uploaded if the total size of the lambda package is less than 3MB
Now that you can build lambda packages for Python, the possibilities are endless. Get out there and start building your serverless applications!
Coming soon I will explore the use of this package to read and write data stored in your RDS MySQL Server. Stay tuned!