Deploying a Serverless REST API (FaaS) with Python — Part 1

6 min readNov 7, 2020

How AWS Lambda, AWS CloudWatch, AWS API Gateway, and AWS RDS PostgreSQL integrate and work hard so that you don’t have to.

In this tutorial series, we will:

Create and connect to a AWS RDS PostgreSQL database instance
Set up new database tables and insert JSON data requested from an external API via a Jupyter notebook
Construct an IAM role with permissions to integrate AWS API Gateway, AWS RDS, AWS Lambda, and AWS CloudWatch
Run an environment for your Lambda functions by creating a Dockerfile which builds an image based on Amazon Linux and includes Python and Pip, installing any additional necessary packages such as Psycopg2
Create, package, and upload Lambda functions to AWS
Connect these functions with your AWS RDS and configure your environment variables
Integrate with AWS API Gateway so your function can be accessed as an endpoint
Trigger your tasks/functions with AWS CloudWatch rules
Bask in your sweat and glory

Prerequisites for this tutorial:

a knowledge of Python and SQL
an AWS account
a RDBMS tool like DBeaver

Let’s say you’re an amateur data engineer. You’ve completed several Python projects, deployed some applications using Flask and a service like Heroku, and you’re ready to get hot and heavy with AWS. Alright, maybe just dip your toes in. No one loves AWS that much.

So what now? How can AWS help you?

To illustrate, I’ll use a recent project as an example. We, a five-person team of data scientists and front-end web developers, created a product called PlanetData.Vision whose aim was to build an educational dashboard to teach middle school students the fundamentals of data visualization as well as earth science curriculum.When I began this remote cross-functional project in March, I took the usual route of creating a backend server with a simple web framework called Flask that could be accessed as a series of endpoints by our front-end developer.

Our GUI looked similar to this, sans pets.

Our project involved displaying a multitude of interactive data visualizations, but part of the MVP for this product was to incorporate real-time data, meaning that the database needed to be updated each day without any supervision. As I considered the scalability and oversight involved, I started to research more efficient ways to build a backend capable of dealing with CRUD operations in a scalable and cost-effective way. AWS Lambda was all of that.

Therefore, I began to investigate Amazon Web Services as a preferable method to create an API and transform Flask route functions into packaged code logic that could perform ETL without my supervision. AWS Lambda allows for serverless applications (or Functions-as-a-Service), meaning you only pay for the milliseconds when your code actually runs and your resources scale automatically rather than requiring the need to provision instances or manage servers. A small troop of self-sufficient functions like AWS Lambda, when connected with AWS API Gateway, allow for a serverless backend.

The joke goes that there is no such thing as **serverless**, just running an application on someone else’s server (cue laugh track).

Without further ado, let’s get our hands dirty with Amazon Web Services!

Step 1a — Create an AWS RDS PostgreSQL database instance, the easy way

Thanks to AWS’ new database creation flow, creating a db instance is quite straight-forward. Once logged into your account on AWS and in whatever region you would like to work, click the Services dropdown menu at the top left of the navigation bar.

On the menu, select RDS which is beneath the Databases category.

This will bring to all of your instances.

Click DB Instances (0/40), and then Create database.

You will then have the option to choose between Standard create and Easy create. As we want to make our database publicly accessible so that we can connect to it, we may as well use Standard Create. With every option you choose on an AWS form, you’ll notice the options afterward shift completely depending on how you answer the question before it, which can be stressful if you don’t really know what you want.

We, however, want a PostgreSQL database, a free one. By the end of each month, you’ll receive an alert from AWS letting you know you’ve used 85% of the free tier-eligible services.

After this, we’ll choose a DB instance identifier (our database name), as well as a username and passsword (if you forget to write these down now you’ll have another chance after the database is created to store these credentials). Then, under DB instance size, we’ll want to flip the switch on “Include previous generation classes” so that we can use the db.t2.micro instance. This has worked for my projects so far. Finally, under Connectivity, you’ll want to click the dropdown “Additional connectivity configuration” and under Public access select Yes. Now you’re good to go! Creating the database will take about 5 minutes.

Step 1b — Connect to your database

Depending on which RDBMS you use, connecting to your database can be rather confusing, especially since AWS doesn’t really lay it all out for you. Let’s go through it together. First, navigate to your database:

There, below Summary, you’ll see a bunch of tabs.

We’ll need information from Connectivity & security as well as Configuration.

Under Connectivity & security, you’ll see a column called Endpoint & port. You’ll need both of those. Then, you’ll go over to the Configuration tab, where you’ll want to grab the DB name and the Master username. Hopefully you stored your password! Now, let’s open up our RDBMS of choice, DBeaver.

We will place our Endpoint into the Host field of the form (your endpoint will probably be something like planetdatavision.zyxwvut.us-east-1.rds.amazonaws.com. Our Port goes into the Port field (might even have a default of 5432 already). For the Database field, place the DB Name we grabbed from the Configur-ation tab. Next, the Username we picked out when we created our database should go into this form. If you don’t remember it, it’s under Configuration and called Master username. And finally, the only truly obvious field, Password should contain the password that you hopefully remember.

And that’s it! You should be able to connect. Of course, you can go ahead and play around in DBeaver, but at the moment, we haven’t added any of the tables or data we need…