Skip to content

Is this movie a thriller? Exposing a SageMaker Deep Learning model in an end-to-end serverless Web Application

Reading Time: 14 minutes

Links to the Github repo and to the website I have developed

The product

First things first. Before diving into the infrastructure, let’s check how the final web application looks like when in action.

Check it out at is-this-movie-a-thriller.com!

Pretty neat! Ok, time to have fun with AWS.

The idea

In this previous post, I have discussed how to define, train and deploy an image classifier in AWS SageMaker. The idea consisted in having a model capable of identifying whether a movie is a thriller/crime or not, by looking at its poster. I had concluded the tutorial by calling the SageMaker endpoint from a Jupyter notebook, testing it out on the 2005 movie Hostage, starring Bruce Willis. Everything worked pretty nicely. We had basically ended the discussion there, with an `InService` endpoint looking like the following.

The SageMaker endpoint we need to invoke

Now, the question is, how do we make this endpoint accessible to the world? Differently stated, how do we serve the model in production?

We expose the endpoint in a web application. Translated in plain English, this means building a website which gives users the possibility to upload a poster of a movie, and get back a model’s prediction. Let’s lay down a diagram of the project.

  1. Users hit is-this-movie-a-thriller.com.
  2. Amazon Route 53 redirects traffic to an S3 bucket containing a static website.
  3. Users upload the poster of a movie and ask the web application to infer the genre.
  4. This action triggers a POST request to API Gateway.
  5. API Gateway triggers AWS Lambda.
  6. Lambda calls Amazon SageMaker, invoking the endpoint and passing the image. The model replies back with a prediction and the information flows back from SageMaker to Lambda to API Gateway to Route 53 to the end users.

The plan

Now that we have a project’s structure let’s figure out an execution plan. We’ll implement the pipeline backwards. The backend first, the frontend last. I am not sure whether this is the recommended way of approaching this kind of projects, as it is one of the first ones I work on. Regardless, it seemed natural to me to address it this way and, eventually, it worked pretty well.

  1. Make sure we have the right permissions: IAM Role.
  2. We already have the SageMaker endpoint in place, so we’ll start implementing the previous step in the chain: Lambda.
  3. Once the backbone of Lambda ready, it is rather obvious to work on what triggers it: API Gateway.
  4. With SageMaker, Lambda and API Gateway up and running, our backend is ready.
  5. At this point, we can start focusing on the frontend. We’ll need a website (index.html plus javascript/CSS files). The website files are going to be stored in an S3 bucket.
  6. Eventually, we’ll buy a web domain on Route 53 and redirect traffic to the S3 bucket.

Creating an IAM Role with the right permissions

First things first. Nothing can happen within the AWS ecosystem without proper permissions. To make sure Lambda had the right privileges to access all the resources we need it to interact with, I created the `movies_thrillers-dev` IAM Role. Here how it looks like.

I experimented a lot with different AWS infrastructures. For each one of those, I tweaked a little bit the original role. This is why the policies attached to it look a bit messy. In the end, I made sure the role had Lambda and API Gateway full access (not completely sure this is actually needed; I have to test stricter permissions). As for `is-this-movie-a-thriller` and `movies_thrillers-dev`, those inline policies take care of SageMaker’s endpoint invocation and CloudWatch’s logging. Everything could be easily merged into a single JSON for compactness. Didn’t take the time to do that. Here the two separate custom policies for completeness.

###############################################
## is-this-movie-a-thriller
###############################################
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "sagemaker:InvokeEndpoint",
            "Resource": "sagemaker endpoint arn"
        }
    ]
}
###############################################
## movies_thrillers-dev
###############################################
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        }
    ]
}

Setting up Lambda

Now that we have a proper IAM Role we can go ahead with Lambda. Creating a function is very easy. Navigate to the Lambda console, select `Create function`, Author from scratch, pick a name ( movies_thrillers), `Python 3.6` as runtime, and `movies_thrillers-dev` as IAM Role.

New functions, by default, have no triggers and return a dummy JSON response. Below the code you will find within the IDE. Nothing exciting.

import json
def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

I will directly skip to the final code of my Lambda function, keeping all the testing and related thoughts to the API Gateway section. Here what I came up with (you can just replace the dummy code with the below).

import base64, os, boto3, ast, json 
endpoint = 'is-thriller-movie-ep--2019-02-21-18-30-19'
def format_response(message, status_code):
    return {
        'statusCode': str(status_code),
        'body': json.dumps(message),
        'headers': {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
            }
        }
def lambda_handler(event, context):
    try:
        body = json.loads(event['body'])
        image = base64.b64decode(body['data'].replace('data:image/png;base64,', '')) 
    
        runtime = boto3.Session().client(service_name='sagemaker-runtime', region_name='eu-west-1')
        response = runtime.invoke_endpoint(EndpointName=endpoint, ContentType='application/x-image', Body=image)
        probs = response['Body'].read().decode()
        probs = ast.literal_eval(probs)
        pred = probs.index(max(probs))
    
        if pred == 0:
            resp = 'This is not a thriller/crime movie! Not sure you will like it...'
        else:
            resp = 'This is a thriller/crime movie! Enjoy!'
    
        return format_response(resp, 200)
    except:
        return format_response('Ouch! Something went wrong on the server side! Sorry about that', 200)

The code is encapsulated into a try-except statement to avoid the function failing and returning nothing to the end user. In case of error, the frontend will show a message indicating that something went wrong server side. Bare in mind that in a real production environment you wouldn’t want to do that. What you want is to gracefully handle the exception, catch it and log it somewhere (CloudWatch does that automatically). This is not happening here. Whatever goes wrong in the `try` section, gets hidden by the exception part and the error is not recorded anywhere. I wanted to close the loop fast and I skipped this implementation detail.

This is what happens within the `try` statement:

  1. Line 17: Load `event[‘body’]` in JSON format. event is what API Gateway passes along with the POST request. It contains several fields. `body` is among those, carrying the contents of the request itself. In our case we’ll setup the frontend to send over a `body` in the following format: `{‘data’: ‘image encoded in base64 format’}`.
  2. Line 18: given what the frontend passes over, what we need to do is access `event[‘body’][‘data’]` and decode the image. This is what base64.b64decode is for. There is just one additional detail to consider. When encoding the image, javascript adds the string `’data:image/png;base64,’` as a prefix. Hence the body['data'].replace('data:image/png;base64,', '') to get rid of it.
  3. Lines 20-22: this is where we invoke the SageMaker endpoint, pinging the decoded image and getting a list of two probabilities (`probs`) back.
  4. Lines 24-30 take care of converting probs to a list of floats and finding the index of the highest element. This can be either `0` (not thriller/crime) or 1(thriller/crime). As soon as the index (pred) is found, the `resp` variable string is defined.
  5. Line 32 is of paramount importance. It calls the `format_response` function which wraps resp into the appropriate JSON format requested by API Gateway (recall we are sending the output of Lambda back to the frontend via API Gateway). Specifically, lines 8-10, i.e. `‘headers’: {‘Content-Type’: ‘application/json’, ‘Access-Control-Allow-Origin’: ‘*’}` are the trick which helped me solve an incredibly annoying CORS-related issue. Those headers, plus enabling CORS on the API Gateway console, actually.

What is CORS and why does it matter? CORS stands from Cross Origin Resource Sharing. This is a mechanism getting triggered as soon as two resources sitting on different domains try sending data to each other. The two resources are the frontend, i.e. the website sitting on http://is-this-movie-a-thriller.com, and the backend API Gateway, sitting on https://myapi.execute-api.eu-west-1.amazonaws.com. When the API returns the output of Lambda to the frontend, the browser checks whether this communication is allowed by looking at the headers of the resource being sent. Specifically, the header in question is `‘Access-Control-Allow-Origin’: ‘*’`. If this is missing, the browser blocks the incoming JSON. Initially, I had just enabled CORS (more details in the next section) in the API Gateway console. This turned out not to be sufficient though. The output of Lambda needs to explicitly contain that header too. This is the reason behind using the `format_response` function.

Setting up API Gateway

We have a Lambda function. Now we need something triggering it, so time to have fun with API Gateway. For illustration purposes, I have created a mock API (`blog_post_test_api`) and a mock Lambda function (blog_post_test_lambda – create this one first!), with the first triggering the second. Here a series of screenshots showcasing the entire process of API creation, Lambda integration, testing, and deployment. I have added notes to pictures’ captions.

1. This is the first step. Land on the API Gateway console and create a new API
2. Once the API generated, we need to create a new method to attach to it. Let’s go for POST.
3. Setting up the POST method. In here we define how we want the method to integrate with the existing AWS infrastructure. We want it to call Lambda, so we state the name of the function (previously created), we ask to use Proxy Integration and we save.
4. AWS realizes you are asking API Gateway to invoke Lambda so it prompts you about permissions. This is what we want, so hit OK.
5. Great! We have created the POST method. As you can see the console shows how information flows from the Client to Lambda and back.
6. (Screenshot from the Lambda console) As a result of creating the POST method and giving it permissions to invoke Lambda, AWS takes care of adding a trigger to our previously created function, `blog_post_test_lambda`. What you are looking at is the Lambda console (not the API Gateway one!), clearly displaying the relevant info about the API’s integration with the function. The first triggers the second.
7. Back to the API Gateway console. Before moving further make sure to test if the POST method is able to invoke Lambda correctly. To do that, click TEST. You will land on the screenshot below (8). You will just see the left side of it. Obviously, the right panel with the response, appears only after having actually clicked Test.
8. Pick a Request Body (I chose `{“data”: “test”}` and it does not really matter as Lambda is not parsing what we send for the time being) and hit Test. If everything goes as planned, the right side panel shows up with a bunch of useful info. The most important of which is the Response Body. It should read `”Hello from Lambda!”` as this is the default Lambda response. This is, of course, a very simple test, just to figure out if the integration works as expected. You can customize the Request Body and how Lambda handles it in more meaningful ways, and test more complicated scenarios using this same approach.
9. As stated previously, together with adding the `Access-Control-Allow-Origin’: ‘*’` header to Lambda’s output, we also need to Enable CORS within the API Gateway console. Hitting that option will prompt for confirmation to replace the relevant headers and then show the below output (a list with green check marks).
10. As you can see the main difference compared to before enabling CORS is the creation of the OPTIONS method, which appears within Resources on top of POST.
11. Always recall that whatever we do to our API, AWS does not take care of automatic deployment for us. So go and deploy it. Create a new stage (or select an existing one) and deploy the API onto it.
12. The result of the deployment is the creation of a new stage with an appropriate URL to invoke the API. That is the URL we are going to make calls to, so note it down!

How to test Lambda + API Gateway from localhost

There are multiple ways to test Lambda-API Gateway integration. We saw above how to do it via the API Gateway console. Other viable options are to use Postman or to invoke the API via the AWS Python SDK.

Last but not least, there is this nice trick I learned from this Medium post by Julien Simon. The idea is to fire up a PHP localhost server (so, yes, on your laptop) and ping the API from there. Here the commands. Open up a terminal and type

## THIS FIRES A LOCALHOST SERVER LISTENING ON PORT 8000
$ php -S localhost:8000
## OPEN UP A NEW TERMINAL TAB AND TYPE
$ export URL='THE URL OF YOUR API'
## WE SUBMIT THE POSTER FROM THE THRILLER/CRIME MOVIE HOSTAGE
$ export PIC='hostage.jpg'
## WE SEND THE IMAGE OVER TO THE API
$ (echo -n '{"data": "'; base64 $PIC; echo '"}') | curl -H "Content-Type: application/json" -d @- $URL
## IF EVERYTHING GOES AS PLANNED, YOU SHOULD SEE THIS OUTPUT ON SCREEN
"This is a thriller/crime movie! Enjoy!"

Nice, this works too!

Creating an S3 Bucket to host a website

We are all set with the backend, so let’s switch to the frontend. Now, let’s pretend that our entire website is just a dummy HTML page stored in a file called `index.html` which looks like this

<h1>Is this movie a thriller?</h1>

Where do I need to store this file to make it accessible to the world?

A Public S3 bucket would be a great starting point. As the name suggests, public buckets are buckets whose objects are made available to anyone. This is a critical condition to be met, as users need to be able to access the website. Before creating the bucket, there is one detail to be kept in mind. Later on, we will want to register a web domain on Route 53 and point it to S3. In order to achieve that, the name of the domain needs to match exactly the bucket name, i.e. we cannot buy `my-awesome-website.com` and point it to `my-awesome-website` S3 endpoint. The strings need to match exactly. As S3 buckets’ names are global and unique it would be really unfortunate to pay for a domain, then go to S3, try creating a bucket with the same name and figure out that it is already taken. You are better off doing the following instead:

  1. Navigate to the Route 53 console (more details in the Route 53 section below) and check whether the domain you want to register is still available for sale.
  2. If this is the case, navigate to the S3 console and check if you can create a bucket with the exact same name. If it is available, do it and then go back to Route 53 to buy your domain.

This is what I did, and created the `is-this-movie-a-thriller.com` S3 bucket.

To avoid having to manually make every object I would upload public, I edited the bucket policy as the following screenshot (and policy) show.

# S3 BUCKET POLICY TO AUTOMATICALLY MAKE PUBLIC ANY OBJECT WE UPLOAD INTO IT
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::is-this-movie-a-thriller.com/*"
        }
    ]
}

Last but not least, I edited the bucket’s properties, setting it as a static website host. As showed below, this means that navigating to the S3 endpoint would invoke the `index.html` file contained in it.

We are all set here. Now we need to focus on the actual `html`, `javascript` and `css` code to run the website.

Building the website

As a result of the previous section, we have a public S3 bucket hosting a dummy HTML page. Next step is make that page functional for our purposes. Here what we would like our website to do:

  1. Users select a poster image from their filesystem.
  2. The poster image gets uploaded and previewed in the frontend within a `canvas` HTML element.
  3. Users click a `Check if it’s a thriller` button which:
    1. gets the image from the canvas.
    2. converts the image into `base64` format (i.e. a string).
    3. sends the encoded image over to the backend API in a POST request.
  4. The backend responds back with a prediction, which gets displayed on the frontend within a `span` HTML element.

Let’s get to work. Here the files we need.

HTML: `index.html`

<!DOCTYPE html>
<html>
   <!-- Favicon -->
   <link rel="icon" href="favicon.ico" type="image/x-icon">
   <head>
      <h1>Is this movie a thriller?</h1>
      <link rel="stylesheet" href="styles.css">
      <link href='https://fonts.googleapis.com/css?family=Lato:400,700' rel='stylesheet' type='text/css'>
   </head>
   <body>
      <h3>Upload the poster and check it out!</h3>
      <div class="input-container">
         <input type="file" value="Choose a movie poster" id="inp" class="buttons" accept="image/*">
         </br></br>
         <canvas id="canvas"></canvas>
         </br></br>
         <input type="submit" value="Check if it's a thriller!" id="genre" class="buttons">
         </br></br>
         <span id="genreReturned"></span>
      </div>
      <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.0/jquery.min.js"></script>
      <script src="movies_scripts.js"></script>
   </body>
</html>

Javascript: `movies_scripts.js`

var API_ENDPOINT = "the API Gateway endpoint"
document.getElementById('inp').onchange = function(e) {
    var img = new Image();
    img.onload = draw;
    img.onerror = failed;
    img.src = URL.createObjectURL(this.files[0]);
  };
  function draw() {
    var canvas = document.getElementById('canvas');
    canvas.width = this.width;
    canvas.height = this.height;
    var ctx = canvas.getContext('2d');
    ctx.drawImage(this, 0,0);
  }
  function failed() {
    alert("The provided file couldn't be loaded as an Image media");
};
document.getElementById("genre").onclick = function(){
    var canvas = document.getElementById("canvas")
    var image = canvas.toDataURL()
	  var inputData = {"data": image};
    $.ajax({
      url: API_ENDPOINT,
      type: 'POST',
      crossDomain: true,
      data: JSON.stringify(inputData),
      dataType: 'json',
      contentType: "application/json",
      success: function (response) {
        document.getElementById("genreReturned").textContent = response;
      },
  });
}

CSS: `styles.css`

.buttons {
	border : solid 0px #e6b215;
	border-radius : 8px;
	moz-border-radius : 8px;
	font-size : 16px;
	color : #ffffff;
	padding : 5px 18px;
	background-color : #FF9900;
	cursor:pointer;
}
.buttons:hover {
	background-color:#ffc477;
}
.buttons:active {
	position:relative;
	top:1px;
}
span {
	font-size: 120%;
	font-weight: bold;
}
h1,
canvas,
h3 {
  text-align: center;
}
.input-container {
	text-align: center;
  }
body {
	font-family: 'Lato';
}

Putting everything together, the website looks like the following screenshot. Now we are only missing the last piece of the puzzle. Register a domain on Route 53 and redirect traffic from it to S3 (you might notice that in the below screenshot I am already using the is-this-movie-a-thriller.com domain and not the S3 endpoint).

Buy a domain on Route 53 and redirect traffic to S3

After everything we have accomplished so far, this is going to look like a stroll in a park. It all boils down to:

  1. Navigating to the Route 53 console.
  2. Registering (buying) a web domain.
  3. Pointing the domain to the S3 bucket with the static website.

Here the 3 steps with screenshots.

1. Navigate to the Route 53 console, to Registered domains and then hit `Register Domain`. As you can see within this panel, I already have the one I have bought for myself.
2. Type the domain you are interested in and click Check. In the case of is-this-movie-a-thriller, as I have already purchased the `.com` domain, it results Unavailable. If instead, the domain is available (and the S3 bucket name too!), go ahead and buy it. The flow is pretty straightforward. It will take some time from the moment you hit Buyto when all the relevant data is processed. AWS updates the console as soon as everything is done.
3. Now, navigate to `Hosted zones`, click on the domain you have just registered and then on Create Record Set. The right panel in the screenshot will show up. Leave Name empty and then click the `Alias Target` text box. The console displays a drop-down menu with all the available targets. The S3 endpoint of the bucket we created previously (matching the domain name) will appear there. Select it and click Create. Now you can open a browser, navigate to your brand new domain and Route 53 will redirect traffic to the bucket hosting the static website!

Testing the web application

We are at the end of our journey. We are only left with checking if everything fits in place as expected. Let’s test the website once again submitting the poster of the movie Hostage.

Great! This is working! That was quite a lot of work but we finally made it. We now have a fully functional end-to-end serverless web application exposing a SageMaker Deep Learning model to the world. Enjoy! 

Tags:

Discover more from

Subscribe now to keep reading and get access to the full archive.

Continue reading