In this post, I will go through the process of using an AWS Lambda function served via AWS API Gateway HTTPS endpoint. My existing NodeJS API performs one task and is not called too often which is a perfect candidate for an AWS Lambda function, where I can possibly reduce server resources and cost.
The Existing API
This specific API that I used in a contact form allows visitors to submit their enquiry to the site owner and send it via email. Since it only performs one task and is not used often, it is a perfect opportunity to move it into an AWS Lambda function instead of running it 24/7 idle most of the time.
The website is written in Angular which is mostly static files and the contact form section calls a small NodeJS API for sending emails. Although both applications are very efficient (nodejs + nginx), I decided to give AWS Lambda a try and remove the existing NodeJS API entirely.
AWS Lambda Function
To create a new Lambda Function, go to the AWS Lambda dashboard, click Create function, then click Author from scratch.
Since the original server is written in NodeJS, I just copy and paste the code into the AWS Lambda Function for NodeJs v6.10 and adjust some of the parameters. The packaging involves zipping the whole project including the node_modules directory and upload it to the AWS Lambda console. I’ll get into the details later in this post.
The index.js code looks like below:
function sendEnquiry(data, callback) {
// Codes to send the enquiry
}
function validData(body) {
// Code that validates and return the input data
return data;
}
function createResponse(statusCode, data) {
return {
isBase64Encoded: false,
statusCode: statusCode,
headers: {
"Content-Type": "application/json",
"Access-Control-Allow-Headers": 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with',
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": 'POST,GET,OPTIONS'
},
body: JSON.stringify(data)
};
}
exports.handler = function (event, context, callback) {
var data = validData(JSON.parse(event.body));
if (data.error) {
callback.call(null, null, createResponse(400, data));
} else {
sendEnquiry(data.data, function () {
callback.call(null, null, createResponse(200, data));
});
}
};
AWS Lambda Code Packaging and Deployment
I haven’t explored the AWS command line tool for deploying Lambda Function codes, therefore, I will only show how to upload if in the AWS Lambda Console as a zip file.
Project structure:
sendEnquiry/ -> build.sh -> index.js -> package.json -> node_modules/
Note that when zipping the file, it must not contain the project directory. In Linux, we can do something like this:
zip -r sendEnquiry.zip index.js package.json node_modules
When you try to extract this, it will extract the contents, but without the sendEnquiry directory. This is important so that your index.js file will be located by the AWS Lambda process. Also note that the name of the zip file must be the same name as the Lambda Function name, in my case, sendEnquiry.zip.
I have created a very simple deployment script to automate the process. It will upload the package into an AWS S3 bucket to simplify the process and also overcome the 10MB upload limit by AWS Lambda upload zip process. The upload script uses the pre-configured s3cmd tool to upload files into my S3 buckets.
#!/bin/bash SOURCE_DIR=$(pwd) DEPLOY_TEMP_DIR=/tmp/deploy-lambda-enquiry # Prepare deploy dir, sorry for some kung-fu mkdir -p $DEPLOY_TEMP_DIR rm -rf $DEPLOY_TEMP_DIR mkdir -p $DEPLOY_TEMP_DIR # Copy source files (will not include dot files, which we don't need) cp -r $SOURCE_DIR/* $DEPLOY_TEMP_DIR/ # Zip it cd $DEPLOY_TEMP_DIR zip -r sendEnquiry.zip index.js package.json node_modules # Push to S3 s3cmd put sendEnquiry.zip s3://my-lambda-functions/deploys/sendEnquiry.zip cd $SOURCE_DIR echo 'Done'
Login to the AWS Lambda Console and upload a zip file via AWS S3, then provide the AWS S3 HTTPS link to the zip file.
API Gateway Integration
We can actually call the AWS Lambda Function directly via the AWS SDK. However, in doing so, we may need to create yet another API server to call the Lambda Function, or call the Lambda Function directly in the browser which could potentially expose our AWS authorization keys. Not cool! Therefore, we will expose the Lambda Function using the API Gateway service from Amazon.
Within the AWS Lambda Function console, click on the Triggers tab to select how we call the Lambda function. Select API Gateway from the list and configure the service name. Be sure you click the Enter value button so that you can customize the service name. For security, I choose open to allow website visitors to call the API. If you need authentication, you may choose AWS IAM or an access key which is configured in the API Gateway dashboard.
We may encounter CORS issues with our new API, therefore, we need to enable CORS in our API. Go to the API Gateway dashboard and click the service we’ve just created. Click the sendEnquiry method, then click Actions -> Enable CORS. Just accept the defaults and proceed.
After configuring the API resource, be sure to deploy the api using the default prod stage.
API Gateway with Lambda Proxy Integration
The steps that we did above will configure the API Gateway as proxy to the Lambda function. Unlike the regular Lambda integration, the Lambda proxy integration will send the original HTTP request to the Lambda function handler as part of the event parameter. This way, we can read data from the request just like a regular API do.
exports.handler = function (event, context, callback) {
// We can read headers
console.log(event.headers);
// And the request body as well if it contains a JSON payload
console.log(JSON.parse(event.body));
callback.call(null, 'Error: done nothing...');
};
Lambda proxy integration also requires that a certain data format must be passed as part of the success callback. The callback syntax looks like below:
callback.call(null, optionalError, optionalSuccessResult);
// Error
callback.call(null, 'An error occured');
// Success
callback.call(null, null, { foo: "bar" });
For the Lambda proxy integration, the result format must be:
{
isBase64Encoded: false,
statusCode: 200,
headers: {
"Content-Type": "application/json",
"Access-Control-Allow-Headers": "Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "POST,GET,OPTIONS"
},
body: "string body here..."
}
Note: When passing a JSON body, stringify it first.
Also note that we also added some CORS headers. This is due to the error encountered when calling the API where CORS headers are not found on the response. The CORS enabled in API Gateway will only respond to the OPTIONS method and will send a mock response. However, calling the actual API will not contain the CORS headers unless we specify them ourselves. We can add these headers in the API Gateway integration settings, however, I choose to manually add it in the NodeJS handler instead.
The object returned in the AWS Lambda must be in the specified format as above, therefore, we have the following code in our handler.
function createResponse(statusCode, data) {
return {
isBase64Encoded: false,
statusCode: statusCode,
headers: {
"Content-Type": "application/json",
"Access-Control-Allow-Headers": 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with',
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": 'POST,GET,OPTIONS'
},
body: JSON.stringify(data)
};
}
exports.handler = function (event, context, callback) {
var data = validData(JSON.parse(event.body));
if (data.error) {
callback.call(null, null, createResponse(400, data));
} else {
sendEnquiry(data.data, function () {
callback.call(null, null, createResponse(200, data));
});
}
};
Returning an incompatible format will result in Error 502 since the API Gateway integration expects the format to be followed. Notice that, even in the error condition, we don’t pass an error object. This is because it will still cause a Error 502 due to the incorrect format. Therefore, we call the success callback but set the appropriate HTTP status code to indicate error, like Error 400.
Testing the API
API Gateway has a testing tool within the dashboard to mimic an HTTP request against the API. You can use that tool to test if the integration is working. You can view the result and logs within the testing tool.
For testing the actual HTTPS endpoint, we can try to use curl or postman. In my example service, the API call will look like this:
POST https://1sfdr3wz46.execute-api.us-west-2.amazonaws.com/prod/sendEnquiry
{
"name": "Guest",
"email": "guest@example.com",
"subject": "I have a question",
"message": "I need help with your open source library..."
}
You can debug the API by looking at the CloudWatch logs within the AWS Lambda function monitoring tab.
Cost
Since I have replaced my old API with AWS Lambda, I have disabled the old NodeJS API, therefore, reducing the server resource consumed. The current server is running multiple applications and killing one app will make room for new apps.
AWS Lambda functions can be called up to 1 million requests per month for free and my website won’t probably reach 1 million requests in a year so I take it as a free Lambda service for life for me.
However, the AWS API Gateway will charge me 3.5x USD per 1 million requests, so I have to take a closer look at it. I may not reach 1 million requests a month so cost could be a few cents. All good then.
That’s it!