Skip to main content

It’s hard to believe that Lambda function have already been around only for the last 5 years (at the time of writing this), and it became so popular so quickly.

deploying Serverless services

I’ve been working as a DevOps engineer for the last 8 years or so (working on AWS for the last 7 years), and I wanted to review some of the tools I came across for deploying serverless solutions on AWS.

I. First wave: bash script

It works! And it still does, although it’s not very scalable, it’s mainly (as I still does today) for installing dependencies, testing and packaging them into zip.

One example of bash script code I keep re-using is this one

#!/usr/bin/env bash # Create python packages for lambda #title #description :This script will install pip packages, run pylint and pytest and finally zip python scripts #author :Oli #date :07/01/2020 #version :0.1 #bash_version :3.2.57(1)-release #=================================================================================== set -o errexit set -o pipefail set -o nounset function install_packages() { pip install -r requirements_test.txt } function zip_python() { pushd sns-guardduty-slack/ zip -r ../ popd mv ../ } function test_python() { pushd sns-guardduty-slack/ pylint popd } install_packages test_python zip_python
Code language: PHP (php)

This is working quite well for the first part (Continuous Integration) but doesn’t fit the second part (Continuous Delivery). Back in these days, I started using CloudFormation for CD requirements, let’s see an example.

II. Cloudformation Era

Around 2016, a Cloudformation allowed the use of Yaml format for declaring AWS resources, that was a massive change specially for the sake of my eyes

{ "Type" : "AWS::Lambda::Function", "Properties" : { "Code" : Code, "DeadLetterConfig" : DeadLetterConfig, "Description" : String, "Environment" : Environment, "FunctionName" : String, "Handler" : String, "KmsKeyArn" : String, "Layers" : [ String, ... ], "MemorySize" : Integer, "ReservedConcurrentExecutions" : Integer, "Role" : String, "Runtime" : String, "Tags" : [ Tag, ... ], "Timeout" : Integer, "TracingConfig" : TracingConfig, "VpcConfig" : VpcConfig } }
Code language: JSON / JSON with Comments (json)

From this to this:

Type: AWS::Lambda::Function Properties: Code: Code DeadLetterConfig: DeadLetterConfig Description: String Environment: Environment FunctionName: String Handler: String KmsKeyArn: String Layers: - String MemorySize: Integer ReservedConcurrentExecutions: Integer Role: String Runtime: String Tags: - Tag Timeout: Integer TracingConfig: TracingConfig VpcConfig: VpcConfig
Code language: JavaScript (javascript)

Declaring AWS Lambda functions for Operational purposes, Cloudformation was indeed doing the job. Unfortunately, it can become rather complexe if you start adding API Gateway / SNS Topics and so on to the equations:

GatewayRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: GatewayRolePolicy PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - sns:Publish Resource: "*" - Effect: Allow Action: - logs:PutLogEvents - logs:CreateLogGroup - logs:CreateLogStream Resource: "*" ApiGatewayRestApi: Type: AWS::ApiGateway::RestApi Properties: Name: API_GW Fn::Join: - "" - - Ref: AWS::StackName - "-api" ApiGatewayGETMethod: Type: AWS::ApiGateway::Method Properties: AuthorizationType: NONE HttpMethod: GET RequestParameters: method.request.querystring.message: false method.request.querystring.subject: false method.request.querystring.topic: false Integration: Type: AWS Credentials: Fn::GetAtt: [ GatewayRole, Arn ] #use the already defined role Uri: #required URI for using SNS service Fn::Join: - "" - - "arn:aws:apigateway:" - Ref: AWS::Region - ":sns:action/Publish" ...................
Code language: PHP (php)

This is a truncated example of an API Gateway declaration in Cloudformation format. Let’s see Terraform way

III. Does Terraform make it better?

Depending on who you ask, I believe it slightly does but not that much:

AWS: aws_api_gateway_integration – Terraform by HashiCorpProvides an HTTP Method Integration for an API Gateway

Based on my experience, I had trouble deploying AWS Lambda functions with Terraform, especially around Stages and Testings which would see later. Terraform seems to have issues picking up the latest zip file containing the source code (even when build locally), I had to ran it

resource "aws_lambda_function" "sns_slack_lambda" { filename = var.filename function_name = var.lambda_function_name role = aws_iam_role.sns_slack_lambda_role.arn handler = "sns_guardduty_slack.lambda_handler" source_code_hash = base64sha256(var.filename) runtime = "python3.7" environment { variables = { SLACK_CHANNEL = data.aws_ssm_parameter.slack_channel.value HOOK_URL = data.aws_ssm_parameter.slack_incoming_webhook.value } } }
Code language: JavaScript (javascript)


Around 2018, AWS introduced AWS Serverless Application Model to answer the needs for managing Serverless deployments and the management of them. SAM is simply an extension of CloudFormation, is also using Yaml format and the installation process is as simple as AWS CLI one:

pip install aws-sam-cli

SAM CLI is then available as sam. Once completed, let’s crack on the first step: Package

sam package \
--template-file template.yml \
--output-template-file package.yml \
--s3-bucket my-bucket

This command is to upload any artifacts to an AWS S3 bucket, generate a package AWS SAM template file that’s ready to be used.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
sam-appSample SAM Template for sam-appGlobals:
Timeout: 3Resources:
Type: AWS::Serverless::Function
CodeUri: hello_world/
Handler: app.lambda_handler
Runtime: python3.7
Type: Api
Path: /hello
Method: getOutputs:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}"
Description: "Hello World Lambda Function ARN"
Value: !GetAtt HelloWorldFunction.Arn
Description: "Implicit IAM Role created for Hello World function"
Value: !GetAtt HelloWorldFunctionRole.Arn

Here is a template example of an hello world application deploy with AWS SAM, the syntax is a mixed between Cloudformation template and addons AWS SAM, I personally found this confusing. Once your package is ready with the package command, you can deploy by using the following command:

sam deploy \
--template-file ./packaged.yaml
--stack-name mystack
--capabilities CAPABILITY_IAM

AWS SAM is helping solving the previous mentionned issues with Cloudformation and Terraform, this tool helps for building scaffolding for Lambda / API Gateway / SNS and so on, it also helps for performing integration testings locally before pushing your code to production. After using AWS SAM, I also discovered that this tool would inherited problems from Cloudformation such as modifying records in Route53. If you compare AWS SAM with Serverless (which we will see next), you will quickly switch to Serverless as I did.

V. Serverless Framework

Serverless framework managed to solve most of these issues:

Installing this tool require Node.js installed, simply install that tool that way:

npm install -g serverless

Packing and deploying by only using a single command line:

sls deploy

After using this tool extensively, I’m no longer having issue managing API Gateway / SNS / Lambda and link them automagically together, deploying is really easy and fast.

VI. My take on deploying Serverless services

My main recommendation would be to jump straight to Serverless framework and avoid investing too much of your time on Cloudformation / Terraform for deploying a whole set of serverless services together.