Introduction

CloudGTO is a platform that aims to help developers of any experience level accelerate building and deploying applications using the Serverless Framework. It is still in active development phase but public beta access is available. This is an excellent opportunity to check out CloudGTO and provide some feedback and suggestions to the team!

The Service Builder uses a wizard-like interface to help build out the desired infrastructure. It supports common Serverless patterns for REST APIs. It allows customizations, such as choosing to enable Tracing or not, and selecting the appropriate memory settings for Lambda Functions, to name a few. The builder is flexible and allows additional resources to be created as well.

The Service Builder generates a downloadable 'serverless.zip' file which contains the IaC templates and Lambda Function code for performing CRUD operations against DynamoDB tables. The project structure is organized in an opinionated manner by incorporating best practices and leaning into the experience of Serverless developers who work extensively with the technology.

Building Meal Prep Service REST APIs

In this blog post, we are going to use CloudGTO to help build REST APIs for a Meal Prep Service.

In the following architecture diagram, we configure our API Routes and Methods in API Gateway and use Lambda Proxy Integration to invoke the appropriate Lambda Functions based on the request.

We create a Cognito User Pool to secure the Order Management APIs (Orders and Order Line Items). We only allows users with a valid Cognito Identity to place orders.

We will store the Orders in a DynamoDB table using the Single Table Design model. For our example, an Order can have 1 or more Line Items. Each Line Item will include the meal ID selected, quantity, unit price and sub total. When an order is checked out, we calculate the total price due by adding the sub total for the line items.

To highlight the flexibility offered by CloudGTO, we create an additional DynamoDB table to store our Menu. Menu items simply include the meal ID, price, seller and category. Menu items can be viewed by Seller or by Category (pork/chicken/seafood/vegetarian/beef).

Notice that we opted to keep the Menu APIs available and open to everyone. This is because we want potential customers to see the options we have to offer.

If you wish to follow along with the example, you will need the following:

• AWS Account
• AWS Credentials setup on your machine
• Access to CloudGTO

CloudGTO Observations/Takeaways

I would like to share my experience and some of the things I learned using CloudGTO to build the Meal Prep REST API Services.

We will discuss the REST API Blueprints currently provided and share some thoughts on how we decided to go with the CloudGTO recommended design.

We also share how CloudGTO features helped us adhere to AWS Best Practices.

CloudGTO Promotes Adoption of Proven Serverless Patterns

CloudGTO currently supports the 3 most common patterns (aka Blueprints) for building Serverless REST APIs.

Monolithic

The Monolithic Lambda pattern utilizes a single Lambda Function to support the different HTTP methods allowed by the 'ANY' method. Because we are allowing all possible HTTP methods, the Lambda Function will need to perform a lot more work to determine which method was requested and whether it is a supported method. The Lambda Function also gets significantly larger in size as it has to include all the code for all the methods. These are factors that can impact a function’s overall performance, deployment speed, maintainability, observability, cost, etc. Least Privilege Security is also harder to achieve as the Lambda Function will need full read/write access to the DynamoDB table. Troubleshooting also gets a lot more cumbersome.

All that being said, this is still a valid pattern that can be considered to support migrations from Monolithic APIs to Microservices architecture.

Single Lambda Many Routes

The Single Lambda Many Routes pattern breaks up the different HTTP methods into their own routes. This provides a bit more flexibility as each route can be managed at a more granular level in API Gateway. For example, we can choose to secure certain routes via Cognito, while leaving others open and available. A single Lambda Function supports all the available API routes, but we at least have a well defined idea on what is supported. In this example, we know for sure that only GET/POST/PUT methods are supported by the '/items' route. Most of the disadvantages of the Monolithic approach also apply to this pattern.

Single Lambda Per Route

This is the recommended pattern for building Serverless REST APIs, which is what we picked when we built our service.

This choice allowed us to adhere to security best practices. Because each Lambda has a single responsibility, implementing least privilege security is simplified.

Keeping functions lean and simple makes code easier to maintain. Lean functions generate smaller packages which can speed up deployments and improve overall performance. Cold start latencies can be reduced as well.

Scalability and observability can be as granular as we need them to be. This helps provide insights on how our services are doing and find optimization opportunities.

Comparison Chart for the Patterns

Here is a summary of the what we just talked about and the pillars we used to drive our decision:

CloudGTO Simplifies Organization and Management of Cloud Resources

Naming Conventions

While building our service, we noticed that inputs for resource names are restricted to certain lengths. This is intended to allow CloudGTO to add prefixes or suffixes to the provided base name. Useful information like Region, Stage and Resource Type are typically good tokens to easily identify what a resource is for.

Isolated Stack Management

CloudGTO uses Serverless Compose to manage the infrastructure. A CFN Stack is created for each group of resources (1 stack for each 'serverless.yml' file).

  
meal-prep-service/
├── resources
│   ├── api
│   │   └── serverless.yml       --> 1 serverless.yml
│   ├── cognito
│   │   ├── email.yml
│   │   └── serverless.yml       --> 2 serverless.yml
│   └── dynamodb
│       ├── mealOrders
│       │   └── serverless.yml   --> 3 serverless.yml
│       └── mealsMenu
│           └── serverless.yml   --> 4 serverless.yml 
├── scripts
│   └── auth.bash
├── serverless-compose.yml
├── services
│   └── meal-prep
│       ├── serverless.yml       --> 5 serverless.yml
  

It created 5 CFN stacks as shown in the image below.

Infrastructure components that tend to remain static (infrequent updates), and do not have code, are grouped in the 'resources/' folder. Another advantage of managing individual CFN stacks is reducing the risk of accidental updates to unintended resources.

For example, suppose we have a single stack that includes both DynamoDB and Lambda Functions. The Lambda Functions will be updated more frequently while the DynamoDB table most likely remains unchanged after it is created. There are times when deployment errors occur in CloudFormation, and unfortunately, the easiest and simplest way to address this issue is to delete and recreate the stack. In our example scenario, we would risk losing data if we do choose, to delete the stack. In my opinion, having Monolithic CFN Stacks complicates resource management, leads to tight coupling, causes headaches and reduces the options we have for fixing deployment issues quickly.

  
resources
├── api
│   └── serverless.yml
├── cognito
│   ├── email.yml
│   └── serverless.yml
└── dynamodb
    ├── mealOrders
    │   └── serverless.yml
    └── mealsMenu
        └── serverless.yml
  

Every approach has its own pros and cons. What we like about managing isolated stacks is how much it simplifies everything and reduces possible mistakes during manual stack management tasks. This helps simplify and speed up deployments as well.

CloudGTO Emphasizes Least Privilege Security Principle

The Service Builder allows us to customize permissions to be granted to our Lambda Function. Because we opted for the recommended blueprint for REST APIs, we can be as granular as we need when assigning permissions. This allows us to apply the principle of least privilege security in our environment.

The image below shows how we can apply the minimum required permissions for the Lambda Execution Role in CloudGTO:

To further illustrate, let’s focus on the operations implemented by the Orders API. Each Lambda Function is responsible for a single task only. As a result, we only need to grant each function the minimum permissions required to perform its job.

'createOrder' function creates a new Order. We only grant it 'create' permissions against the table.

'getOrder' function retrieves an Order from the Orders table. We grant it 'read-only' permissions against the table.

'updateOrder' function updates details for an Order. We only grant 'update' permissions against the table.

'deleteOrder' function deletes an Order. We only grant it 'delete' permissions against the table.

CloudGTO Alleviates Repetitive, Tedious and Boilerplate Tasks from Developers

Dev Tools Configuration

CloudGTO sets up common configurations for Development Tools such as ESLint, Prettier, Webpack. These usually remain static between projects. Instead of having to copy/paste reusable configuration files from old to new projects, CloudGTO takes care of this for us!

Auto Generated Starter Code for Lambda Functions

CloudGTO generates code that performs simple CRUD operations against DynamoDB. The code can be deployed as is if we do not have additional logic or requirements to implement. We can also modify the code to meet our needs, which is what we did for our sample project. Because there is already code to start, it speeds up the development process tremendously. It’s always easier to work off of something that having to create everything from scratch.

Auto Generated Unit Tests

CloudGTO also generates 'Jest' unit tests based on the starter code generated. Of course, as we make code changes, the tests will need updated. But again, it is nice to already have the scaffolding and configurations in place.

The 'events/' folder provides event templates that are used for testing.

  
events
├── createOrder.json
├── createOrderLine.json
├── deleteOrder.json
├── deleteOrderLine.json
├── getMeals.json
├── getMealsByCategory.json
├── getMealsBySeller.json
├── getOrder.json
├── getOrderLines.json
├── updateOrder.json
└── updateOrderLine.json
  

Solution Walkthrough

CloudGTO is flexible and allows us to extend its opinionated approach with the ability to add additional resources to support our application.

In our meal prep service, we created an additional DynamoDB table to store Menu Items.

The image shows additional resource types that can be added to the infrastructure:

When providing the Table Name, you will notice that it restricts the length of the base name you define so it can prefix/suffix it with useful metadata details.

The Builder also allows us to select our Billing Mode. By default, it is 'PAY PER REQUEST'.

CloudGTO supports both Simple and Composite Primary Keys.

We can also add Global Secondary Indexes to support additional access patterns. GSIs also support both Simple and Composite Primary Keys. The GSIs inherit the table’s Billing Method.

We also added additional Lambda functions that will perform CRUD operations for the Menu DynamoDB table.

We can customize the Runtime, Memory Configuration, Timeout, and Tracing (default is OFF) options as shown in the image:

Set the runtime, memory, and timeout. Optionally, you may choose to enable Tracing for your function

CloudGTO provides a summary of all the resources that will become part of our infrastructure.

We set up the routes and methods for our REST APIs. The default integration type is Lambda Proxy. It also infers that we want to use the Cognito User Pool to secure our endpoints. In our example, we want both existing and new customers to see the available meal options. As a result, we opted not to secure the '/menu' endpoints with Cognito. However, in order to place orders, users must have a valid Cognito identity.

Code Changes for Customizing Generated Code

I wanted to highlight a few of the code changes I had to make in order to make the generated code work for my business logic.

Some of these changes were driven by CloudGTO limitations. I’ve shared feedback with the CloudGTO team so these may become native functionality in the future.

DynamoDB Query by Primary Key is the Default Behavior

CloudGTO generates code for CRUD operations with the assumption that we will query the DynamoDB table by primary key. However, in our example, we want to return the complete list of menu items so we will need to perform a scan operation.

The following code snippet shows the original generated code by CloudGTO:

  
const handler = async (event) => {
    try {
        const id = event.pathParameters?.id;

        if (!id) {
            throw { statusCode: 400, message: "invalid param" };
        }

        const keySchema = {"PK":"mealId"};

        let Item = {
            [keySchema.PK]: id
        };

        const ddbRes = await getItem(TableName, Item);

        if (!ddbRes.Item)
            throw {
                statusCode: 400,
                message: "Item not found"
            };

        return buildResponse(200, ddbRes.Item);
    } catch (error) {
        return errorResponse(error);
    }
};
  

The following code is what we had to change the original code to in order to perform a scan against our table.

  
const handler = async (event) => {
  try {
    const scanInput = {
      TableName: TableName
    };

    const ddbRes = await scan(scanInput);
    return buildResponse(200, ddbRes.Items);
  } catch (error) {
    return errorResponse(error);
  }
};
  

CloudGTO Generates Code for Queries Against DynamoDB Tables Only (Not GSIs)

By default, CloudGTO generates code that interacts only with the DynamoDB table. The code does not include running queries against Global Secondary Indexes. Since I have REST APIs that rely on the GSI, I needed to write some code for it (really, I asked Bedrock for some help 😃).

The code below is generated by the CloudGTO Service Builder. As we can see, it assumes that we will query the table by Primary Key.

  
const handler = async (event) => {
    try {
        const id = event.pathParameters?.id;

        if (!id) {
            throw { statusCode: 400, message: "invalid param" };
        }

        const keySchema = {"PK":"mealId"};

        let Item = {
            [keySchema.PK]: id
        };

        const ddbRes = await getItem(TableName, Item);

        if (!ddbRes.Item)
            throw {
                statusCode: 400,
                message: "Item not found"
            };

        return buildResponse(200, ddbRes.Item);
    } catch (error) {
        return errorResponse(error);
    }
};
  

Because we need to query the table by a different attribute value, we have created a Global Secondary Index with a Primary Key of 'category_id'

We need to query the GSI in order to group the Menu Items by Category. The following code includes our changes:

Missing GSI Permissions

I also received an error when calling API endpoints that relied on reading the GSIs. This is because CloudGTO only generates permissions for the table. Additional permissions need to be added manually for GSI access.

Notice, in the original function definition, we only grant permissions to the table:

  
getMealsByCategory:
    handler: src/handlers/getMealsByCategory.handler
    timeout: 30
    runtime: nodejs18.x
    memorySize: 128
    iamRoleStatements:
      - Effect: Allow
        Action:
          - dynamodb:GetItem
          - dynamodb:Scan
          - dynamodb:Query
          - dynamodb:BatchGetItem
          - dynamodb:DescribeTable
        Resource: 
          - arn:aws:dynamodb:${aws:region}:${aws:accountId}:table/${param:DYNAMODB_TABLE_MEALSMENU}
      - Effect: Allow
        Action:
          - xray:PutTraceSegments
          - xray:PutTraceSegment
          - xray:PutTelemetryRecords
          - xray:GetSamplingRules
          - xray:GetSamplingTargets
          - xray:GetSamplingStatisticSummaries
        Resource: "*"
    environment:
      DYNAMODB_TABLE: ${param:DYNAMODB_TABLE_MEALSMENU}
    events:
      - http:
          path: /menu/category/{category_id}
          method: GET
          cors: true
          private: true
  

We had to add the GSI in the Resource List as well to have proper permissions to query:

  
getMealsByCategory:
    handler: src/handlers/getMealsByCategory.handler
    timeout: 30
    runtime: nodejs18.x
    memorySize: 128
    iamRoleStatements:
      - Effect: Allow
        Action:
          - dynamodb:GetItem
          - dynamodb:Scan
          - dynamodb:Query
          - dynamodb:BatchGetItem
          - dynamodb:DescribeTable
        Resource: 
          - arn:aws:dynamodb:${aws:region}:${aws:accountId}:table/${param:DYNAMODB_TABLE_MEALSMENU}
          - arn:aws:dynamodb:${aws:region}:${aws:accountId}:table/${param:DYNAMODB_TABLE_MEALSMENU}/index/category-gsi
      - Effect: Allow
        Action:
          - xray:PutTraceSegments
          - xray:PutTraceSegment
          - xray:PutTelemetryRecords
          - xray:GetSamplingRules
          - xray:GetSamplingTargets
          - xray:GetSamplingStatisticSummaries
        Resource: "*"
    environment:
      DYNAMODB_TABLE: ${param:DYNAMODB_TABLE_MEALSMENU}
    events:
      - http:
          path: /menu/category/{category_id}
          method: GET
          cors: true
          private: true
  

Deploy to Non-Default AWS Account

CloudGTO will deploy to your 'default' AWS Account. This is based on the AWS Credentials/Profiles you have configured.

Because I needed the service deployed to my 'dev' account, I needed to specify that by setting the 'AWS_PROFILE' variable right before invoking 'npm run setup', which takes care of installing dependencies and deployment ('AWS_PROFILE=dev npm run setup').

I kept forgetting to set the variable and end up wondering where my resources are at 😂

I found 'cross-env' which helps address my problem. This makes any variable assignment in the command line to work across various platforms (Mac/Windows/Linux…).

Here is the script definition from CloudGTO:

  
"scripts": {
		"lint": "node_modules/.bin/eslint .",
		"setup": "npm i && sls deploy",
		"auth": "bash ./scripts/auth.bash",
		"format": "prettier --write .",
		"getJWT": "npm run auth dev us-east-2 signup && npm run auth dev us-east-2 signin",
		"getAPIKey": "aws apigateway get-api-keys --name-query lost-n-found-api-dev-api-key --region us-east-2 --include-values --query 'items[0].value' --output text",
		"test": "jest",
		"test:coverage": "jest --coverage"
	}
  

Here is the updated script definitions utilizing 'cross-env' to allow deployment to non-default AWS Account:

  
"scripts": {
		"lint": "node_modules/.bin/eslint .",
		"setup": "npm i && npx cross-env AWS_PROFILE=dev sls deploy",
		"auth": "npx cross-env AWS_PROFILE=dev bash ./scripts/auth.bash",
		"format": "prettier --write .",
		"getJWT": "npx cross-env AWS_PROFILE=dev npm run auth dev us-east-2 signup && npm run auth dev us-east-2 signin",
		"getAPIKey": "npx cross-env AWS_PROFILE=dev aws apigateway get-api-keys --name-query meal-prep-api-dev-api-key --region us-east-2 --include-values --query 'items[0].value' --output text",
		"test": "npx cross-env POWERTOOLS_DEV=true jest",
		"test:coverage": "npx cross-env POWERTOOLS_DEV=true jest --coverage"
	}
  

The rest of the code changes can be found in this Github Repository Pull Request. I wanted to show what we’ve changed from the original CloudGTO-generated code files.

Instructions for how to test deployed service can be found in this ReadMe.

Conclusion

CloudGTO helped me accelerate the development of the REST APIs for a Meal Prep Service. It is a great kick-starter because to me, getting started is the hardest part of building a project. Since CloudGTO Blueprints adhere to Serverless best practices, I did not have to worry about creating anti-patterns and problems for myself. It saved so much time by taking care of boilerplate configurations, code, project setup/organization, and unit tests! As someone who is fairly new to the Serverless Framework and NodeJS, it is also a great learning tool! I did not have to 'google' or research as much because CloudGTO took care of the heavy lifting for me.

This is just the beginning for CloudGTO. The team is working diligently to improve the current features, address bugs, and come up with new functionality.

So, what are you waiting for? 😉 Give it a shot!! Let us know what you think.

We would love to receive some feedback so we can continue to make the product work better for Serverless builders!

Need to reach the team?

You can reach CloudGTO team via email: [contact@cloudgto.com] or by submitting feedback using the link in the top right corner of the Service Builder Start Page (you will only see this once you have created an account to access the public beta).

Report a bug by using the button on the bottom left hand corner of the Service Builder Start Page:

Feedback/Comments/Thoughts

Thank you very much for your time! I would love to hear some feedback so I can improve on my knowledge and writing. If you have any requests on Serverless topics you want to learn more about, let us know!!