Amazon API Gateway empowers organizations to build secure and reliable APIs that make internal services accessible to other applications and external users. As microservices become increasingly common within evolving environments, it's critical to ensure the API layer can handle this growth without compromising security or performance. Amazon API Gateway offers a variety of API types and integration options, but understanding how each choice impacts scalability, security, and performance in a flourishing microservices landscape is essential.

Overview

In this series of articles, we will peel back the layers and reveal the technologies that power API Gateway private integrations. We'll analyze both RESTful and HTTP APIs along with their private integration methods, providing valuable insights for developing secure and scalable microservices architectures. We will also explore a few patterns from simple to complex microservices architectures that require private integrations.

Grasping these details will equip you to make informed decisions about the features and advantages of each option. Ultimately, this knowledge will empower you to design robust Amazon API Gateway APIs with a solid architectural foundation.

Below are individual articles in the series:

1. Mastering Private Integrations in Amazon API Gateway: A Comprehensive Guide
2. HTTP API Private Integrations
3. REST API Private Integrations
4. Scaling Amazon API Gateway Private Integrations
5. How to create HTTP API endpoints to ECS  and EC2-based services
6. How to Create REST API Endpoints to ECS  and EC2-based Services
7. How to Create Private Cross-account Integrations

💡Please note: This article series assumes you're already familiar with creating APIs in API Gateway.  Our focus here is to provide a deeper understanding of the mechanisms behind private integrations.

Part 1: A Deep dive into VPC Links in Amazon API Gateway

Imagine you have a web application hosted in AWS that uses Amazon API Gateway to access backend services like Amazon S3, AWS Lambda, Amazon DynamoDB, Amazon Cognito etc. However, your application suddenly needs to securely access private resources such as databases or microservices within a Virtual Private Cloud (VPC). However, directly exposing your VPC to the public internet increases the attack surface for malicious actors. Some industries have mandatory compliance standards that avoid this type of architecture due to the inherent security risks.

An Elastic Load Balancer (ELB) is natively designed to distribute traffic to multiple internal servers while acting as the boundary between requests coming from the internet. Why should you consider using Amazon API Gateway instead of an ELB? The simple answer is it comes with advanced API management features.  For simple scenarios, an Application Load Balancer (ALB) is adequate for distributing traffic and can also do path-based and host-based routing for different URLs. But if you require more functionality like API versioning, caching, throttling, generation of SDKs, authorization etc. then the Amazon API Gateway is your champion. Let's explore these reasons further.

Amazon API Gateway vs Application Load Balancer

‍


1. When you have low to moderate API traffic: For low to moderate API traffic levels, API Gateway's pay-per-use pricing model can be more cost-effective than the hourly pricing of an ALB, which has a minimum monthly charge.
2. When you need advanced API management features: API Gateway provides features like API versioning, caching, throttling, monitoring, and usage plans that are not available in ALB. If you require these advanced API management capabilities, API Gateway is the better choice.
3. When you need to integrate with AWS services directly: API Gateway can directly integrate with other AWS services like AWS Lambda, AWS Step Functions, and AWS Service Catalog, making it easier to build serverless architectures. ALB does not have this direct integration capability without relying on AWS PrivateLink.
4. When you need advanced authentication and authorization mechanisms: API Gateway supports various authentication methods like API keys, AWS Identity and Access Management (IAM) roles, Lambda authorizers, and Amazon Cognito user pools. It also supports advanced authorization mechanisms like resource policies and AWS WAF integration. ALB only has support for OpenID Connect (OIDC) compliant identity providers (IdP) and mutual TLS Authentication.
5. When you need to generate SDKs and API documentation: API Gateway can automatically generate client SDKs and API documentation based on your API definitions, which can be useful for developers consuming your APIs or producing APIs with OpenAPI specifications. ALB does not provide this functionality.

If you don't require the advanced features of Amazon API Gateway and only require simple load-balancing capabilities, ALB is a simpler and more lightweight solution for you. But as you will see with the patterns we will discuss in the next parts of this series, they are complimentary services.

If advanced API management features are required for your API, then you can easily build robust APIs with first-class integrations to many AWS services. However, EC2-based backends reside in a Virtual Private Cloud (VPC). But Amazon API Gateway is a public service and doesn’t natively reside in a client’s VPC. To be able to connect to VPC-based backends such as EC2 instances, special private integrations are required. These integrations enable and obfuscate the networking complexities required to make the connection secure and private. Let’s examine some of the networking underpinning private networking connections in your AWS environment.

VPC Links, AWS PrivateLink and AWS Hyperplane

AWS Hyperplane

AWS Hyperplane is a Network Function Virtualization (NFV) platform that provides horizontally scalable state management, supporting terabits of multi-tenant capacity. It powers your Network Load Balancer (NLB), NAT Gateway, Transit Gateway and other AWS services. It also has support for inter-VPC connectivity for offerings like AWS PrivateLink. There isn’t much public information about the inner workings of AWS Hyperplane except perhaps in this AWS re:Invent 2017 keynote.

AWS PrivateLink

‍
AWS PrivateLink allows you to establish private connectivity between Amazon Virtual Private Clouds (VPC), other AWS services such as Amazon API Gateway and even on-premises applications without exposing your traffic to the public internet.

AWS PrivateLink is the service underpinning private connections to multiple public AWS services from a VPC like S3, DynamoDB, SSM, Secrets Manager, Parameter Store, KMS etc. However, AWS customers can also use AWS PrivateLink to expose their SaaS products privately to their customers. The service is exposed via a private IP address which prevents direct exposure of data to the internet.

PrivateLink uses two main components:

1. Interface VPC Endpoints: These are Elastic Network Interfaces (ENI) created in the consumer VPC. The interfaces serve as entry points for traffic destined for backend services such as AWS public services or services from AWS partners.
2. Endpoint Services: These are services hosted by a provider such as AWS, AWS Partners, or your applications that enable other AWS accounts to access the exposed services from their own VPCs. The exposed service inside the provider’s VPC is privately connected by AWS PrivateLink to the ENI in the consumer VPC.

VPC Links

A VPC Links is a resource in Amazon API Gateway that serves as a vital conduit for connecting API routes to private resources within a Virtual Private Cloud (VPC). Acting as integration endpoints, they offer a simplified configuration mechanism, abstracting complexities inherent in AWS networking resources.

Amazon API Gateway has two types of RESTful APIs: REST and HTTP. Both are tools for building RESTful APIs but the HTTP API is cheaper, simpler and designed for basic functionality while the REST API is for more complex requirements.

When connecting to VPC-based backend services, private integrations for API Gateway REST and HTTP APIs use the VPC Link resource. But only REST APIs leverage AWS PrivateLink which is not required when using VPC Links with HTTP APIs. This has to do with the nature in which Amazon API Gateway HTTP API’s are designed which consequently enables different connectivity and scaling options with backend resources in VPCs.

We will explore the different scenarios that are possible with API Gateway REST and HTTP API starting in the next parts of this blog series.

Why not use Amazon API Gateway Private APIs to connect to Amazon VPC?

Amazon API Gateway also offers the creation of private RESTful APIs which is a source of confusion because the word “private” is also present in “private integrations”. They both leverage the AWS Hyperplane and AWS PrivateLink but private APIs are for a completely different scenario.

With private API Gateway APIs, you want to create a RESTful API for resources within and between VPCs. The private API cannot be reached from the internet which is the opposite of public API Gateway APIs which are reachable from clients over the internet.

Private integrations enable clients from the internet to reach HTTP/HTTPS endpoints in VPCs via the private integration powered by VPC links to the VPC backend resources. Once again, both REST APIs and HTTP APIs offer private integrations but only VPC links for REST APIs use AWS PrivateLink to achieve this.

Summary

In this post, we reviewed reasons why using Amazon API Gateway to build a RESTful API to your VPC resources is a suitable option for using an ALB which is a native VPC resource. We also saw that for Amazon API Gateway public APIs, you need to enable a private integration to access the VPC resources. The private integrations are powered by VPC links which are resources in the Amazon API Gateway service. The connectivity and scalability options for VPC links depend on whether REST or HTTP tooling is used.

Stay tuned for deeper explorations practical patterns and scaling options in the upcoming parts starting with ”HTTP API Private Integrations” where we will dive deep into the peculiarities of the VPC link resources with the HTTP API tooling option.

References

Understanding VPC links in Amazon API Gateway private integrations | Amazon Web Services

What is AWS PrivateLink? - Amazon Virtual Private Cloud

Private REST APIs in API Gateway - Amazon API Gateway

Best Practices for Designing Amazon API Gateway Private APIs and Private Integration - Best Practices for Designing Amazon API Gateway Private APIs and Private Integration