This is part 2 of a two-part tutorial series where we delve into a case study on how a privateServerless REST API can be built with Amazon API Gateway (APIGW) and accessed privately by clients in another Amazon VPC.
In part 1, we built the private API in a VPC. In part 2, we will show how clients in another VPC can privately access our private API via an Amazon VPC peering connection and Route53 resolver endpoints.
Architecture
In this part, we will build the Client VPC, VPC Peering connection, and Route53 resolver endpoints.
Building the client VPC Infrastructure
Creating the VPC
Create a file called client-vpc.tf where we will define all the networking configurations necessary for our private API:
➜ tf-private-apigw git:(main) touch client-vpc.tf
Next, define the VPC CIDR, enable DNS hostnames and DNS support to allow resources within the client VPC to be automatically assigned DNS names, and enable Amazon DNS for name resolution respectively:
In order for the client VPC to communicate with the API VPC, we must create a layer 3 connection between the VPCs.
In our case, we chose VPC peering over AWS Transit Gateway because our scenario is simple. We will set up the peering connection for the Client VPC now and the other leg of the peering connection in the API VPC later on.
Create the VPC peering connection for the client VPC, in the client-vpc.tf file:
For clients in the Client VPC to be able to resolve the DNS name of our private API Endpoint in our API VPC, we will need to set up an outbound Route53 resolver endpoint and a resolver rule in the Client VPC. This will ensure DNS queries for “execute-api.eu-central-1.amazonaws.com" will be resolved by the Authoritative DNS servers in the API VPC.
Setting up Outbound DNS resolution in the Client VPC
The first step is to create the security groups in the security-groups.tf file to lock down what traffic is allowed to hit the Route53 resolver endpoint:
Create the outbound Route53 resolver endpoint in the client-vpc.tf file:
resource "aws_route53_resolver_endpoint" "outbound_resolver_ep" {
name = "private-api-outbound-resolver-endpoint"
direction = "OUTBOUND"
security_group_ids = [aws_security_group.outbound_resolver_ep_sg.id]
ip_address {
subnet_id = aws_subnet.client_private_sn_az1.id
ip = "172.128.1.10"
}
ip_address {
subnet_id = aws_subnet.client_private_sn_az2.id
ip = "172.128.2.10"
}
tags = {
Name = "private-api-resolver-endpoint"
}
}
Create a resolver rule, to forward all DNS queries for our private API endpoint’s domain name “execute-api.eu-central-1.amazonaws.com” to the inbound Route53 resolver endpoint in our API VPC. We will set up the inbound Route53 Resolver endpoint in the API VPC later in this tutorial:
Next, we check the resources that would be created/modified by Terraform by running terraform plan in the root of our project directory:
tf-private-apigw git:(main) terraform plan
If there are no syntax errors to fix and we are ok with the changes highlighted, the next step is to deploy to AWS using terraform apply —auto-approve.
Update the private route tables with the client VPC’s CIDR range so all return traffic to the client VPC will be routed to the VPC Peering connection:
Here we will set up the inbound Route 53 Resolver endpoint with an inbound direction, to enable the resolution of external DNS queries to the API VPC from the Client VPC.
We will start by setting up the security group for the Route 53 Resolver endpoint in the security-groups.tf file:
Next, we set up the Route53 inbound resolver in the api-vpc.tf file:
resource "aws_route53_resolver_endpoint" "inbound_resolver_ep" {
name = "private-api-inbound-resolver-endpoint"
direction = "INBOUND"
security_group_ids = [aws_security_group.inbound_resolver_ep_sg.id]
ip_address {
subnet_id = aws_subnet.private_sn_az1.id
ip = "10.0.1.10"
}
ip_address {
subnet_id = aws_subnet.private_sn_az2.id
ip = "10.0.2.10"
}
tags = {
Name = "private-api-inbound-resolver-endpoint"
}
}
Subnet Reservations
We optionally set up subnet reservations for the static IP addresses assigned to the inbound Route53 Resolver endpoints in each availability zone. This will ensure Amazon VPC doesn’t automatically assign any IP address from that range to other VPC resources, which could create conflicts.
Just like we did in part 1 of this tutorial, we will setup an instance in the Client VPC from which we will make API calls to the private API endpoint in the API VPC.
Setting up an EC2 client
The first step is to create a security group for the client instance in the security-groups.tf file:
Since our client instance is in a private subnet, we will use AWS System Manager Session Manager to access the instance for testing via a System Manager VPC Interface endpoint.
AWS System Manager Session Manager service requires two Interface endpoints (ssm and ssmmessages) to be able to access our private EC2 instance. We will set up the endpoints in the client-vpc.tf file.
Firstly, we create the security groups in the security-groups.tf file for the interface endpoints. The security group will allow only HTTPS traffic on port 443:
Run terraform validate, to verify the syntax and structure of our updated Terraform code and configuration files. Then run terraform plan to see the changes that Terraform will make to our infrastructure based on the updated configurations. Finally, run terraform apply —auto-approve to apply the changes to our infrastructure on AWS.
Once Terraform is done applying the changes successfully, you will see similar outputs like this in our CLI:
Testing System Manager Session Manager access
Follow the steps below to launch the System Manager Session Manager to our client-vpc-instance:
1. Open the Amazon EC2 console
2. Click on the Instance ID of our client-vpc-instance
3. Click on “Connect”
4. Click on Session Manager and Connect
5. If you see the CLI then the session was successfully established
We are now ready to test the private API endpoint from the Client VPC.
Testing the Endpoint
All tests will be run in the System Manager Session Manager connection to our client-vpc-instance.
Below are the scripts that we will run to test the private API Endpoints. We obtain the endpoint from the outputs from our Terraform deployment.
CreateClaim
We will make 3 calls to our claim API endpoint to create 3 items in the DynamoDB claimsTable. Make sure you update the —location with your own claim_url or claim_id_url output if you are following along:
Let’s have a look at the DNS resolution from the instance in the Client VPC. We will do a name lookup for our private API endpoint:
As you can see, the client instance can resolve the domain name of our private API endpoint and obtain the private IP addresses of the Interface endpoints for the private API in the API VPC in multiple availability zones (10.0.1.109 & 10.0.2.150).
Notice how the Client VPC’sAmazon DNS server at 172.128.0.2 resolves the domain name of the private API and its Interface VPC Endpoint to the same private IP addresses, 10.0.1.109 in private_sn_az1 and 10.0.2.150 in private_sn_az2.
Let’s look at how the traffic flows from the client VPC to the private API Gateway Endpoint:
The client calls the private API endpoint (in our case, GET https://9yccs3c029.execute-api.eu-central-1.amazonaws.com/dev/claim/2afe60e7-3f11-485a-8a69-16389ff52fbd?policyId=123&memberId=123&memberName=JohnDoe).
DNS Resolution Flow
The DNS query is sent to the VPC+2 IP (172.128.0.2) in the Client VPC that connects to the outbound Route 53 Resolver endpoint.
The Route 53 Resolver forwarding rule is configured to forward the queries for “execute-api.eu-central-1.amazonaws.com” to the inbound endpoints in the API VPC. The query is forwarded to the Route53 outbound Resolver endpoint in the Client VPC.
The outbound endpoint forwards the query to the Route53 inbound Resolver in the API VPC which goes over the VPC Peering connection.
The inbound Route53 Resolver endpoint in the API VPC forwards the DNS query to its VPC + 2 (10.0.0.2) which returns the the Layer 3 IP address of the private API’s Interface endpoint to the client instance in the Client VPC via the same path in reverse.
Packet Forwarding Flow
The client instance in the Client VPC sends the packet containing the payload with the destination IP of the private API Endpoint in the API VPC.
The VPC peering connection forwards the traffic based on the destination IP in the packet to the Interface Endpoint of the private API
Amazon API Gateway passes the payload to our private Lambda through the integration request.
Depending on the request in the payload, the Lambda functions perform CRUD operations on DynamoDB Table via the DynamoDB Gateway Endpoint.
As we did in the first part of this tutorial, we will use the AWS Network Manager VPC Reachability Analyzer to see how the Layer 3 IP Packet moves from the EC2 instance in the client VPC, through the VPC Peering link, to the API VPC and finally hitting the private API Interface Endpoint. Below is a snippet of an analysis.
Cleaning Up
To clean up all resources created by Terraform, run terraform destroy --auto-approve in the project root directory:
In the first part of this tutorial, we created a VPC for our private API Gateway endpoint, and in this part, we created the layer 3 networking and DNS infrastructure so clients in the Client VPC can make API requests to the private API Gateway endpoint successfully.
At Serverless Guru, we're a collective of proactive solution finders. We prioritize genuineness, forward-thinking vision, and above all, we commit to diligently serving our members each and every day.
This is part 2 of a two-part tutorial series where we delve into a case study on how a privateServerless REST API can be built with Amazon API Gateway (APIGW) and accessed privately by clients in another Amazon VPC.
In part 1, we built the private API in a VPC. In part 2, we will show how clients in another VPC can privately access our private API via an Amazon VPC peering connection and Route53 resolver endpoints.
Architecture
In this part, we will build the Client VPC, VPC Peering connection, and Route53 resolver endpoints.
Building the client VPC Infrastructure
Creating the VPC
Create a file called client-vpc.tf where we will define all the networking configurations necessary for our private API:
➜ tf-private-apigw git:(main) touch client-vpc.tf
Next, define the VPC CIDR, enable DNS hostnames and DNS support to allow resources within the client VPC to be automatically assigned DNS names, and enable Amazon DNS for name resolution respectively:
In order for the client VPC to communicate with the API VPC, we must create a layer 3 connection between the VPCs.
In our case, we chose VPC peering over AWS Transit Gateway because our scenario is simple. We will set up the peering connection for the Client VPC now and the other leg of the peering connection in the API VPC later on.
Create the VPC peering connection for the client VPC, in the client-vpc.tf file:
For clients in the Client VPC to be able to resolve the DNS name of our private API Endpoint in our API VPC, we will need to set up an outbound Route53 resolver endpoint and a resolver rule in the Client VPC. This will ensure DNS queries for “execute-api.eu-central-1.amazonaws.com" will be resolved by the Authoritative DNS servers in the API VPC.
Setting up Outbound DNS resolution in the Client VPC
The first step is to create the security groups in the security-groups.tf file to lock down what traffic is allowed to hit the Route53 resolver endpoint:
Create the outbound Route53 resolver endpoint in the client-vpc.tf file:
resource "aws_route53_resolver_endpoint" "outbound_resolver_ep" {
name = "private-api-outbound-resolver-endpoint"
direction = "OUTBOUND"
security_group_ids = [aws_security_group.outbound_resolver_ep_sg.id]
ip_address {
subnet_id = aws_subnet.client_private_sn_az1.id
ip = "172.128.1.10"
}
ip_address {
subnet_id = aws_subnet.client_private_sn_az2.id
ip = "172.128.2.10"
}
tags = {
Name = "private-api-resolver-endpoint"
}
}
Create a resolver rule, to forward all DNS queries for our private API endpoint’s domain name “execute-api.eu-central-1.amazonaws.com” to the inbound Route53 resolver endpoint in our API VPC. We will set up the inbound Route53 Resolver endpoint in the API VPC later in this tutorial:
Next, we check the resources that would be created/modified by Terraform by running terraform plan in the root of our project directory:
tf-private-apigw git:(main) terraform plan
If there are no syntax errors to fix and we are ok with the changes highlighted, the next step is to deploy to AWS using terraform apply —auto-approve.
Update the private route tables with the client VPC’s CIDR range so all return traffic to the client VPC will be routed to the VPC Peering connection:
Here we will set up the inbound Route 53 Resolver endpoint with an inbound direction, to enable the resolution of external DNS queries to the API VPC from the Client VPC.
We will start by setting up the security group for the Route 53 Resolver endpoint in the security-groups.tf file:
Next, we set up the Route53 inbound resolver in the api-vpc.tf file:
resource "aws_route53_resolver_endpoint" "inbound_resolver_ep" {
name = "private-api-inbound-resolver-endpoint"
direction = "INBOUND"
security_group_ids = [aws_security_group.inbound_resolver_ep_sg.id]
ip_address {
subnet_id = aws_subnet.private_sn_az1.id
ip = "10.0.1.10"
}
ip_address {
subnet_id = aws_subnet.private_sn_az2.id
ip = "10.0.2.10"
}
tags = {
Name = "private-api-inbound-resolver-endpoint"
}
}
Subnet Reservations
We optionally set up subnet reservations for the static IP addresses assigned to the inbound Route53 Resolver endpoints in each availability zone. This will ensure Amazon VPC doesn’t automatically assign any IP address from that range to other VPC resources, which could create conflicts.
Just like we did in part 1 of this tutorial, we will setup an instance in the Client VPC from which we will make API calls to the private API endpoint in the API VPC.
Setting up an EC2 client
The first step is to create a security group for the client instance in the security-groups.tf file:
Since our client instance is in a private subnet, we will use AWS System Manager Session Manager to access the instance for testing via a System Manager VPC Interface endpoint.
AWS System Manager Session Manager service requires two Interface endpoints (ssm and ssmmessages) to be able to access our private EC2 instance. We will set up the endpoints in the client-vpc.tf file.
Firstly, we create the security groups in the security-groups.tf file for the interface endpoints. The security group will allow only HTTPS traffic on port 443:
Run terraform validate, to verify the syntax and structure of our updated Terraform code and configuration files. Then run terraform plan to see the changes that Terraform will make to our infrastructure based on the updated configurations. Finally, run terraform apply —auto-approve to apply the changes to our infrastructure on AWS.
Once Terraform is done applying the changes successfully, you will see similar outputs like this in our CLI:
Testing System Manager Session Manager access
Follow the steps below to launch the System Manager Session Manager to our client-vpc-instance:
1. Open the Amazon EC2 console
2. Click on the Instance ID of our client-vpc-instance
3. Click on “Connect”
4. Click on Session Manager and Connect
5. If you see the CLI then the session was successfully established
We are now ready to test the private API endpoint from the Client VPC.
Testing the Endpoint
All tests will be run in the System Manager Session Manager connection to our client-vpc-instance.
Below are the scripts that we will run to test the private API Endpoints. We obtain the endpoint from the outputs from our Terraform deployment.
CreateClaim
We will make 3 calls to our claim API endpoint to create 3 items in the DynamoDB claimsTable. Make sure you update the —location with your own claim_url or claim_id_url output if you are following along:
Let’s have a look at the DNS resolution from the instance in the Client VPC. We will do a name lookup for our private API endpoint:
As you can see, the client instance can resolve the domain name of our private API endpoint and obtain the private IP addresses of the Interface endpoints for the private API in the API VPC in multiple availability zones (10.0.1.109 & 10.0.2.150).
Notice how the Client VPC’sAmazon DNS server at 172.128.0.2 resolves the domain name of the private API and its Interface VPC Endpoint to the same private IP addresses, 10.0.1.109 in private_sn_az1 and 10.0.2.150 in private_sn_az2.
Let’s look at how the traffic flows from the client VPC to the private API Gateway Endpoint:
The client calls the private API endpoint (in our case, GET https://9yccs3c029.execute-api.eu-central-1.amazonaws.com/dev/claim/2afe60e7-3f11-485a-8a69-16389ff52fbd?policyId=123&memberId=123&memberName=JohnDoe).
DNS Resolution Flow
The DNS query is sent to the VPC+2 IP (172.128.0.2) in the Client VPC that connects to the outbound Route 53 Resolver endpoint.
The Route 53 Resolver forwarding rule is configured to forward the queries for “execute-api.eu-central-1.amazonaws.com” to the inbound endpoints in the API VPC. The query is forwarded to the Route53 outbound Resolver endpoint in the Client VPC.
The outbound endpoint forwards the query to the Route53 inbound Resolver in the API VPC which goes over the VPC Peering connection.
The inbound Route53 Resolver endpoint in the API VPC forwards the DNS query to its VPC + 2 (10.0.0.2) which returns the the Layer 3 IP address of the private API’s Interface endpoint to the client instance in the Client VPC via the same path in reverse.
Packet Forwarding Flow
The client instance in the Client VPC sends the packet containing the payload with the destination IP of the private API Endpoint in the API VPC.
The VPC peering connection forwards the traffic based on the destination IP in the packet to the Interface Endpoint of the private API
Amazon API Gateway passes the payload to our private Lambda through the integration request.
Depending on the request in the payload, the Lambda functions perform CRUD operations on DynamoDB Table via the DynamoDB Gateway Endpoint.
As we did in the first part of this tutorial, we will use the AWS Network Manager VPC Reachability Analyzer to see how the Layer 3 IP Packet moves from the EC2 instance in the client VPC, through the VPC Peering link, to the API VPC and finally hitting the private API Interface Endpoint. Below is a snippet of an analysis.
Cleaning Up
To clean up all resources created by Terraform, run terraform destroy --auto-approve in the project root directory:
In the first part of this tutorial, we created a VPC for our private API Gateway endpoint, and in this part, we created the layer 3 networking and DNS infrastructure so clients in the Client VPC can make API requests to the private API Gateway endpoint successfully.