Introduction

Amazon Virtual Private Cloud (VPC) Flow Logs offer a powerful native tool for monitoring and gaining observability into the network traffic flowing in and out of your VPC. They capture information about IP traffic to and from network interfaces, providing valuable insights into application behavior, security posture, and overall network health.

In this blog post, we are going to:

• Explore usage scenarios
• Demonstrate implementation steps
• Review alternatives and important considerations

Scenarios for Use

VPC Flow Logs shine in various scenarios:

• Security and Traffic Monitoring: By analyzing VPC Flow logs, you can detect and investigate suspicious network activity, such as unauthorized access attempts. This enhances your ability to identify and respond to potential security threats effectively. By logging all traffic from an Elastic Network Interface (ENI) or subnet, you identify pertinent gaps in security as it will expose how malicious traffic flows in and out of your network. This enables effective root cause analysis and tightening of security loopholes without sacrificing application performance. In general, monitoring application traffic patterns provides you with information instrumental in optimizing resource allocation and network configurations.
• Troubleshooting Network Issues: Have you ever been in a situation where you can’t figure out why traffic isn’t reaching a particular EC2 instance? Well VPC flow logs can help you verify if the traffic is blocked at the level of the Network ACLs or the Security groups. You can also investigate traffic flow anomalies that might indicate network disruptions or misconfigurations. Analyzing flow logs can pinpoint the source of issues and expedite troubleshooting.
• Compliance Auditing: VPC Flow Logs provide a detailed record of network traffic, which can be crucial for meeting compliance requirements and conducting audits. By retaining and analyzing these logs, you can demonstrate adherence to regulatory standards and internal policies

Catching the flows with Terraform

As implied in the name, Amazon VPC Flow Logs is a feature of Amazon VPC. This could be the default VPC or a custom VPC defined by you. For a demonstration, we will explore how to set up VPC Flow Logs using Terraform. The complete code for the below demonstration can be found in this GitHub repo.

Prerequisites:

• An active AWS account with a VPC
• An EC2 instance running within the VPC

Step 1: Configure Destination

Once the IP traffic going to and from your VPC network interfaces is captured by VPC Flow Logs, you must publish the logs to a destination. As of now, only the following destinations are supported:

• Amazon CloudWatch Logs
• Amazon S3
• Amazon Data Firehose.

For VPC Flow Logs to be published to any of the aforementioned destinations, you must create an IAM role that has sufficient permissions to publish to the specific location. Each destination must have a resource already set up such as CloudWatch log group, an Amazon S3 bucket or an Amazon Data Firehose delivery stream. We will use CloudWatch log group destination for our demo.

If you don’t already have a CloudWatch log group, here is how you can create one with Terraform:

  
resource "aws_cloudwatch_log_group" "flow_logs" {
  name              = "/aws/vpc/vpc-flow-logs"
  retention_in_days = 7  
}
  

‍Here is how you create an IAM role with the required permissions for VPC Flow Logs to publish logs to the CloudWatch log group:

  
resource "aws_iam_role" "vpc_flow_logs" {
  name = "vpc-flow-logs-role"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "vpc-flow-logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
}


resource "aws_iam_policy" "vpc_flow_logs_policy" {
  name        = "vpc-flow-logs-policy"
  path        = "/"
  policy = <<EOF 
  {
	  "Version": "2012-10-17",
	  "Statement": [
	    {
	      "Effect": "Allow",
	      "Action": [
	        "logs:CreateLogGroup",
	        "logs:CreateLogStream",
	        "logs:PutLogEvents",
	        "logs:DescribeLogGroups",
	        "logs:DescribeLogStreams"
	      ],
	      "Resource": [
	        "arn:aws:logs:*:*:log-group:/aws/vpc/flow-logs*",
	        "arn:aws:logs:*:*:log-stream:/aws/vpc/flow-logs*"
	      ]
	    }
	  ]
} 
EOF
}
``` 
  

Step 2: Enable Flow Logs

Enabling flow logs is very straightforward. You can enable it for an entire VPC (all ENIs will be monitored), specific subnets or ENIs.

Here is how you can enable flow logs for your VPC:

  
resource "aws_flow_log" "vpc_flow_logs" {
  iam_role_arn             = aws_iam_role.vpc_flow_logs.arn
  log_destination          = aws_cloudwatch_log_group.flow_logs.arn
  traffic_type             = "ALL"
  vpc_id                   = aws_vpc.this.id
  log_format               = "$${version} $${account-id} $${interface-id} $${srcaddr} $${dstaddr} $${srcport} $${dstport} $${protocol} $${packets} $${bytes} $${start} $${end} $${action} $${log-status}"
	max_aggregation_interval = 60
}
  

• iam_role_arn: The ARN for the IAM role that's used to post flow logs to a CloudWatch Logs’ log group.
• log_destination: Specifies the ARN of the CloudWatch Logs log group where the flow logs will be stored.
• traffic_type: Indicates the type of traffic to capture. In this case, "ALL" means both “REJECT” and “ACCEPT” for traffic that is rejected and allowed respectively.
• vpc_id: Specifies the ID of the VPC for which you want to create flow logs.
• log_format: Specifies the format of the flow logs. It uses variables like $${version}, $${account-id}, $${interface-id}, $${srcaddr}, etc., to represent different fields in the log entries.
• max_aggregation_interval: The maximum interval of time in seconds during which a flow of packets is captured and aggregated into a flow log record.

Step 3: Review and Create

As usual, when deploying a Terraform project, we start by running terraform validate, to verify the syntax and structure of Terraform code and configuration files.

Then run terraform plan to see the changes that Terraform will make to our infrastructure based on the updated configurations.

Finally, run terraform apply —auto-approve to apply the changes to our infrastructure on AWS.

Note that after you create a flow log, it could take a few minutes for the logs to show up in the destination. Remember is it not traffic mirroring but metadata about the actual traffic.

Step 4: Verification

I sent a few HTTP requests to the EC2 instance in the VPC and waited for approximately 15 minutes for the flow logs to populate. Then I went over to CloudWatch to view the captured flow log data in the /aws/vpc/vpc-flow-logs log group.

In this scenario, there is a security group that allows HTTP traffic on TCP port 80 from the internet to an EC2 instance in a public subnet. When HTTP requests are made to the instance’s DNS address, the IP traffic is captured by VPC Flow Logs and sent to the CloudWatch log group. Here is a snippet of some records:

HTTP traffic from the internet to and from the instance is accepted as you can see from the ACCEPT OK field.

If we try opening an SSH session to the instance, it gets rejected as you can see in the snippet below:

The security group for the instance doesn’t allow SSH traffic so it gets rejected.

Alternatives to Amazon VPC Flow Logs for VPC monitoring

While VPC Flow Logs offer a robust solution, consider these alternatives:

Traffic Mirroring

You can use Traffic Mirroring to copy network traffic from a network interface of an Amazon EC2 instance and send it to third-party out-of-band security and monitoring appliances for deep packet inspection. These appliances could be hosted in the cloud or on-premise. You can detect network and security anomalies, gain operational insights, implement compliance and security controls, and troubleshoot issues.

Reachability Analyzer

Reachability Analyzer is a tool you can use to analyze and debug network reachability between two resources in your VPC. After you specify the source and destination resources, Reachability Analyzer produces hop-by-hop details of the virtual path between them when they are reachable, and identifies the blocking component when they are unreachable.

Network Access Analyzer

You can use Network Access Analyzer to understand network access to your resources. This helps you identify improvements to your network security posture and demonstrate that your network meets specific compliance requirements.

Caveats with Amazon VPC Flow Logs

• Cost Considerations: Flow logs stored in CloudWatch Logs incur charges based on data volume. Consider cost-optimization strategies like log filtering or using S3 for long-term storage with lower access costs.
• Limited Visibility: VPC Flow Logs capture metadata about network flows, not the actual packet content. For a deeper analysis of application behavior, consider tools that inspect packet payloads.
• DNS Traffic: VPC flow logs do not capture traffic to and from AWS DNS services. If you need to capture DNS traffic, you will have to set up your own EC2 DNS servers.
• DHCP: Similarly, dynamic host configuration protocol (DHCP) traffic is not recorded
• Multiple IP Addresses: If your network interface has multiple IPv4 addresses and traffic is sent to a secondary private IPv4 address, the flow log displays the primary private IPv4 address in the dstaddr field. To capture the original destination IP address, create a flow log with the pkt-dstaddr field.

These are just some of the limitations I personally experience but there are quite a lot more. You can get the full list of limitations in the Amazon Virtual Private Cloud user guide.

Conclusion

Amazon VPC Flow Logs are a valuable asset for monitoring and analyzing network traffic within your VPC. By understanding its use cases, implementation steps, and limitations, you can leverage them to optimize security, troubleshoot network issues, and gain valuable insights into your application's network behavior. Explore VPC Flow Logs and experiment with different configurations to enhance your VPC's observability and ensure smooth operation.

References

https://docs.aws.amazon.com/vpc/latest/userguide/monitoring.html

https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html#flow-logs-iam-role