vRealize Network Insight provides traffic visibility in AWS by ingesting the VPC Flow Logs. It correlates the flows to EC2 instances and adds more context, like the security groups, tags, and more. 

VPC Flow Logs are not enabled by default and need to be configured before vRNI can start ingesting the logs. That’s where this post comes in. The AWS lab that I use at VMware for demos is for the entire CMBU and has a bunch of smart people doing cool things. In other words, a ton of VPCs get created and deleted, EC2 instances deployed dynamically. I wanted to make sure all the flow logs get into vRNI for our demos.

While you can use Control Tower and set policies to always have flow logs enabled on VPCs, you might not have Control Tower or have access to create those policies. I didn’t. 

PowerShell & AWS

AWS has PowerShell modules for almost all services, including EC2, VPC, and CloudWatch – all I needed to automate the creation of the flow logs. This is the end result:

This script runs through all regions, lists all VPCs and Flow Log configurations, then compares the VPCs and Flow Logs configs to see which VPCs do not have Flow Logs enabled and then creates Flow Logs for those VPCs that don’t have it. In the above screenshot, you can see that some VPCs already have flow logs enabled and that it created CloudWatch Log Groups and configured flow logs.

You can find it here: https://github.com/vrealize-network-insight/vrni-aws-create-flow-logs 

Configuration

The script has a few configuration options, some of which are optional and some are required. Let’s walk through them:

  • $global:VerboseOutput = $True
    When $True, this option makes the script output more. I.e., all the VPCs and flow logs it’s finding.
    When $False, it’ll only tell you the highlights, like which region it’s processing and that it created a new flow log config.
  • $global:logGroupPrefix = "vRNI-FlowLogs_"
    This logGroupPrefix will be put in front of the log group name. The last part will be the VPC ID, so this will result in a log group called “vRNI-FlowLogs_vpc-ewqenjnr3298ru3“
  • $global:logGroupRetention = 1
    vRNI fetches flows every 10 minutes, so 1-day retention for the flow logs is plenty. Let’s not waste more storage on the logs than we have to.
  • $global:awsPermissionsArn = "arn:aws:iam::000000000000:role/vRNI-Flow-Logs-Role"
    This has to be a role that has access to write to the Cloud Watch log groups.
  • $global:skipRegions = @()
    Regions to skip (if any). Example: $global:skipRegions = @(‘eu-west-1’, ‘us-east-1’) 

Change these in the vrni-aws-create-flow-logs.ps1 file and save it. 

Running the Script

After changing the configuration options, you can run the script using: ./vrni-aws-create-flow-logs.ps1 in a PowerShell window. It’ll warn you about making sure the configuration options are updated and that you need to put in AWS credentials first. Check the AWS docs on how to set up the authentication for their PowerShell modules. Check out the previous screenshot for the full runtime.

Run Regularly

I built this to run regularly. As new VPCs are added, I want flow logs to follow. In the script version on GitHub, I intentionally added a pause before continuing with the script to make sure people running it, understand they need to do something before running it. In a scheduled task, you don’t want it to wait. So, if you want to run this automatically in a scheduled task, comment or remove line 42 (Read-Host) and add a Set-AWSCredential -ProfileName YourProfile at the same line. The last bit is to make sure the AWS credentials are always activated before proceeding.

Then set up a scheduled task in Windows, or a crontab job on Linux.



Share the wealth!