Formerly known as VeloCloud, the VMware SD-WAN solution is a combination of hardware (the branch routers) and software. The software is the magic sauce, which they prove by providing virtual Edges for vSphere, so you can place the edge anywhere. vSphere isn’t the only virtual format; they’ve also made it available for AWS, Azure, AliCloud, and GCP.
I’ve been deploying regional (US, EMEA, APJ) POPs of SD-WAN HUBs to better replicate an enterprise SD-WAN in the vRealize Network Insight demo lab (more on that in a future post), and ran into some things to take notice of, or even fix during the documented deployment of a virtual edge to AWS.
Network Design
In AWS, the edge appliance is an EC2 instance. Any client behind the edge needs its traffic routed through the edge. This translates into having multiple subnets and routing tables. The clients will be associated with a private routing table, that uses the edge as the default gateway. The edge will be in the private routing table with 1 interface, and also in a public routing table with another interface. Here’s a good visual from the docs:
I do have to mention that this is a simple deployment. Simply putting some workloads behind the edge for regional caching and making the edge itself a Hub for the region. When you’ve got a massive AWS deployment, it’s more likely that you’ll deploy redundant edges and use a transit VPC between the SD-WAN and all other VPCs.
Deployment Options
Setting up large configurations in AWS is a cakewalk with CloudFormation. VMware provides 2 templates that make the deployment a lot easier. The Greenfield template creates everything: a VPC, routing tables, subnets, internet gateway, elastic IPs, deploys the edge AMI, the whole shebang. The Brownfield template works with an existing VPC, subnets, internet gateway, and simply deploys the edge into the existing resources. For all steps and prerequisites, check the documentation.
For my deployment, I went with the greenfield template. Here’s what it deploys:
Deployment Steps
I’ll defer to the documentation for the detailed explanation, but here’s the summary:
- Provision a new edge in the VeloCloud Orchestrator
- Accept the terms of the virtual edge on the AWS Marketplace
- Deploy the CloudFormation template
- Watch the edge get activated in the VCO
Things That Didn’t Work
There’s a couple of things I had to change to make traffic flow and everything to be happy. The steps outlined in the docs will get you as far as an activated edge, but before traffic flows, here are a few things to change.
VCO: Static IP on GE3
The docs tell you to set GE3 (the internal, client-facing interface) to DHCP and let it advertise the subnet it gets to the SD-WAN routing table. DHCP worked fine for me, but the subnet did not get into the SD-WAN routing table (and nothing behind the edge was reachable). After changing it to a static IP (the one AWS had given it), the client subnet got into the routing table. Get the IP from the EC2 instance of the edge and put it into the VCO:
 |
 |
AWS: Associate Private Subnet
In the current greenfield template, there’s 1 step missing that took me a while to figure out. When it creates the private routing table, the private subnet needs to be associated with that routing table, otherwise, it’ll be associated to the main routing table. The route towards the private subnet will work (and so will ping or UDP), but the clients have an incomplete route back to the SD-WAN.
Make sure the private routing table looks like below. Use the Edit subnet associations button to associate the private subnet to the table.
Summary
After making a few minor tweaks, it’s pretty easy to deploy regional hubs, where you don’t have infrastructure yourself, by using AWS’s infrastructure.
Leave a Reply