This post is part of my VMware VCIX-NV Study Guide and covers the Logical Distributed Routers in NSX.

Documentation

Index

 

What is a Logical Distributed Router?
Logical distributed routing is an advanced feature of NSX. It enabled the virtual network to be way more efficient when routing between subnets, which requires a router. The Distributed Logical Router (DLR) is a feature that lives inside the ESXi kernel and acts as the first hop router of a virtual machine. This enables virtual machines in different subnets on the same ESXi host, to keep their network traffic local inside the same ESXi host.

Deploying the LDR entails deploying a virtual appliance (NSX Edge) which is called the LDR control VM. This control VM maintains the routing data for the attached virtual networks and virtual machines, maintains the dynamic routing relationships (OSPF, BGP or IS-IS) and keeps the NSX controllers updated with this information. The NSX controllers update the ESXi hosts, which do the actual routing. Important to know is that (normally) the network traffic going outside the virtual network, does not go through the control VM.

There are a lot of details in the logical distributed router I could go in to, but no-one explains it better than Anthony Burke in his NSX Compendium. Really do give that a very good read and you’ll be verse in the LDR in no time. We’ll dive in the required tasks below.

 

Configure default gateway parameters

Requirements:

  • Existing NSX Edge Logical Distributed Router.

VMware Documentation: Specify Global Configuration

When not using dynamic routing to receive external routes, you can define a default gateway within the LDR.

Configuring default gateway

  • Login to your vSphere Web Client.
  • Navigate to “Networking & Security” and select the “NSX Edges” menu.
  • Choose the NSX Edge you want to modify and select the “Manage” tab.
  • Then select the “Routing” sub-tab and select the “Global Configuration” sub-menu.
  • Once there, click the “Edit” button on the right side of the “Default gateway” table.
  • In the popup window, enter the default gateway details; the external interface, gateway IP address, MTU and an optional description.
  • Click “OK” when done and finally click on “Publish changes” on the top of the page.

 

Add/Remove static routes

Requirements:

  • Existing NSX Edge Logical Distributed Router.

VMware Documentation: Add a Static Route

For smaller LDR deployments, static routes might make the configuration easier then using dynamic routing. Maintaining these static routes can be a time consuming and sometimes confusing task, so try to keep this at a minimal.

Adding static routes

  • Login to your vSphere Web Client.
  • Navigate to “Networking & Security” and select the “NSX Edges” menu.
  • Choose the NSX Edge you want to modify and select the “Manage” tab.
  • Then select the “Routing” sub-tab and select the “Static Routes” sub-menu.
  • Click the “+” icon. In the popup window, enter the destination network in CIDR notation, next hop IP address, outgoing interface, MTU and an optional description.
  • Click “OK” when done and finally click on “Publish changes” on the top of the page.

 

Configure dynamic routing protocols

Dynamic routing is what NSX allows to be very flexible and allow for rapid deployment of new virtual networks, which will get propagated into the rest of the network and activated on the fly. NSX supports three types of dynamic routing protocols, which are basically the three most used protocols in the modern datacenter. These protocols are: OSPF, BGP and IS-IS. Basics of these protocols and configuration guides are below. If you’re a virtualisation administrator venturing into the networking world and want to learn more about these protocols, there are plenty online and offline resources about these protocols, a quick search will get you plenty.

 

OSPF

Requirements:

  • Existing NSX Edge Logical Distributed Router.
  • OSPF neighbour.

VMware Documentation: Configure OSPF Protocol

OSPF (or Open Shortest Path First) is a lightweight routing protocol heavily used in datacenters. OSPF gathers link state information from available routers and constructs a topology map of the network inside it’s own database and decides routing information using that database. When configuring a LDR instance to use OSPF, make sure you have an OSPF-capable neighbour (usually the NSX Edge Services Gateway) inside the same network that the LDR is in. Also create a network design for OSPF (areas, authentication, route redistribution) before beginning with this configuring.

Important definitions to know, before beginning:

Forwarding Address: This IP address will be used by the LDR to forward network traffic and is shared by the ESXi hosts. This one should exist on an interface attached to the LDR.
Protocol Address: This IP address is used by the LDR control VM to maintain the peering connections.

Configuring OSPF on the LDR

  • Login to your vSphere Web Client.
  • Navigate to “Networking & Security” and select the “NSX Edges” menu.
  • Choose the NSX Edge you want to modify and select the “Manage” tab.
  • Then select the “Routing” sub-tab and select the “Global Configuration” sub-menu.
  • Click the “Edit” button on the “Dynamic Routing Configuration” table.
  • Select a “Router ID” and whether or not you want to log dynamic routing events. The Router ID can be an interface address or a fictional IP address you make up.
  • Click “Publish changes” on the top of the page and navigate to the “OSPF” sub-menu.
  • Click the “Edit” button at the top right corner. Tick “Enable OSPF” and fill out the “Protocol” and “Forwarding” addresses and click “OK”.
  • In the view called “Area Definitions”, click the “+” icon to add an area.
  • In the popup window, enter the area ID, type (normal or NSSA) and whether you would like to have authentication (the “Value” field is the password) between the OSPF peers. Like regular network equipment, NSX supports area IDs of numeric value of IP address format. Click “OK”.
  • In the view called “Area to Interface mappings”, click the “+” icon.
  • Select the interface and the matching area ID and optional timer settings. Adjust the timer settings to the OSPF neighbour or leave them as default if you’re peering with another NSX Edge. Click “OK”.
  • Click “OK” when done and finally click on “Publish changes” on the top of the page.

Once your configuration is done, you can verify the OSPF peering status by logging into the LDR management console (KVM or SSH) and executing the following commands:

  • show ip ospf neighbors
  • show ip route ospf

The exact output depends on your network configuration, but it should look a bit like this:


 

BGP

Requirements:

  • Existing NSX Edge Logical Distributed Router.
  • BGP neighbour.

VMware Documentation: Configure BGP Protocol

Border Gateway Protocol (or BGP) is a dynamic routing protocol usually found at the edge of your network, peering with transit providers or public peers sharing their network routes. For internal network use, it can be rather slow. The convergence speed depends on your timer configuration, but generally speaking: BGP is for scale, not for convergence speed.

BGP works with Autonomous Systems (AS) which identify a network. When using it in your internal network, they usually say you’re using iBGP (internal BGP, as opposed to external BGP (eBGP)). When creating a peering between routers, you can define prefix filters which determine which IP prefixes (subnets) are accepted or rejected by the router and which IP prefixes are sent out to the neighbours. You can also secure a peering with a password.

NSX supports BGP on the LDR and on the ESG, but my personal recommendation is to stick with OSPF or IS-IS for internal peerings (unless your networking team requires otherwise).

Adding a BGP Neighbour

  • Login to your vSphere Web Client.
  • Navigate to “Networking & Security” and select the “NSX Edges” menu.
  • Choose the NSX Edge you want to modify and select the “Manage” tab.
  • Then select the “Routing” sub-tab and select the “Global Configuration” sub-menu.
  • Click the “Edit” button on the “Dynamic Routing Configuration” table.
  • Select a “Router ID” and whether or not you want to log dynamic routing events. The Router ID can be an interface address or a fictional IP address you make up.
  • Click “Publish changes” on the top of the page and navigate to the “BGP” sub-menu.
  • Click the “Edit” button at the top right corner. Tick “Enable BGP” and fill out the “Local AS” field with the desired Autonomous System (AS) number and click “OK”.
  • Next, define a BGP peer. In the “Neighbours” table, click the “+” icon to start adding a BGP peer.
  • Enter the peer details. “IP Address” is the IP of the remote peer. “Forwarding” and “Protocol” IP addresses are the same as in the OSPF configuration. Enter the “Remote AS” local to the remote peer. Optionally enter a customised weight, keep alive and hold down timers. Also provide an optional peering password and IP filters. Click “OK” when you’re done.
  • Click “OK” when done and finally click on “Publish changes” on the top of the page.

 

Once your configuration is done, you can verify the BGP peering status by logging into the LDR management console (KVM or SSH) and executing the following commands:

  • show ip bgp neighbors
  • show ip route bgp

The exact output depends on your network configuration, but it should look a bit like this:

 

IS-IS

Requirements:

  • Existing NSX Edge Logical Distributed Router.
  • IS-IS neighbour.

VMware Documentation: Configure IS-IS Protocol

The Intermediate System to Intermediate System (IS-IS) is a widely used protocol as underlay dynamic routing protocol. Examples are Ciscos Overlay Transport Virtualization and FabricPath.

Configuring IS-IS

  • Login to your vSphere Web Client.
  • Navigate to “Networking & Security” and select the “NSX Edges” menu.
  • Choose the NSX Edge you want to modify and select the “Manage” tab.
  • Then select the “Routing” sub-tab and select the “Global Configuration” sub-menu.
  • Click the “Edit” button on the “Dynamic Routing Configuration” table.
  • Select a “Router ID” and whether or not you want to log dynamic routing events. The Router ID can be an interface address or a fictional IP address you make up.
  • Click “Publish changes” on the top of the page and navigate to the “IS-IS” sub-menu.
  • Click the “Edit” button at the top right of the page. In the popup window, tick “Enable IS-IS”, enter a “System ID“, select the “IS Type” and enter an domain and area password. Click “OK” when you’re done.
  • In the “Areas” view, click “Edit” to define the IS-IS areas. Click “OK when you’re done.
  • Next, activate an interface for IS-IS by clicking the “+” icon at the “Interface Mapping” table.
  • Select the interface, the “Circuit Type” and optionally enter the different timers to tweak the IS-IS behaviour.
  • Click “OK” when done and finally click on “Publish changes” on the top of the page.

 



Share the wealth!