Cisco OTV (or Overlay Transport Virtualisation) is a technology inside Cisco Nexus switches (7K) for extending VLANs across a routed network. You can read all about OTV here and here. This post consists of an example configuration for a lab where you have a single Nexus 7K and you want to get OTV over multicast running between VDCs and comes from my CCIE study notes for when I was practicing with OTV. I’ve heard some people have issues with getting the multicast configuration working, so I figured I would share mine here.

First off, to create the illusion of a routed network, this will be an “OTV on a stick” type configuration and not a directly connected one (that would be too easy). Let’s take a look at the topology we want to create:

Cisco_OTV_Lab_with_Multicast_concept

We want the two servers in 10.0.10.0/24 to reach each other through the routed fabric in between. The supervisor we’re using has a 4 VDC limit and in this example I will be using all 4. The smaller router icons are VRFs on the SW1 and SW2 VDCs which you will see return in the configuration.

I’m going to assume you have a working knowledge of basic NX-OS, OSPF and OTV knowledge and have the VDCs set up, have the proper licenses, made the proper interface allocations (mind your M and F linecards and make sure absolutely no other connections are active and trunking, you’ll have a field day looking for why it doesn’t work otherwise) and can start from there. First, some basic information:

OTV Extended VLAN: 10
OTV Site-VLAN: 99
OTV Site-IDs: 0x1 & 0x2
OTV Control: 239.1.1.1
OTV Data Range: 232.1.1.0/24

Let’s begin with numbering all the interfaces and creating the OSPF network:

OTV1

feature ospf

! Define VLANs
vlan 10 
  name SERVERS
vlan 99 
  name OTV-SITE

! Enable OSPF 
router ospf 1
  log-adjacency-changes

! Join interface
interface Ethernet1/27
  ip address 172.16.0.2/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

! Internal interface
interface Ethernet8/1
  switchport mode trunk
  switchport trunk allowed vlan 10,99
  no shutdown

SW1

feature ospf

! Define VLANs
vlan 10 
  name SERVERS
vlan 99 
  name OTV-SITE

! Define vrf context for separating 'intra-dc' traffic from local traffic
vrf context OTV

! Used for OSPF router-id
interface loopback0 
  vrf member OTV 
  ip address 1.1.1.1/32

! OSPF redistribution for following ranges
ip prefix-list OSPF-REDIST permit 172.16.0.0/30
ip prefix-list OSPF-REDIST permit 1.1.1.1/32

route-map OSPF_REDIST permit 10 
  match ip address prefix-list OSPF_REDIST

! Have OSPF send default gateway to following peers:
ip prefix-list OSPF_DEF_ROUTE permit 172.16.0.0/30
route-map OSPF_DEF_ROUTE permit 10 
  match ip address prefix-list OSPF_DEF_ROUTE

! Enable OSPF and redistribute proper prefixes and default gateway to OTV VDC
router ospf 1 
  default-information originate always route-map OSPF_DEF_ROUTE 
  redistribute direct route-map OSPF_REDIST 
  log-adjacency-changes 
  vrf OTV

! Link to OTV1 (Join interface)
interface Ethernet1/28
  vrf member OTV
  ip address 172.16.0.1/30   
  ip ospf network point-to-point 
  ip router ospf 1 area 0.0.0.0   
  no shutdown

! Link to SW2
interface Ethernet1/37 
  vrf member OTV 
  ip address 192.168.0.1/30 
  ip ospf network point-to-point 
  ip router ospf 1 area 0.0.0.0 
  no shutdown

! Link to OTV1 (Internal interface)
interface Ethernet8/9 
  switchport mode trunk 
  switchport trunk allowed vlan 10,99 
  no shutdown

! Link to access switch
interface Ethernet8/10 
  switchport mode trunk 
  switchport trunk allowed vlan 10 
  no shutdown

SW1

feature ospf

! Define VLANs
vlan 10   
  name SERVERS
vlan 99   
  name OTV-SITE

! Define vrf context for separating 'intra-dc' traffic from local traffic
vrf context OTV

! Used for OSPF router-id
interface loopback0 
  vrf member OTV 
  ip address 2.2.2.2/32

! OSPF redistribution for following ranges
ip prefix-list OSPF-REDIST permit 172.16.1.0/30
ip prefix-list OSPF-REDIST permit 2.2.2.2/32

route-map OSPF_REDIST permit 10 
  match ip address prefix-list OSPF_REDIST

! Have OSPF send default gateway to following peers:
ip prefix-list OSPF_DEF_ROUTE permit 172.16.1.0/30
route-map OSPF_DEF_ROUTE permit 10 
  match ip address prefix-list OSPF_DEF_ROUTE

! Enable OSPF and redistribute proper prefixes and default gateway to OTV VDC
router ospf 1 
  router-id 2.2.2.2 
  default-information originate always route-map OSPF_DEF_ROUTE 
  redistribute direct route-map OSPF_REDIST 
  log-adjacency-changes 
  vrf OTV

! Link to OTV2 (Join interface)
interface Ethernet1/36
  vrf member OTV   
  ip address 172.16.1.1/30   
  ip ospf network point-to-point   
  ip router ospf 1 area 0.0.0.0
  no shutdown

! Link to SW1
interface Ethernet1/38   
  vrf member OTV 
  ip address 192.168.0.2/30 
  ip ospf network point-to-point 
  ip router ospf 1 area 0.0.0.0 
  no shutdown

! Link to OTV2 (Internal interface)
interface Ethernet8/17 
  switchport mode trunk 
  switchport trunk allowed vlan 10,99 
  no shutdown

! Link to access switch
interface Ethernet8/18 
  switchport mode trunk 
  switchport trunk allowed vlan 10 
  no shutdown

OTV2

feature ospf

! Define VLANs
vlan 10 
  name SERVERS
vlan 99 
  name OTV-SITE

! Enable OSPF
router ospf 1 
  log-adjacency-changes

! Join interface
interface Ethernet1/48 
  ip address 172.16.1.2/30 
  ip ospf network point-to-point 
  ip router ospf 1 area 0.0.0.0 
  no shutdown

! Internal interface
interface Ethernet8/15 
  switchport mode trunk 
  switchport trunk allowed vlan 10,99 
  no shutdown

With that out of the way you should have a routed network and OTV1 should be able to reach OTV2:

OTV1# ping 172.16.1.2
PING 172.16.1.2 (172.16.1.2): 56 data bytes
64 bytes from 172.16.1.2: icmp_seq=0 ttl=254 time=48.052 ms
64 bytes from 172.16.1.2: icmp_seq=1 ttl=254 time=4.923 ms
64 bytes from 172.16.1.2: icmp_seq=2 ttl=254 time=20.752 ms
64 bytes from 172.16.1.2: icmp_seq=3 ttl=254 time=2.935 ms
64 bytes from 172.16.1.2: icmp_seq=4 ttl=254 time=4.965 ms

--- 172.16.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 2.935/16.325/48.052 ms

Now it’s time for the multicast and OTV configuration. Disclaimer: I am not a multicast expert, but I know enough to get it to work. It might be very well that there is a more efficient way of setting this up, this is just what I did to get it working.

OTV1

feature otv

! Define local OTV identifiers
otv site-vlan 99
otv site-identifier 0x1

! Join interface
interface Ethernet1/27
  ip igmp version 3

! Define OTV overlay with join interface, the extended VLAN and multicast addresses
interface Overlay0  
  otv join-interface Ethernet1/27 
  otv control-group 239.1.1.1 
  otv data-group 232.1.1.0/24 
  otv extend-vlan 10 
  no shutdown

SW1

! Enable multicast routing between SW1 and SW2
feature pim

ip pim send-rp-announce loopback0 group-list 224.0.0.0/4
ip pim auto-rp mapping-agent loopback0
ip pim ssm range 232.0.0.0/8
ip pim auto-rp forward listen

! Link to OTV1 (Join interface)
interface Ethernet1/28 
  ip pim sparse-mode 
  ip igmp version 3

! Link to SW2
interface Ethernet1/37 
  ip pim sparse-mode 
  ip igmp version 3

interface loopback0 
  ip pim sparse-mode

SW2

! Enable multicast routing between SW1 and SW2
feature pim

ip pim send-rp-announce loopback0 group-list 224.0.0.0/4
ip pim auto-rp mapping-agent loopback0
ip pim ssm range 232.0.0.0/8
ip pim auto-rp forward listen

! Link to OTV2 (Join interface)
interface Ethernet1/36 
  ip pim sparse-mode 
  ip igmp version 3

! Link to SW1
interface Ethernet1/38 
  ip pim sparse-mode 
  ip igmp version 3

interface loopback0 
  ip pim sparse-mode

OTV2

feature otv

! Define local OTV identifiers
otv site-vlan 99
otv site-identifier 0x2

! Join interface
interface Ethernet1/48 
  ip igmp version 3

! Define OTV overlay with join interface, the extended VLAN and multicast addresses
interface Overlay0 
  otv join-interface Ethernet1/48 
  otv control-group 239.1.1.1 
  otv data-group 232.1.1.0/24 
  otv extend-vlan 10 
  no shutdown

If you’ve come this far, you’ve got the following topology:

Cisco_OTV_Lab_with_Multicast_topo

At this time, you should have a working OTV adjacency between OTV1 and OTV2. Don’t freak out when there’s no connectivity at first, as OTV adjacencies could take a while before they are actually forwarding. If there’s no connectivity, check the output of this command:

OTV1# sh otv vlan
OTV Extended VLANs and Edge Device State Information (* - AED)
Legend:
(NA) - Non AED, (VD) - Vlan Disabled, (OD) - Overlay Down
(DH) - Delete Holddown, (HW) - HW: State Down
(NFC) - Not Forward Capable
VLAN Auth. Edge Device                   Vlan State             Overlay
---- ----------------------------------- ---------------------- -------
10   OTV1                                inactive(33 s left)    Overlay0

If your output is similar to the above output, wait patiently for the countdown to finish. After that grace period the VLAN status should switch to active.

If everything is working correctly, you should have these types of output:

OTV1# sh otv adjacency 
Overlay Adjacency database

Overlay-Interface Overlay0 : 
Hostname  System-ID        Dest Addr    Up Time    State
OTV2      001e.bd27.c000   172.16.1.2   00:11:21   UP

OTV1# sh otv vlan
OTV Extended VLANs and Edge Device State Information (* - AED)
Legend:
(NA) - Non AED, (VD) - Vlan Disabled, (OD) - Overlay Down
(DH) - Delete Holddown, (HW) - HW: State Down
(NFC) - Not Forward Capable
VLAN Auth. Edge Device                   Vlan State             Overlay
---- ----------------------------------- ---------------------- -------
10   OTV1                                active                 Overlay0

If your adjacency is not online, you have some troubleshooting to do. Here’s a few links that could help you in that case:



Share the wealth!

2 comments on “Cisco OTV Lab with Multicast example

  • Hello there…

    Im also looking to setting up otv in a lab…but luckily, i have two separate 77ks to mess with…my question is, what is the reason for connecting sw1 and sw2?…also, is the multicast routing b/w sw1 and sw2 needed since that is done at the aed vdc?
    thanks in advance

    • Hi Sam,

      Thanks for your message. As for your questions, I’ll try to answer here. The reason for connecting sw1 and sw2 is to form essentially what is your dark fiber running between locations. This is where your traffic goes from location 1 to location 2. There is no way to create a ‘virtual wire’ between sw1 and sw2 internally, so you have to use a physical cable.

      Multicast is definitely also necessary on the sw1 and sw2 as that is the backbone of OTV. You’re right that multicast is used in the OTV VDCs, but it uses it to reach the other OTV instance on the other location. If your transport devices (sw1 and sw2) do not route multicast, the OTV instances won’t be able to talk to each other and form the OTV adjacency.

      Hope that helps.

Leave a Reply

Your email address will not be published. Required fields are marked *