Cisco SD-WAN – OMP Protocol

OMP Introduction

Cisco SD-WAN uses proprietary protocol – OMP (Overlay Management Protocol) for exchanging information between Edge routers <–> vSmarts and vSmarts <—> vSmarts. OMP is TCP based protocol and similar to BGP protocol but specifically designed for exchanging information on Cisco SD-WAN overlay network. This protocol responsible for exchanging –

• Routing Information of Service VPNs
• TLOCs Information
• Service Chaining
• IPSec Security parameters, Keys and VPN Labels
• Centralized Data and AAR (Application Aware Routing) Policies from vSmarts to Edge Routers
• Multicast Routing Information

Every Edge router forms DTLS based (default) Control connection to vSmarts. The number of Control connections from a Edge router to vSmarts will depend on number of local TLOCs and configuration. By default, Edge router forms two Control connections to vSmarts per TLOC. OMP protocol runs over these Control Connections sessions. Although we can have multiple Control connections between Edge and vSmart, a single OMP session gets established between them. OMP establish session between System-IPs of peers (Edge and vSmart), so it can use any of Control connection to forward it’s packets. Also, since Control Connections are encrypted sessions (DTLS / TLS), OMP doesn’t require it’s native encryption and OMP packets are encrypted using Control Connection session Encryption.

OMP Peering

As can be seen in below output, Edge router has 2 Control Connections to each vSmart Controller but there is only one OMP session per vSmart. It’s internal to OMP peers to choose DTLS/TLS session for OMP peering.

• vSmart1 System-IP – 1.1.1.2
• vSmart2 System-IP – 1.1.1.4

Edge-Router# show control connections
                                                                                                         
PEER    PEER PEER            SITE  PEER             PUB                       
TYPE    PROT SYSTEM IP       ID    PUBLIC IP        PORT    LOCAL COLOR  STATE   
-------------------------------------------------------------------------------
vsmart  dtls 1.1.1.2         1     123.100.1.4      12346     private1     up 
vsmart  dtls 1.1.1.4         1     123.100.1.5      12346     private1     up 
vsmart  dtls 1.1.1.2         1     123.100.1.4      12346     silver       up 
vsmart  dtls 1.1.1.4         1     123.100.1.5      12346     silver       up 
vmanage dtls 1.1.1.100       2     123.100.1.12     12946     silver       up        


Edge-Router# show omp peers
R -> routes received
I -> routes installed
S -> routes sent

                         DOMAIN    OVERLAY   SITE                                
PEER             TYPE    ID        ID        ID        STATE    UPTIME           R/I/S  
------------------------------------------------------------------------------------------
1.1.1.2          vsmart  1         1         1         up       5:01:53:22       11/1/4
1.1.1.4          vsmart  1         1         1         up       5:01:53:12       9/0/4

Edge-Router#

OMP Route Advertisement

In Cisco SD-WAN, Edge routers learns routes in respective VPN/VRF via Connected, Static/Routing Protocols and those routes gets redistributed into OMP. Edge router share this routing information with vSmarts using OMP session. Then vSmarts send this routing information to other Edge routers.

Route Advertisement into OMP can be done globally using OMP Feature template or per VPN level using VPN Feature template. OMP disseminates the following types of routes

• OMP Routes (vRoutes) – These are prefixes learnt from Service VPN via Connected/Static/Routing Protocols to OMP. OMP routes also includes TLOCs information to perform recursive lookup and forward the traffic. This TLOC information can be considered as Next-Hop in BGP.
• TLOC Routes – A TLOC route represents a WAN link serving as a tunnel endpoint, uniquely identified by {System-IP, Color, Encapsulation}. TLOC route also have information of Public/Private WAN IP/Ports information. Notably, the System IP address, rather than the interface IP address, is utilized for identification, as the latter can change if using dynamic IP allocation using DHCP/PPPoE etc. Utilizing the fixed System-IP ensures continuous and unique identification of the TLOC, irrespective of interface IP changes. This distinction is crucial because an OMP route (vRoute) references a next-hop pointing to a TLOC. By separating this information, TLOC routes can be updated with new parameters without necessitating the invalidation of dependent vRoutes.
• Service Routes – Cisco SD-WAN supports Service Chaining which means traffic redirect to a Centralized location for special treatment like Firewall, load-balancer or any custom application. This require advertisement of service route for these services to other branches which need to utilize that service. OMP help in exchange of these service routes.

OMP Route Redistribution

OMP redistributes following types of routes by default from Service VPNs either learned locally or from routing peers. Routes learned from other protocols need explicitly redistribution.

• Connected
• Static
• OSPF Intra Area
• OSPF Inter Area
• OSPFv3 (IPv6) Intra Area
• OSPFv3 (IPv6) Inter Area

For IPv4 routes present in Service VPN, we can also use advertise network <ipv4-prefix> command to advertise a specific prefix. This command is only available in IPv4 address family.

omp
  no shutdown
  graceful-restart
  address-family ipv4 vrf 1
   advertise connected
   advertise static
   advertise network X.X.X.X/X         <<<

Note, route redistribution from OMP to Service Side Routing protocol require explicit configuration to avoid routing loops/sub-optimal routing.

OMP Graceful Restart

Graceful restart for OMP allows the data plane in Cisco SD-WAN overlay to continue functioning if OMP session with vSmarts becomes down. This is done using last known good information Edge routers received from vSmarts. Once connectivity to vSmarts is restored, old routing information is purged with new routing information received from vSmarts.

BR101-vEdge-1# sh omp peers                      
R -> routes received
I -> routes installed
S -> routes sent

                         DOMAIN    OVERLAY   SITE                                
PEER             TYPE    ID        ID        ID        STATE    UPTIME           R/I/S  
------------------------------------------------------------------------------------------
150.1.1.4        vsmart  1         1         1         init-in-gr                 66/42/0

As seen in above output, OMP session with vSmart has gone into Graceful State (init-in-gr) state. Router will mark these OMP routes as Stale but will keep using for forwarding traffic until default Graceful-restart interval of 12 hours which can be configured to max 7 days.

Note, When a change to an OMP graceful restart configuration is made, the OMP session between the Cisco SD-WAN Controllers and the device is flapped. This causes all OMP routes belonging to different address families, such as TLOC, IPv4 or IPv6 unicast, IPv4 multicast, and other families to be withdrawn locally and relearned a few seconds later when the OMP session with the Cisco SD-WAN Controllers comes back up. As the TLOC routes are temporarily removed and added back, Bidirectional Forwarding Detection (BFD) sessions also flap momentarily. This is the expected behavior.

For more information, please refer Cisco’s official documentation here.