Agenda
In last article, we discussed about Cisco SD-WAN Centralized Control Policies and different use cases. Now we will explore first use case – Hub and Spoke Topology in Cisco SD-WAN using Centralized Control Policy.
Often Network Engineers require Branch to Branch communication thru Data Center/Regional Hub. This can be easily achieved in SD-WAN with the help of Centralized Topology Policy. Further as we can have different Service VPNs in SD-WAN overlay, we can have different topologies per VPN. In this example, we will make Hub n Spoke topology for VPN10 communication while for VPN20 we will keep as full mess(default).
Hub and Spoke Topology in any VPN can be achieved via below methods –
- Block the advertisement of Branch Routes to other Branch sites using Centralized Topology policy via vSmarts. Note, we will need Summary/Default route advertisement from Hub Sites to Branches which will be used by Branch routers to send traffic destined to other Branches to Hub Router. Then Hub router will do the route lookup in it’s routing table and per longest prefix match, will redirect traffic to destined Branch site.
- Optionally, we can also block the advertisement of TLOC routes of Branch site to other Branches if all Service VPNs need to be Hub n Spoke. This will remove Branch to Branch BFD sessions.
- As in SD-WAN, TLOCs are used as next hop for Routes, we can modify TLOC of Branch routes to Hub TLOCs before advertising Branch routes to other Branches. This way when Branch-A need to send traffic to Branch-B, it will send traffic to Hub Site since next Hop is Hub Site TLOC.
Prerequisite
- vSmarts should be in vManage mode i.e. Template attached.
- Note, We have option to configure Policies directly on vSmart using CLI option but it’s not Cisco Recommended.
Topology
Initial-State
Traceroute from LAN Host of BR101 – VPN10 to BR103 LAN is showing direct path to BR103.
BR101-LAN-Host#traceroute vrf br101-vrf10 10.1.103.4
Type escape sequence to abort.
Tracing the route to 10.1.103.4
VRF info: (vrf in name/id, vrf out name/id)
1 10.1.101.2 1 msec 1 msec 1 msec << BR101-vEdge1 LAN Interface
2 172.16.103.2 7 msec 1 msec 1 msec << TLOC IP of BR103-cEdge1
3 10.1.103.4 2 msec * 1 msecMethod 1 – Using Summary / Default Route from Hubs
As mentioned above, in this method we need to block advertisement of Branch routes to other Branches. Also summary/default route advertisement will also be required for this method. You can use any solution for advertisement of Summary/Default route. In this example, I am using default route advertisement from Hub Core routers.
All Branch Edge routers has default route installed via OMP which is pointing to Hub Routers as next-hop. Here are some outputs from BR101-vEdge1 router.
BR101-vEdge-1# show ip route vpn 10 0.0.0.0/0
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive, L -> import
PROTOCOL NEXTHOP NEXTHOP NEXTHOP
VPN PREFIX PROTOCOL SUB TYPE IF NAME ADDR VPN TLOC IP COLOR ENCAP STATUS
---------------------------------------------------------------------------------------------------------------------------------------------
10 0.0.0.0/0 omp - - - - 10.0.10.1 mpls ipsec F,S
10 0.0.0.0/0 omp - - - - 10.0.10.1 biz-internet ipsec F,S
10 0.0.0.0/0 omp - - - - 10.0.10.2 mpls ipsec F,S
10 0.0.0.0/0 omp - - - - 10.0.10.2 biz-internet
ipsec F,S
BR101-vEdge-1# show omp routes vpn 10 0.0.0.0/0 | tab
Code:
C -> chosen
I -> installed
Red -> redistributed
Rej -> rejected
L -> looped
R -> resolved
S -> stale
Ext -> extranet
Inv -> invalid
Stg -> staged
IA -> On-demand inactive
U -> TLOC unresolved
PATH ATTRIBUTE
FROM PEER ID LABEL STATUS TYPE TLOC IP COLOR ENCAP PREFERENCE
-----------------------------------------------------------------------------------------------------------
150.1.1.4 9 1007 C,I,R installed 10.0.10.2 mpls ipsec -
150.1.1.4 10 1007 C,I,R installed 10.0.10.2 biz-internet ipsec -
150.1.1.4 11 1005 C,I,R installed 10.0.10.1 mpls ipsec -
150.1.1.4 12 1005 C,I,R installed 10.0.10.1 biz-internet ipsec -
150.1.1.4 13 1005 C,R installed 10.0.20.2 mpls ipsec -
150.1.1.4 14 1005 C,R installed 10.0.20.2 biz-internet ipsec -
150.1.1.4 15 1007 C,R installed 10.0.20.1 mpls ipsec -
150.1.1.4 16 1007 C,R installed 10.0.20.1 biz-internet ipsec -
BR101-vEdge-1#
BR101-vEdge-1# show ip route vpn 10 10.1.103.0/24 << Routing to BR103 - VPN10 Subnet
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive, L -> import
PROTOCOL NEXTHOP NEXTHOP NEXTHOP
VPN PREFIX PROTOCOL SUB TYPE IF NAME ADDR VPN TLOC IP COLOR ENCAP STATUS
---------------------------------------------------------------------------------------------------------------------------------------------
10 10.1.103.0/24 omp - - - - 10.0.103.1 mpls ipsec F,S
10 10.1.103.0/24 omp - - - - 10.0.103.1 lte ipsec F,S
BR101-vEdge-1#
BR101-vEdge-1# show omp routes vpn 10 10.1.103.0/24 | tab
Code:
C -> chosen
I -> installed
Red -> redistributed
Rej -> rejected
L -> looped
R -> resolved
S -> stale
Ext -> extranet
Inv -> invalid
Stg -> staged
IA -> On-demand inactive
U -> TLOC unresolved
PATH ATTRIBUTE
FROM PEER ID LABEL STATUS TYPE TLOC IP COLOR ENCAP PREFERENCE
-----------------------------------------------------------------------------------------------------------
150.1.1.4 1 1006 C,I,R installed 10.0.103.1 mpls ipsec -
150.1.1.4 2 1006 C,I,R installed 10.0.103.1 lte ipsec -
BR101-vEdge-1# Create Centralized Topology Policy
1. Login to vManage GUI
2. Navigate to Configuration > Policies. Under Centralized Policy, click Add Policy.
3. Define Site and VPN lists and move to next section.
4. Under Topology tab, click Add Topology and choose Custom Control (Route & TLOC). This provide custom control on policy elements, however, there is a dedicated workflow for Hub-and-Spoke topology which also can be used.
5. Provide a name and description of this policy and click Sequence Type, select Route.
6. Define the policy rule having match conditions as All Branches Site List and VPN 10, then reject it. Make sure for the default action, you accept it. Save the policy and move to next section.
7. Provide Policy Name to this Centralized and Description. Topology policy we created above is already imported here. Under Site List, select All-Branches list in Outbound Site List. Here outbound is the direction of policy apply to vSmarts. Add the Policy and check preview/save.
8. Now activate this centralized Policy. Below is Config-Diff of policy activation.
Policy Configuration
policy
lists
vpn-list VPN10
vpn 10
!
site-list All-Branches
site-id 100-200
!
prefix-list _AnyIpv4PrefixList
ip-prefix 0.0.0.0/0 le 32
!
!
control-policy VPN10-HUBnSPOKE
sequence 1
match route
prefix-list _AnyIpv4PrefixList
site-list All-Branches
vpn-list VPN10
!
action reject
!
!
default-action accept
!
!
apply-policy
site-list All-Branches
control-policy VPN10-HUBnSPOKE outVerify Hub and Spoke Topology
- Traceroute for BR101 LAN to BR103 LAN shows traffic is going via Hub Router. Also reverse traceroute depict path via Hub Router.
# PING from BR101 LAN to BR103 LAN
----------------------------
BR101-LAN-Host#traceroute vrf br101-vrf10 10.1.103.4 probe 1 numeric
Type escape sequence to abort.
Tracing the route to 10.1.103.4
VRF info: (vrf in name/id, vrf out name/id)
1 10.1.101.2 1 msec
2 10.1.10.1 2 msec <<< DC1-vEdge1 VRF10 LAN interface
3 172.16.103.2 3 msec
4 10.1.103.4 2 msec
# PING from BR103 LAN to BR101 LAN
----------------------------------
BR103-LAN-Host#traceroute vrf br103-vrf10 10.1.101.4 probe 1 numeric
Type escape sequence to abort.
Tracing the route to 10.1.101.4
VRF info: (vrf in name/id, vrf out name/id)
1 10.1.103.1 1 msec
2 10.1.10.5 1 msec << DC1-vEdge2 VRF10 LAN Interface
3 10.1.101.2 1 msec
4 10.1.101.4 1 msec- Specific Route for remote Branches doesn’t exist branch Edge router, so traffic is using default route learnt from DC to sent traffic.
# Checking route for BR103 LAN subnet on BR101 Edge router.
-----------------------------------------------------------
BR101-vEdge-1# sh ip route vpn 10 10.1.103.0/24 <<< Route not available.
show ip routes-table vpn 10 ipv4 10.1.103.0/24 *
------------------------------------^
syntax error: unknown argument
Error executing command: CLI command error -
BR101-vEdge-1#
# As seen there are no routes for any Branch. Only routes from DC sites available.
----------------------------------------------------------------------------------
BR101-vEdge-1# show ip routes vpn 10
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive, L -> import
PROTOCOL NEXTHOP NEXTHOP NEXTHOP
VPN PREFIX PROTOCOL SUB TYPE IF NAME ADDR VPN TLOC IP COLOR ENCAP STATUS
---------------------------------------------------------------------------------------------------------------------------------------------
10 0.0.0.0/0 omp - - - - 10.0.10.1 mpls ipsec F,S
10 0.0.0.0/0 omp - - - - 10.0.10.1 biz-internet ipsec F,S
10 0.0.0.0/0 omp - - - - 10.0.10.2 mpls ipsec F,S
10 0.0.0.0/0 omp - - - - 10.0.10.2 biz-internet ipsec F,S
10 10.1.10.0/30 omp - - - - 10.0.10.1 mpls ipsec F,S
10 10.1.10.0/30 omp - - - - 10.0.10.1 biz-internet ipsec F,S
10 10.1.10.4/30 omp - - - - 10.0.10.2 mpls ipsec F,S
10 10.1.10.4/30 omp - - - - 10.0.10.2 biz-internet ipsec F,S
10 10.1.20.0/24 omp - - - - 10.0.20.1 mpls ipsec F,S
10 10.1.20.0/24 omp - - - - 10.0.20.1 biz-internet ipsec F,S
10 10.1.20.0/24 omp - - - - 10.0.20.2 mpls ipsec F,S
10 10.1.20.0/24 omp - - - - 10.0.20.2 biz-internet ipsec F,S
10 10.1.101.0/24 connected - ge0/2.10 - - - - - F,S
10 192.168.10.10/32 omp - - - - 10.0.10.1 mpls ipsec F,S
10 192.168.10.10/32 omp - - - - 10.0.10.1 biz-internet ipsec F,S
10 192.168.10.10/32 omp - - - - 10.0.10.2 mpls ipsec F,S
10 192.168.10.10/32 omp - - - - 10.0.10.2 biz-internet ipsec F,S
BR101-vEdge-1# show omp routes vpn 10
Code:
C -> chosen
I -> installed
Red -> redistributed
Rej -> rejected
L -> looped
R -> resolved
S -> stale
Ext -> extranet
Inv -> invalid
Stg -> staged
IA -> On-demand inactive
U -> TLOC unresolved
PATH ATTRIBUTE
VPN PREFIX FROM PEER ID LABEL STATUS TYPE TLOC IP COLOR ENCAP PREFERENCE
--------------------------------------------------------------------------------------------------------------------------------------
10 0.0.0.0/0 150.1.1.4 9 1007 C,I,R installed 10.0.10.2 mpls ipsec -
150.1.1.4 10 1007 C,I,R installed 10.0.10.2 biz-internet ipsec -
150.1.1.4 11 1005 C,I,R installed 10.0.10.1 mpls ipsec -
150.1.1.4 12 1005 C,I,R installed 10.0.10.1 biz-internet ipsec -
150.1.1.4 13 1005 C,R installed 10.0.20.2 mpls ipsec -
150.1.1.4 14 1005 C,R installed 10.0.20.2 biz-internet ipsec -
150.1.1.4 15 1007 C,R installed 10.0.20.1 mpls ipsec -
150.1.1.4 16 1007 C,R installed 10.0.20.1 biz-internet ipsec -
10 10.1.10.0/30 150.1.1.4 1 1005 C,I,R installed 10.0.10.1 mpls ipsec -
150.1.1.4 2 1005 C,I,R installed 10.0.10.1 biz-internet ipsec -
10 10.1.10.4/30 150.1.1.4 1 1007 C,I,R installed 10.0.10.2 mpls ipsec -
150.1.1.4 2 1007 C,I,R installed 10.0.10.2 biz-internet ipsec -
10 10.1.20.0/24 150.1.1.4 1 1005 C,I,R installed 10.0.20.2 mpls ipsec -
150.1.1.4 2 1007 C,I,R installed 10.0.20.1 mpls ipsec -
150.1.1.4 3 1007 C,I,R installed 10.0.20.1 biz-internet ipsec -
150.1.1.4 4 1005 C,I,R installed 10.0.20.2 biz-internet ipsec -
10 10.1.101.0/24 0.0.0.0 66 1011 C,Red,R installed 10.0.101.1 mpls ipsec -
0.0.0.0 68 1011 C,Red,R installed 10.0.101.1 biz-internet ipsec -
10 192.168.10.10/32 150.1.1.4 1 1005 C,I,R installed 10.0.10.1 mpls ipsec -
150.1.1.4 2 1005 C,I,R installed 10.0.10.1 biz-internet ipsec -
150.1.1.4 3 1007 C,I,R installed 10.0.10.2 mpls ipsec -
150.1.1.4 4 1007 C,I,R installed 10.0.10.2 biz-internet ipsec -
BR101-vEdge-1# Method 2 – Modify Branches TLOC Routes
Using this method, you don’t require default/summary route from Hub sites. However, if you can keep default route advertisement from Hub sites for Internet Breakout using Hub Sites.
A Transport Locator (TLOC) acts as a unique identifier for the SD-WAN tunnel interfaces within the SD-WAN infrastructure. It plays a crucial role as the next-hop attribute for all OMP and Network Service routes across the overlay network.
When WAN Edge routers advertise routes to the vSmart controller, each route includes the local TLOCs as next-hop values. In the first step, WAN Edge routers share all locally reachable prefixes with the vSmart controller via OMP updates. Then, in the second step, the vSmart controller redistributes these prefixes to all other WAN Edge routers, utilizing TLOC values as the next-hop addresses.
Utilizing a centralized control policy enables the manipulation of TLOC attributes included in OMP routes, effectively altering the “next-hop address” to which OMP routes point. This capability makes centralized control policies a potent tool for conducting traffic engineering within the SD-WAN fabric. Furthermore, this approach maintains simplicity by requiring policy application from just one location: the vSmart controller.
Now we will first create a TLOC List of Hub Routers and then will modify Topology policy used in first method above.
Create Hub Routers TLOC List
1. Navigate to Configuration > Policies on vManage GUI. Under Custom Options, select Lists for Centralized Policy.
2. Select TLOC from left section and click New TLOC List. Then enter TLOCs of all Hub Edge routers one by one.
Modify Topology Policy
1. Navigate to Configuration > Policies. Then from Custom Options, select Topology under Centralized Policy. From Topology Tab, select edit of Topology Policy.
2. Click Edit icon of Route Rule.
3. Under Action, select Accept. Then choose TLOC and select Hub-TLOCs List created above. Save the policy.
4. Since this topology policy currently active, after modification the changes will be directly pushed to vSmarts. Here is config diff of this change.
Updated Policy Configuration on vSmart
vSmart150-1# show run policy
policy
lists
vpn-list VPN10
vpn 10
!
tloc-list Hub-TLOCs
tloc 10.0.10.1 color mpls encap ipsec
tloc 10.0.10.1 color biz-internet encap ipsec
tloc 10.0.10.2 color mpls encap ipsec
tloc 10.0.10.2 color biz-internet encap ipsec
tloc 10.0.20.1 color mpls encap ipsec
tloc 10.0.20.1 color biz-internet encap ipsec
tloc 10.0.20.2 color mpls encap ipsec
tloc 10.0.20.2 color biz-internet encap ipsec
!
site-list All-Branches
site-id 100-200
!
prefix-list _AnyIpv4PrefixList
ip-prefix 0.0.0.0/0 le 32
!
!
control-policy VPN10-HUBnSPOKE
sequence 1
match route
prefix-list _AnyIpv4PrefixList
site-list All-Branches
vpn-list VPN10
!
action accept
set
tloc-list Hub-TLOCs
!
!
!
default-action accept
!
!Verify Hub and Spoke Topology
# Traceroute from BR101 LAN to BR103 LAN is going via Hub Router.
=================================================================
BR101-LAN-Host#traceroute vrf br101-vrf10 10.1.103.4 probe 1 numeric
Type escape sequence to abort.
Tracing the route to 10.1.103.4
VRF info: (vrf in name/id, vrf out name/id)
1 10.1.101.2 1 msec
2 10.1.10.1 1 msec <<< DC1-vEdge1 VRF 10 LAN Interface.
3 172.16.103.2 1 msec
4 10.1.103.4 1 msec
# Traceroute from BR103 LAN to BR101 LAN is going via Hub Router.
=================================================================
BR103-LAN-Host#traceroute vrf br103-vrf10 10.1.101.4 probe 1 numeric
Type escape sequence to abort.
Tracing the route to 10.1.101.4
VRF info: (vrf in name/id, vrf out name/id)
1 10.1.103.1 1 msec
2 10.1.10.5 1 msec << DC1-vEdge2 VRF 10 LAN Interface
3 10.1.101.2 1 msec
4 10.1.101.4 2 msec
# Now specific routes of other branches available but via Hub Routers.
=================================================================
BR101-vEdge-1# sh ip route vpn 10 10.1.103.0/24
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive, L -> import
PROTOCOL NEXTHOP NEXTHOP NEXTHOP
VPN PREFIX PROTOCOL SUB TYPE IF NAME ADDR VPN TLOC IP COLOR ENCAP STATUS
---------------------------------------------------------------------------------------------------------------------------------------------
10 10.1.103.0/24 omp - - - - 10.0.10.1 mpls ipsec F,S
10 10.1.103.0/24 omp - - - - 10.0.10.1 biz-internet ipsec F,S
10 10.1.103.0/24 omp - - - - 10.0.10.2 mpls ipsec F,S
10 10.1.103.0/24 omp - - - - 10.0.10.2 biz-internet ipsec F,S
BR101-vEdge-1# show omp routes vpn 10 10.1.103.0/24 | tab
Code:
C -> chosen
I -> installed
Red -> redistributed
Rej -> rejected
L -> looped
R -> resolved
S -> stale
Ext -> extranet
Inv -> invalid
Stg -> staged
IA -> On-demand inactive
U -> TLOC unresolved
PATH ATTRIBUTE
FROM PEER ID LABEL STATUS TYPE TLOC IP COLOR ENCAP PREFERENCE
-----------------------------------------------------------------------------------------------------------
150.1.1.4 1 1005 C,I,R installed 10.0.10.1 mpls ipsec -
150.1.1.4 67108865 1005 C,I,R installed 10.0.10.1 biz-internet ipsec -
150.1.1.4 134217729 1007 C,I,R installed 10.0.10.2 mpls ipsec -
150.1.1.4 201326593 1007 C,I,R installed 10.0.10.2 biz-internet ipsec -
150.1.1.4 268435457 1007 C,R installed 10.0.20.1 mpls ipsec -
150.1.1.4 335544321 1007 C,R installed 10.0.20.1 biz-internet ipsec -
150.1.1.4 402653185 1005 C,R installed 10.0.20.2 mpls ipsec -
150.1.1.4 469762049 1005 C,R installed 10.0.20.2 biz-internet ipsec -
This concludes this article. For more details, please refer Cisco Documentation for SD-WAN Centralized Policies.
Next we will explore “Traffic Engineering for inbound traffic using TLOC Preference“
Happy Learning !!!