Table of Contents
Agenda
Cisco IOS-XE based routers which support Controller mode (i.e. SD-WAN mode) are well known as cEdge routers. In this article, we will on-board virtual form of Cisco IOS-XE router – C8000v.
Cisco IOS-XE SD-WAN WAN Edge devices can be onboarded using one of the following onboarding options:
- Plug-and-Play: The day-zero automated Plug-and-Play process provides a simple, secure procedure to discover, install and provision the Cisco IOS-XE SD-WAN Edge device to join the SD-WAN overlay network.
- Bootstrap: The bootstrap method helps onboard a factory-shipped WAN Edge device with the configuration needed to securely onboard and join the SD-WAN Network, when a customer is unable to leverage the automated discovery option.
- Manual Configuration: Onboard IOS-XE SD-WAN devices using manual configuration via the console port.
We will be using Manual Configuration method in this article.
TIP: For a complete list of Edge router supported on a specific version of Cisco SD-WAN Controllers, you can login on vManage and navigate to Configuration > Template > Feature > Template > Add Template. This will show full list of supported Edge Devices.
Prerequisites:
- A Cisco SD-WAN deployment with vManage, vBond and vSmart already configured.
- Cisco PNP portal access for getting Edge Serial File.
We have already deployed our Cisco SD-WAN Controllers on previous post.
Topology
Here is topology for SD-WAN Lab Deployment. We will be on-boarding DC2-cEdge – C8000v router in this article.
Steps to On-boarding of cEdge router
Create and Download authorized Edge serial file on PNP portal.
- Login to https://software.cisco.com/
- Look for ‘Network Plug and Play’ and click Manage Devices.
- Select the Smart Account/Virtual account if have multiple accounts associated with your profile.
- Choose “Devices” tab and “Add Software Devices”
- Add Software Device
- Enter PID, desired quantity and choose controller profile. C8000v router used PID – C8000v. Save it.
- Click next, review and submit. Now you would see C8000v routers added to your controller profile on PNP portal.
- Select “Controller Profiles” tab and hit “Provisioning File” to download it.
- Select “18.3 and newer”, since our controllers are above 18.3 version. Click Download and Now serial file will be downloaded to your system.
Upload Serial File to vManage
- Login to vManage GUI and Navigate to Configuration > Devices. Select “WAN Edge List” and hit “Upload WAN Edge List
- Choose the downloaded serial file, select Send to Controllers option – Yes and hit “Upload”
- Now you would see vManage GUI has these devices added.
Deploy the C8000v VM on ESXi
- Download the required software version for C8000v router from Cisco Software Download page.
- Note, Always make sure Edge router version is either same or lower than those of SD-WAN controllers. Also, FCS(First Customer Shipment) of C8000v is from 17.4 software version onwards. Before 17.4, virtual form of IOS-XE router was CSR1000v.
- Deploy C8000v ova image on ESXi host.
- Once deployed, login to this VM console using default credentials – admin / admin. System will prompt you to change the password, complete that.
Initial Configuration of cEdge
Initial configuration of cEdge router can be sub-divided into below parts for easy understanding.
Stop PNPA Discover
Since we are going to use manual bring-up method, stop the pnpa discovery. Wait for it to complete.
Router# pnpa service discovery stop
IOS Config
hostname DC2-cEdge-1
username admin privilege 15 secret 9 $9$2.wI2FEJ2lAH2k$4QG8eTO/U9k9sxeu8xF.B8bWb4kkfI8E.JCBeTddSFc
!
clock timezone IST 5 30
!
vrf definition Mgmt-intf
rd 1:512
address-family ipv4
exit-address-family
!
!
interface GigabitEthernet1
no shutdown
vrf forwarding Mgmt-intf
ip address 192.168.50.44 255.255.255.0
exit
interface GigabitEthernet2
no shutdown
ip address 199.20.1.2 255.255.255.252
exit
interface GigabitEthernet3
no shutdown
ip address 172.16.20.2 255.255.255.252
exit
!
ip route 0.0.0.0 0.0.0.0 172.16.20.1
ip route 0.0.0.0 0.0.0.0 199.20.1.1
!
line vty 0 4
transport input ssh
!
line vty 5 80
transport input ssh
!
ntp server 123.123.123.123 prefer version 4SD-WAN System Config
system
system-ip 10.0.20.1
site-id 20
organization-name Controllers-150
vbond 150.100.1.3SD-WAN config and mapping
interface Tunnel2
no shutdown
ip unnumbered GigabitEthernet2
tunnel source GigabitEthernet2
tunnel mode sdwan
exit
interface Tunnel3
no shutdown
ip unnumbered GigabitEthernet3
tunnel source GigabitEthernet3
tunnel mode sdwan
exit
!
sdwan
interface GigabitEthernet2
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service ntp
exit
exit
interface GigabitEthernet3
tunnel-interface
encapsulation ipsec
color mpls restrict
allow-service ntp
exit
exitInstall Root-CA certificate on cEdge
Since we are using Enterprise CA in our SD-WAN deployment on Controllers, in order to authenticate controllers cEdge router would need Root-CA cert installed. I have copied the RootCA cert to bootflash of router.
- Copy RootCA file to cEdge router’s bootflash.
rakesh@jumpserver:~/Controllers-150$ ls -ltr
total 4
-rw-rw-r-- 1 rakesh rakesh 1452 Feb 29 09:57 XCA-win10-root-ca.pem
rakesh@jumpserver:~/Controllers-150$ scp XCA-win10-root-ca.pem admin@192.168.50.44:/RootCA.pem
(admin@192.168.50.44) Password:
XCA-win10-root-ca.pem 100% 1452 561.1KB/s 00:00
rakesh@jumpserver:~/Controllers-150$ - Login to cEdge and install Root Cert.
DC2-cEdge-1#dir bootflash: | in RootCA
38 -rw- 1452 Mar 1 2024 12:19:31 +05:30 RootCA.pem
DC2-cEdge-1#
DC2-cEdge-1#request platform software sdwan root-cert-chain install bootflash:RootCA.pem
Uploading root-ca-cert-chain via VPN 0
Copying ... /bootflash/RootCA.pem via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain
DC2-cEdge-1#
DC2-cEdge-1#show sdwan certificate root-ca-cert
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 6638730949574886743 (0x5c2182a008cd8557)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = IN, ST = KA, L = KA, O = routingguru, OU = routingguru, CN = RC, emailAddress = admin@routing-guru.com
Validity
Not Before: Feb 24 12:39:00 2024 GMT
Not After : Feb 24 12:39:00 2034 GMT
Subject: C = IN, ST = KA, L = KA, O = routingguru, OU = routingguru, CN = RC, emailAddress = admin@routing-guru.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:ce:36:3f:ae:be:2d:d1:05:c6:ed:2d:42:b1:ca:
74:c8:7d:9c:c9:73:af:27:34:a0:1f:1e:84:f4:a3:
b2:a5:14:42:f8:01:c5:60:8e:bb:96:22:26:47:cb:
2d:96:a4:be:d8:84:94:06:b5:d4:18:e4:ab:93:e2:
85:c0:bb:be:7c:0e:d9:71:d2:5e:bc:e6:29:8f:f7:
74:17:52:00:28:77:f4:72:35:47:d2:c1:10:92:aa:
a7:97:fb:ad:55:1c:7d:86:b0:9a:e0:14:e1:1c:77:
3c:67:fd:ca:11:cd:64:bf:54:41:78:83:ed:fc:cd:
85:73:fa:20:65:9a:ea:58:b5:6d:5e:c2:84:5e:e7:
e2:1a:a1:76:d6:90:40:06:42:8e:c7:7c:fd:0c:01:
55:22:6f:fa:3e:06:9a:ad:53:82:86:ca:aa:2e:62:
e6:47:d5:be:0f:62:d6:f0:cd:41:f9:bd:07:bc:7b:
bd:f7:68:f2:29:76:39:26:7f:66:3e:ed:f0:5f:08:
ad:47:65:56:fa:59:85:62:90:9c:90:02:a2:39:55:
22:e6:07:b8:c9:6f:fa:9a:6c:76:18:eb:db:dc:5f:
a4:f0:48:fe:2d:a0:31:07:08:80:4f:da:24:e4:6d:
93:16:1d:4b:76:eb:29:46:ff:6d:c2:76:c5:50:41:
7f:19
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 Subject Key Identifier:
97:A8:BF:54:8F:0B:10:8B:5D:76:8C:FD:27:B9:3C:C5:C6:03:BE:E7
X509v3 Key Usage:
Certificate Sign, CRL Sign
Netscape Cert Type:
SSL CA, S/MIME CA, Object Signing CA
Netscape Comment:
xca certificate
Signature Algorithm: sha256WithRSAEncryption
2b:13:dd:bf:1e:f4:f4:88:e7:9e:3e:b3:99:8d:45:b8:6f:22:
0a:6e:fb:75:26:9d:e4:88:8d:fb:90:78:16:af:36:df:db:d6:
8f:84:23:bd:0c:0d:69:63:16:2c:c9:5b:a7:a3:ef:50:9a:09:
f1:22:eb:8e:5a:2d:29:10:b1:1e:2d:42:bb:97:d2:cf:31:1c:
d8:1a:65:50:77:63:f0:df:bc:73:be:85:fc:8f:84:d5:23:cc:
5d:93:a8:83:51:e8:53:1c:53:4b:2f:fb:53:23:9b:37:b8:c0:
de:8b:01:07:1f:18:7c:71:28:64:ab:cc:7e:c8:77:bc:bd:22:
b7:4f:5a:ae:c3:2f:0d:4f:5e:1f:4e:6e:0e:45:94:97:83:6b:
ff:6c:67:4a:f8:fd:f4:6a:f8:f5:7c:3b:c3:57:64:78:31:3d:
3e:6e:19:90:18:c2:09:dc:67:1a:71:6e:46:1f:73:33:1b:05:
32:55:8f:c3:dc:09:e9:58:4d:99:45:7b:e4:45:21:a4:cc:ea:
b5:af:2d:0b:a4:61:3b:b8:58:67:90:3c:5c:af:19:c5:48:88:
64:63:1a:fe:32:c7:9f:99:50:e0:4c:5d:b6:4f:cc:ba:37:ca:
3f:2d:0d:23:20:79:5d:a8:65:9f:db:4f:ac:bd:f7:d5:61:7e:
2d:9b:f0:d8
DC2-cEdge-1#Apply Chassis Number and Token to cEdge router.
As we are using virtual router (c8000v), additional set of adding Chassis Number and Token is required.
- On vManage GUI, navigate to Configuration > Certificates. Copy Chassis Number & Serial Number/Token of C8kv router. Make sure this router is valid as per below image.
- Login to cEdge router via SSH and apply chassis number and token.
DC2-cEdge-1#show sdwan control local-properties
personality vedge
sp-organization-name controllers-125
organization-name Controllers-150
root-ca-chain-status Installed
certificate-status Not-Installed
certificate-validity Not Applicable
certificate-not-valid-before Not Applicable
certificate-not-valid-after Not Applicable
enterprise-cert-status Not-Applicable
enterprise-cert-validity Not Applicable
enterprise-cert-not-valid-before Not Applicable
enterprise-cert-not-valid-after Not Applicable
dns-name 150.100.1.3
site-id 20
domain-id 1
protocol dtls
tls-port 0
system-ip 10.0.20.1
chassis-num/unique-id C8K-013B343D-A60B-F19D-6AEA-23122B22AF67
serial-num No certificate installed
subject-serial-num N/A
enterprise-serial-num No certificate installed
token Invalid
keygen-interval 1:00:00:00
retry-interval 0:00:00:17
no-activity-exp-interval 0:00:00:20
dns-cache-ttl 0:00:02:00
port-hopped TRUE
time-since-last-port-hop 0:03:02:50
embargo-check success
number-vbond-peers 0
number-active-wan-interfaces 2
NAT TYPE: E -- indicates End-point independent mapping
A -- indicates Address-port dependent mapping
N -- indicates Not learned
Note: Requires minimum two vbonds to learn the NAT type
PUBLIC PUBLIC PRIVATE PRIVATE PRIVATE MAX RESTRICT/ LAST SPI TIME NAT VM
INTERFACE IPv4 PORT IPv4 IPv6 PORT VS/VM COLOR STATE CNTRL CONTROL/ LR/LB CONNECTION REMAINING TYPE CON
STUN PRF
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GigabitEthernet2 199.20.1.2 12426 199.20.1.2 :: 12426 0/0 biz-internet down 2 no/yes/no No/No 0:02:56:35 0:09:03:24 N 5
GigabitEthernet3 172.16.20.2 12426 172.16.20.2 :: 12426 0/0 mpls down 2 yes/yes/no No/No 0:02:56:35 0:09:03:24 N 5
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#request platform software sdwan vedge_cloud activate chassis-number C8K-393D5960-9FB1-AA4B-2230-ED8476A64151 token c19dc5e0ae2a4d0fbb564ff64d7d3440
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#show sdwan control local-properties
personality vedge
sp-organization-name controllers-125
organization-name Controllers-150
root-ca-chain-status Installed
certificate-status Not-Installed
certificate-validity Not Applicable
certificate-not-valid-before Not Applicable
certificate-not-valid-after Not Applicable
enterprise-cert-status Not-Applicable
enterprise-cert-validity Not Applicable
enterprise-cert-not-valid-before Not Applicable
enterprise-cert-not-valid-after Not Applicable
dns-name 150.100.1.3
site-id 20
domain-id 1
protocol dtls
tls-port 0
system-ip 10.0.20.1
chassis-num/unique-id C8K-393D5960-9FB1-AA4B-2230-ED8476A64151
serial-num No certificate installed
subject-serial-num N/A
enterprise-serial-num No certificate installed
token c19dc5e0ae2a4d0fbb564ff64d7d3440
keygen-interval 1:00:00:00
retry-interval 0:00:00:17
no-activity-exp-interval 0:00:00:20
dns-cache-ttl 0:00:02:00
port-hopped TRUE
time-since-last-port-hop 0:03:03:11
embargo-check success
number-vbond-peers 1
INDEX IP PORT
-----------------------------------------------------
0 150.100.1.3 12346
number-active-wan-interfaces 2
NAT TYPE: E -- indicates End-point independent mapping
A -- indicates Address-port dependent mapping
N -- indicates Not learned
Note: Requires minimum two vbonds to learn the NAT type
PUBLIC PUBLIC PRIVATE PRIVATE PRIVATE MAX RESTRICT/ LAST SPI TIME NAT VM
INTERFACE IPv4 PORT IPv4 IPv6 PORT VS/VM COLOR STATE CNTRL CONTROL/ LR/LB CONNECTION REMAINING TYPE CON
STUN PRF
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GigabitEthernet2 199.20.1.2 12426 199.20.1.2 :: 12426 0/0 biz-internet up 2 no/yes/no No/No 0:00:00:08 0:09:03:04 N 5
GigabitEthernet3 172.16.20.2 12426 172.16.20.2 :: 12426 0/0 mpls up 2 yes/yes/no No/No 0:00:00:08 0:09:03:04 N 5
DC2-cEdge-1#- Now cEdge will form DTLS control connection with vBond. vBond will authorize it as we have already uploaded Serial file to vManage which gets pushed to vBond/vSmart.
- vBond will provide vManage details to it.
- cEdge will form DTLS control connection with vManage and vManage will provide sign certificate to cEdge router.
- Then cEdge will again form DTLS control connection to vBond and then to vManage & vSmart.
DC2-cEdge-1#show sdwan control local-properties
personality vedge
sp-organization-name Controllers-150
organization-name Controllers-150
root-ca-chain-status Installed
certificate-status Installed
certificate-validity Valid
certificate-not-valid-before Mar 1 07:30:01 2024 GMT
certificate-not-valid-after Feb 27 07:30:01 2034 GMT
enterprise-cert-status Not-Applicable
enterprise-cert-validity Not Applicable
enterprise-cert-not-valid-before Not Applicable
enterprise-cert-not-valid-after Not Applicable
dns-name 150.100.1.3
site-id 20
domain-id 1
protocol dtls
tls-port 0
system-ip 10.0.20.1
chassis-num/unique-id C8K-393D5960-9FB1-AA4B-2230-ED8476A64151
serial-num 5E39E227
subject-serial-num N/A
enterprise-serial-num No certificate installed
token Invalid << Token will become Invalid as it's OTP
keygen-interval 1:00:00:00
retry-interval 0:00:00:17
no-activity-exp-interval 0:00:00:20
dns-cache-ttl 0:00:02:00
port-hopped TRUE
time-since-last-port-hop 0:03:12:26
embargo-check success
number-vbond-peers 1
INDEX IP PORT
-----------------------------------------------------
0 150.100.1.3 12346
number-active-wan-interfaces 2
NAT TYPE: E -- indicates End-point independent mapping
A -- indicates Address-port dependent mapping
N -- indicates Not learned
Note: Requires minimum two vbonds to learn the NAT type
PUBLIC PUBLIC PRIVATE PRIVATE PRIVATE MAX RESTRICT/ LAST SPI TIME NAT VM
INTERFACE IPv4 PORT IPv4 IPv6 PORT VS/VM COLOR STATE CNTRL CONTROL/ LR/LB CONNECTION REMAINING TYPE CON
STUN PRF
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GigabitEthernet2 199.20.1.2 12426 199.20.1.2 :: 12426 1/0 biz-internet up 2 no/yes/no No/No 0:00:00:01 0:11:57:59 N 5
GigabitEthernet3 45.1.0.16 12426 172.16.20.2 :: 12426 1/1 mpls up 2 yes/yes/no No/No 0:00:00:01 0:11:57:59 N 5
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#
DC2-cEdge-1#show sdwan control connections
PEER PEER CONTROLLER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP
TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT ORGANIZATION LOCAL COLOR PROXY STATE UPTIME ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart dtls 150.1.1.4 1 1 150.100.1.4 12446 150.100.1.4 12446 Controllers-150 mpls No up 0:00:02:12 0
vsmart dtls 150.1.1.4 1 1 150.100.1.4 12446 150.100.1.4 12446 Controllers-150 biz-internet No up 0:00:02:12 0
vbond dtls 0.0.0.0 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 mpls - up 0:00:02:12 0
vbond dtls 0.0.0.0 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 biz-internet - up 0:00:02:12 0
vmanage dtls 150.1.1.2 1 0 150.100.1.2 12646 150.100.1.2 12646 Controllers-150 mpls No up 0:00:02:12 0
DC2-cEdge-1#- Navigate to vManage GUI Dashboard and you should additional WAN Edge.
- As we already have one WAN Edge router at DC1 location, both routers will form BFD sessions between them. Note, we have restricted mpls color so BFD sessions are between biz-internet <–> biz-internet and mpls <–> mpls.
DC2-cEdge-1#show sdwan bfd sessions
SOURCE TLOC REMOTE TLOC DST PUBLIC DST PUBLIC DETECT TX
SYSTEM IP SITE ID STATE COLOR COLOR SOURCE IP IP PORT ENCAP MULTIPLIER INTERVAL(msec UPTIME TRANSITIONS
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10.0.10.1 10 up mpls mpls 172.16.20.2 172.16.10.2 12366 ipsec 7 1000 10 0:00:12:15 0
10.0.10.1 10 up biz-internet biz-internet 199.20.1.2 199.10.1.2 12366 ipsec 7 1000 10 0:00:12:16 0
DC2-cEdge-1#Congratulation! This completes onboarding of DC2-cEdge router to SD-WAN overlay.
Happy Learning !!!
Very helpful
Thanks Sachin !!!
each step is well explained. Many thanks for sharing your expertise and providing valuable insights
Thanks Pradeep for review 🙂