Agenda
In last article, we had bring up Cisco SD-WAN Controllers components. Now we will continue and on-board Edge routers to overlay. Cisco SD-WAN solution has two types of Edge routers –
- vEdge – Based on Viptela Operating System.
- cEdge – Based on IOS-XE Operating System.
In this article, we will onboard vEdge router’s virtual form ie. vedge-cloud router to our SD-WAN overlay which we had bring on previous article. vEdge router can be further divided into two categories: Hardware Router & Virtual Router. Here is list of HW based vEdge routers.
- vEdge-100: five fixed 10/100/1000 Mbps ports. Comes in three different flavors:
- vEdge 100b: Ethernet only
- vEdge 100m: Ethernet and integrated 2G/3G/4G modem
- vEdge-1000: 8 ports of fixed GE SFP
- vEdge-2000: 2 Pluggable Interface Modules
- vEdge-5000: 4 Network Interface Modules
- ISR 1100-4G: 4 GE WAN ports
- ISR 1100-4GLTE: 4 GE WAN ports, 4G LTE (CAT4)
- ISR 1100-6G: 6 GE WAN ports (4 GE and 2 SFP)
In virtual vEdge category, we have vedge-cloud router which can be installed on following platforms. Refer vEdge cloud data sheet for more details.
- VMware ESXi Host
- Kernal Based Virtual Machine KVM
- Amazon Machine Image (AMI) on Amazon Web Services
- Azure Hyper V
Topology
Here is topology we will be building. SD-WAN controllers are already deployed. Check previous article, if needed. Now we will deploy and on-board DC1-vEdge1 in this topology.
Initial Configuration
- Download the required software version for vedge-cloud router from Cisco Software Download page.
- Note, Always make sure Edge router version is either same or lower than those of SD-WAN controllers. Also, last software version for vEdge routers is 20.9.x. Reference document.
- Install vedge-cloud router image on ESXi host.
- We can onboard Edge routers to SD-WAN overlay manually or thru ZTP (Zero touch provisioning). In this article, we will go for manual method.
- Apply below initial configuration on newly deployed vEdge router.
system
host-name DC1-vEdge-1
system-ip 10.0.10.1
site-id 10
organization-name Controllers-150
clock timezone Asia/Kolkata
vbond 150.100.1.3
!
ntp <<<< NTP is highly recommended to avoid certs issues.
server 123.123.123.123
version 4
prefer
exit
!
!
vpn 0
name "Internet Interface"
interface ge0/0
ip address 199.10.1.2/30
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service ntp
!
no shutdown
!
interface ge0/1
description "MPLS Interface"
ip address 172.16.10.2/30
tunnel-interface
encapsulation ipsec
color mpls restrict
!
no shutdown
!
!
ip route 0.0.0.0/0 172.16.10.1
ip route 0.0.0.0/0 199.10.1.1
!
vpn 512
interface eth0
ip address 192.168.50.33/24
no shutdown
!
ip route 0.0.0.0/0 192.168.50.1- Now this router can reach to vBond from both WAN interfaces.
DC1-vEdge-1# ping 150.100.1.3 source 199.10.1.2
Ping in VPN 0
PING 150.100.1.3 (150.100.1.3) from 199.10.1.2 : 56(84) bytes of data.
64 bytes from 150.100.1.3: icmp_seq=1 ttl=63 time=31.2 ms
64 bytes from 150.100.1.3: icmp_seq=2 ttl=63 time=26.1 ms
^C
--- 150.100.1.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 26.127/28.697/31.268/2.576 ms
DC1-vEdge-1# ping 150.100.1.3 source 172.16.10.2
Ping in VPN 0
PING 150.100.1.3 (150.100.1.3) from 172.16.10.2 : 56(84) bytes of data.
64 bytes from 150.100.1.3: icmp_seq=1 ttl=59 time=21.3 ms
64 bytes from 150.100.1.3: icmp_seq=2 ttl=59 time=20.1 ms
^C
--- 150.100.1.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 20.151/20.725/21.300/0.592 ms
DC1-vEdge-1# Create and Download authorized Edge serial file on PNP portal.
- Login to https://software.cisco.com/
- Look for ‘Network Plug and Play’ and click Manage Devices.
- Select the Smart Account/Virtual account if have multiple accounts associated with your profile.
- Choose “Devices” tab and “Add Software Devices”
- Add Software Device
- Enter PID, desired quantity and choose controller profile. vedge-cloud router used PID – VEDGE-CLOUD-DNA. Save it
- Click next, review and submit. Now you would see vedge-cloud routers added to your controller profile on PNP portal.
- Select “Controller Profiles” tab and hit “Provisioning File” to download it.
- Select “18.3 and newer”, since our controllers are above 18.3 version. Now serial file will be downloaded to your system.
Upload Serial File to vManage
- Login to vManage GUI and Navigate to Configuration > Devices. Select “WAN Edge List” and hit “Upload WAN Edge List
- Choose the downloaded serial file, select Send to Controllers option – Yes and hit “Upload”
- Now you would see vManage GUI has these devices added.
Install Root-CA certificate on vEdge
Since we are using Enterprise CA in our SD-WAN deployment on Controllers, in order to authenticate controllers vEdge router would need Root-CA cert installed. I have copied the RootCA cert to /home/admin directory on vEdge router.
DC1-vEdge-1# vshell
DC1-vEdge-1:~$ pwd
/home/admin
DC1-vEdge-1:~$ ls -ltr
total 8
-rw-r--r-- 1 admin admin 564 Feb 28 22:55 archive_id_rsa.pub
-rw-r--r-- 1 admin admin 1452 Feb 29 10:15 XCA-win10-root-ca.pem <<<<<
DC1-vEdge-1:~$ exit
exit
DC1-vEdge-1# request root-cert-chain install /home/admin/XCA-win10-root-ca.pem
Uploading root-ca-cert-chain via VPN 0
Copying ... /home/admin/XCA-win10-root-ca.pem via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain <<<<
DC1-vEdge-1:~$
DC1-vEdge-1:~$ Apply Chassis Number and Token to vEdge router.
As we are using virtual router (vedge-cloud), additional set of adding Chassis Number and Token is required.
- On vManage GUI, navigate to Configuration > Certificates. Copy Chassis Number & Serial Number/Token. Make sure the this router is valid as per below image.
- Login to vEdge router via SSH and apply chassis number and token.
DC1-vEdge-1# request vedge-cloud activate chassis-number 0147c6b8-a621-c5f0-74d9-5364244f8308 token 779a69b940744f219bd79e9656c6cd9c
DC1-vEdge-1#
DC1-vEdge-1# show control local-properties
personality vedge
sp-organization-name Controllers-150
organization-name Controllers-150
root-ca-chain-status Installed
certificate-status Not-Installed
certificate-validity Not Applicable
certificate-not-valid-before Not Applicable
certificate-not-valid-after Not Applicable
dns-name 150.100.1.3
site-id 10
domain-id 1
protocol dtls
tls-port 0
system-ip 10.0.10.1
chassis-num/unique-id 0147c6b8-a621-c5f0-74d9-5364244f8308
serial-num No certificate installed
subject-serial-num N/A
token 779a69b940744f219bd79e9656c6cd9c
keygen-interval 1:00:00:00
retry-interval 0:00:00:19
no-activity-exp-interval 0:00:00:20
dns-cache-ttl 0:00:00:00
port-hopped TRUE
time-since-last-port-hop 0:01:09:54
pairwise-keying Disabled
embargo-check success
cdb-locked false
number-vbond-peers 1
INDEX IP PORT
-----------------------------------------------------
0 150.100.1.3 12346
number-active-wan-interfaces 2
NAT TYPE: E -- indicates End-point independent mapping
A -- indicates Address-port dependent mapping
N -- indicates Not learned
Note: Requires minimum two vbonds to learn the NAT type
RESTRICT/ LAST VM
PUBLIC PUBLIC PRIVATE PRIVATE PRIVATE MAX CONTROL/ LAST SPI TIME NAT CON
INTERFACE IPv4 PORT IPv4 IPv6 PORT VS/VM COLOR STATE CNTRL STUN LR/LB CONNECTION REMAINING TYPE PRF
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ge0/0 199.10.1.2 12366 199.10.1.2 :: 12366 0/0 biz-internet up 2 no/yes/no No/No 0:00:00:11 0:11:59:48 N 5
ge0/1 172.16.10.2 12366 172.16.10.2 :: 12366 0/0 mpls up 2 yes/yes/no No/No 0:00:00:11 0:11:59:48 N 5
DC1-vEdge-1# - Now vEdge will form control connection to vBond and vBond will share vManage details to it. vEdge will then contact vManage and vManage will issue a signed certificate to it. Using this signed certificate, vEdge will again form control connection with all Controllers.
DC1-vEdge-1# show control local-properties
personality vedge
sp-organization-name Controllers-15
organization-name Controllers-150
root-ca-chain-status Installed
certificate-status Installed
certificate-validity Valid
certificate-not-valid-before Feb 29 05:02:51 2024 GMT
certificate-not-valid-after Feb 26 05:02:51 2034 GMT
dns-name 150.100.1.3
site-id 10
domain-id 1
protocol dtls
tls-port 0
system-ip 10.0.10.1
chassis-num/unique-id 0147c6b8-a621-c5f0-74d9-5364244f8308
serial-num 9763F6EB <<<<<<
subject-serial-num N/A
token Invalid << Token is OTP and will become "Invalid" after use.
keygen-interval 1:00:00:00
retry-interval 0:00:00:19
no-activity-exp-interval 0:00:00:20
dns-cache-ttl 0:00:00:00
port-hopped TRUE
time-since-last-port-hop 0:01:11:12
pairwise-keying Disabled
embargo-check success
cdb-locked false
number-vbond-peers 1
INDEX IP PORT
-----------------------------------------------------
0 150.100.1.3 12346
number-active-wan-interfaces 2
NAT TYPE: E -- indicates End-point independent mapping
A -- indicates Address-port dependent mapping
N -- indicates Not learned
Note: Requires minimum two vbonds to learn the NAT type
RESTRICT/ LAST VM
PUBLIC PUBLIC PRIVATE PRIVATE PRIVATE MAX CONTROL/ LAST SPI TIME NAT CON
INTERFACE IPv4 PORT IPv4 IPv6 PORT VS/VM COLOR STATE CNTRL STUN LR/LB CONNECTION REMAINING TYPE PRF
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ge0/0 199.10.1.2 12366 199.10.1.2 :: 12366 1/0 biz-internet up 2 no/yes/no No/No 0:00:00:09 0:11:59:51 N 5
ge0/1 45.1.0.20 12366 172.16.10.2 :: 12366 1/1 mpls up 2 yes/yes/no No/No 0:00:00:10 0:11:59:49 N 5
DC1-vEdge-1#
DC1-vEdge-1# show control connections
PEER PEER CONTROLLER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP
TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT ORGANIZATION LOCAL COLOR PROXY STATE UPTIME ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart dtls 150.1.1.4 1 1 150.100.1.4 12346 150.100.1.4 12346 Controllers-150 mpls No up 0:00:08:50 0
vsmart dtls 150.1.1.4 1 1 150.100.1.4 12346 150.100.1.4 12346 Controllers-150 biz-internet No up 0:00:08:50 0
vbond dtls 0.0.0.0 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 mpls - up 0:00:08:50 0
vbond dtls 0.0.0.0 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 biz-internet - up 0:00:08:50 0
vmanage dtls 150.1.1.2 1 0 150.100.1.2 12946 150.100.1.2 12946 Controllers-150 mpls No up 0:00:08:50 0
DC1-vEdge-1# - Navigate to vManage Dashboard and you will see newly on-boarded router.
Congratulations ! You have now onboarded your first Edge router to SD-WAN overlay.
Additional Documents
If you encounter issue with control connections, refer tshoot document – SD-WAN Routers: Troubleshoot Control Connections – Cisco Community