Table of Contents
Agenda
In this post, I will guide on steps and considerations for deploying Cisco SD-WAN Controllers. Cisco SD-WAN has 3 types of Controllers. These Cisco SD-WAN Controllers are virtual machines which can be deployed on Public/Private Cloud solutions or On-premises like on ESXi host. In this article, I will be deploying on VMware ESXi host.
Cisco SD-WAN Controllers Components
- vManage – Cisco SD-WAN vManage is the network management platform designed to simplify and automate the deployment, configuration, management, and operation of Cisco SD-WAN solutions. It provides a highly customizable dashboard that gives you a centralized view of your entire SD-WAN network, making it easier to manage and troubleshoot your network devices and services.
- vBond – Cisco SD-WAN vBond is a key component of the Cisco SD-WAN solution that plays a crucial role in establishing secure and reliable connectivity between various network devices.
- vSmart – Cisco SD-WAN vSmart is the brains behind your network, also known as the “control plane”. It can be compared to Route-Reflector in BGP.
Hardware/Software Used
- Hardware – UCSC-C240-M4S
- ESXi version – 7.0
- SD-WAN Controllers version: 20.12.2
Topology
Here is topology, we are using to deploy Cisco SD-WAN Controllers.
Pre-requisite –
ESXi host should have enough resources to accomodate these Cisco SD-WAN Controllers VMs. Note, below resources are for Lab purpose. Refer my previous article for more details on Pre-requisites.
Note, you might encounter vManage NMS services not running if you allocate less resources.
- vManage
- 16 CPU
- 32 GB Memory
- Disk 1: 21 GB (Approx) Keep the default as per OVA deployment.
- Disk 2: 100 GB (Data Disk)
- vSmart
- 2 CPU
- 4 GB Memory
- Disk 1: 10.5GB (Approx) Keep the default as per OVA deployment.
- vBond
- 4 CPU
- 4 GB Memory
- Disk 1: 10.5GB (Approx) Keep the default as per OVA deployment.
- VMWare ESXI v7.0+ instance for managing VMs and VM networks.
- Workstation with network access to UCS server and controllers, with access to a certificate-signing server/software, such as xca.
Deploy Cisco SD-WAN vManage Controller
vManage deployments could be of following types depending upon number of Edges (Routers), SAIE enabled or not, Disaster recovery enabled or not. Further it could be in Single Tenant or Multi-Tenant modes. I will cover the first option listed below. This is easiest one (i.e. Single Node vManage) to start with. Then in later articles we will explore other options.
- Single Node vManage
- Single Node vManage with DR (Disaster Recovery)
- 3 vManage Nodes Cluster
- 3 vManage Nodes Cluster with DR (Disaster Recovery)
- 6 vManage Nodes Cluster
- 6 vManage Noders Cluster with DR (Disaster Recovery)
Similarly, we can have multiple instances of vbond and vsmarts to achieve redundancy and/or load-balancing.
Steps to deploy Cisco SD-WAN vManage –
- Download vManage ova image for ESXi from https://software.cisco.com/download/home/286320995/type. Note, You should have valid contract associated with your Cisco account for that.
- Deploy this ova image on your ESXi server. I am doing thru vCenter Server but same can be done directly on ESXi GUI.
- Login to vCenter server and Right click on ESXi host. Choose “Deploy OVF Template“
- Choose the downloaded vmanage ova file and follow wizard option same as you deploy any ova file.
- Once VM is deployed, we need to add a new hard drive to it. This new hard drive will be used to store logs, stats, configuration etc for all nodes in overlay. Primary hard disk stores vmanage operating system.
- Here we have three interfaces on vManage
- NIC 1 (eth0) – For Mgmt Purpose
- NIC 2 (eth1) – For SD-WAN Tunnel i.e Transport interface
- NIC 3 (eth2) – For Cluster connectivity. This is OOB link. We will be using this interface later.
Now Power On this vManage VM.
Deploy Cisco SD-WAN vBond/vSmart
We are deploying a single vBond/vSmart instance, however you can have more than one vBond/vSmart instances for load-balancing or redundancy purposes. Note, vBond requires one Transport interface for control connections and one Management interface (Optional). As vBond is the orchestration device for SD-WAN overlay, it needs Public IP(or 1:1 NAT) address on it’s Transport Interface.
Steps to deploy Cisco SD-WAN vBond/vSmart
- Download vEdge-cloud ova image (vBond also uses vEdge-cloud image) and vSmart ova image for ESXi from https://software.cisco.com/download/home/286320995/type. Note, You should have valid contract associated with your Cisco account for that.
- Deploy these ova images on your ESXi server. I am doing thru vCenter Server but same can be done directly on ESXi GUI.
- Login to vCenter server and Right click on ESXi host. Choose “Deploy OVF Template“
- Choose the downloaded vmanage ova file and follow wizard option same as you deploy any ova file.
- Once VM is deployed, you can remove extra Interfaces from vBond VM.
- Here we have below interfaces on vBond.
- NIC 1 (eth0) – For Mgmt Purpose
- NIC 2 (ge0/0) – For SD-WAN Tunnel i.e Transport interface
- Below are interfaces on vSmart VM.
- NIC 1 (eth0) – For Mgmt Purpose
- NIC 2 (eth1) – For SD-WAN Tunnel i.e Transport interface
Now Power On vBond and vSmart VMs.
Create a Controller Profile on Cisco PNP Portal
For Cisco SD-WAN deployment, we need an unique organization-name defined on Cisco PNP portal Controller Profile.
- Login to https://software.cisco.com/
- Look for ‘Network Plug and Play’ and click Manage Devices.
- Select the Smart Account/Virtual account if have multiple accounts associated with your profile.
- Choose ‘Controller Profiles‘ tab and Add a new profile. Choose Controller type ‘VBOND’
- Choose organistion name and enter vBond Public IP (or FQDN name if have multiple vBonds)
Initial Configuration on Cisco SD-WAN Controllers
Once VMs are fully booted, you will get “System Ready” message on VM Console. Then login on vManage Console using default credentials: admin / admin. On First login, system will prompt you to change the password so choose a new password for admin account.
For vManage, we need to do following before initial configuration.
- Choose Persona (Applicable from Software version 20.6 onwards). For single vManage node or 3 vManage cluster nodes, you need select ‘COMPUTE_AND_DATA’. We will discuss more about vManage personas later in article for vManage Clustering. You can refer more on vManage Persona at Cisco documentation.
- Choose the harddisk for data storage and format it.
vManage Initial configuration –
system
host-name vManage150-1
system-ip 150.1.1.2
organization-name Controllers-150 << Organization name
site-id 1
clock timezone Asia/Kolkata
vbond 150.100.1.3
!
ntp << NTP is recommended to avoid certificates caused by time differences.
server 123.123.123.123
version 4
exit
!
!
vpn 0
interface eth1
description "WAN Link"
ip address 150.100.1.2/29
tunnel-interface
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
allow-service ntp << Allowed NAT for time sync
no allow-service stun
allow-service https
!
no shutdown
!
ip route 0.0.0.0/0 150.100.1.1
!
no interface eth0 << We need to remove eth0 from vpn0, so that can be configured in vpn512
!
vpn 512
interface eth0
ip address 10.65.104.172/25
no shutdown
!
ip route 0.0.0.0/0 10.65.104.129
commitvBond Initial Configuration
system
host-name vBond150-1
system-ip 150.1.1.3
site-id 1
organization-name Controllers-150
clock timezone Asia/Kolkata
vbond 150.100.1.3 local << vBond require it's IP with 'local' parameter. This software to act as vBond.
!
ntp
server 123.123.123.123
version 4
exit
!
!
vpn 0
interface ge0/0
ip address 150.100.1.3/29
!
no shutdown
!
ip route 0.0.0.0/0 150.100.1.1
!
vpn 512
interface eth0
ip address 10.65.104.173/25
no shutdown
!
ip route 0.0.0.0/0 10.65.104.129vSmart Initial Configuration
system
host-name vSmart150-1
system-ip 150.1.1.4
site-id 1
organization-name Controllers-150
clock timezone Asia/Kolkata
vbond 150.100.1.3
!
ntp
server 123.123.123.123
version 4
exit
!
!
vpn 0
interface eth1
ip address 150.100.1.4/29
ipv6 dhcp-client
tunnel-interface
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
allow-service ntp
no allow-service stun
!
no shutdown
!
ip route 0.0.0.0/0 150.100.1.1
!
vpn 512
interface eth0
ip address 10.65.104.174/25
no shutdown
!
ip route 0.0.0.0/0 10.65.104.129Onboarding vBond/vSmart
- Login to vManage GUI – https://<vManage-IP>
- Navigate to vManage Administration Settings page – Menu > Administration > Settings. Edit the organization-name & Validator (vBond) IP
- Now Add vBond and vSmart to vManage. Navigate to Menu > Configuration > Devices > Controllers Tab. Click Add vBond.
- Enter vBond IP and vBond credentials.
- Repeat steps and add vSmart
- Now you should have both vBond & vSmart listed on vManage GUI as below.
Completing Certificates on vManage/vBond/vSmart
Cisco SD-WAN require all devices to have valid signed certificates to authenticate each other. There are multiple options available for signed certificates which can be found on vManage Administation Settings page as below.
For this lab project, we will be using Enterprise Root Certificate option. Cisco/Digicert option are easy & straight forward but are mostly for production network purpose.
We will be using xca software for Enterprise Certificates.
- Download and install xca certificate software as per your operating system.
- Open xca application and create a new database via File > New Database.
- Give a name and save the database at desired location.
- Enter a password to protect the database once prompted.
- Create a New Private key for Certificate Server.
- Give a name to this key and desired keysize. You can select this as default.
- Create Self Signed certificate for Root CA. Select ‘Certificates’ Tab and choose ‘New Certificate’
- Select the option as per below image.
- Click on Subject Tab and fill the details for CA server and Click OK. This will create self signed certificate for Root CA.
- Now right click the certificate and choose export to Clipboard/File.
- Paste this RootCA certificate content on vManage Administration Setting page and import this RootCA certificate. vManage will sync this RootCA certificate to vBond/vSmart.
- Now on vManage GUI navigate to Menu > Configuration > Certificates > Controllers. Click on options button for vManage and choose ‘Generate CSR‘ option.
- Download the generated CSR file.
- Follow the above steps to generate CSR for vBond and vSmarts too and download the CSR files.
- Now navigate to xca software and import these CSR files.
- Now Sign these CSR files one by one using xca Root-CA private key.
- Export the signed certificates of vManage, vBond & vSmart from ‘Certificates’ Tab.
- Now install these signed Certificates on vManage GUI. First Install for vManage, followed by vBond and then for vSmart.
Below is the final Certificate page on vManage GUI.
Congratulation, now vManage Dashboard now will reflect vBond and vSmarts.
You can also login to vManage GUI and check control connections.
vManage150-1# show control connections
PEER PEER PEER
PEER PEER PEER CONFIGURED SITE DOMAIN PEER PRIV PEER PUB
INDEX TYPE PROT SYSTEM IP SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT ORGANIZATION REMOTE COLOR STATE UPTIME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 vsmart dtls 150.1.1.4 150.1.1.4 1 1 150.100.1.4 12346 150.100.1.4 12346 Controllers-150 default up 0:00:10:36
0 vbond dtls 150.1.1.3 150.1.1.3 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:33
1 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:33
2 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:33
3 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:33
4 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:34
5 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:34
6 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:33
7 vbond dtls 0.0.0.0 - 0 0 150.100.1.3 12346 150.100.1.3 12346 Controllers-150 default up 0:00:11:34
vManage150-1# This completes bring-up of SD-WAN Controllers. In next article, we will deploy/bring-up Edge routers to this overlay.
Happy Learning. Stay tuned!!!
Pingback: Cisco SD-WAN - vEdge Onboarding
Pingback: Cisco SD-WAN - cEdge Onboarding