VMware vCloud Director 9.5 – Cross-VDC Networking Blog Series – Getting Started with Cross-VDC

In this blog series, we will be covering several aspects of Cross-VDC Networking inside of VMware vCloud Director 9.5. This was created by Daniel Paluszek, Abhinav Mishra, and Wissam Mahmassani.

In this post, we will be reviewing the necessary steps to support Cross-VDC Networking inside of VMware vCloud Director 9.5. These are fairly straightforward since it aligns to the standard requirements set forth from Cross-vCenter NSX.

Pre-Requisites:

1. Cross-VC NSX must be setup. This requires setup of Primary/Secondary NSX Managers, Universal Transport Zone, etc. We will cover some of the high-level aspects below.
2. Although a single vCD instance can be used to manage Cross-VDC Networking, in order to use Org VDCs that are from Multiple vCD instances/sites, Multi-Site Integration must be configured. There is a one-time setup at the Provider Level and then the for each Organization, an Org Association must be made between the vCD instances. I will try to add a post on establishing this at a later time. Please review this whitepaper by Steve Dockar on establishing a vCD multisite configuration.
3. Ensure you have a unique vCloud Director installation ID on each vCloud Director instance/installation. If you have duplicate IDs, this can lead to MAC address conflicts. Fojta did a blog post on updating your ID – please accomplish this before continuing.
   1. Typically, production vCD instances will have unique site ID’s, but this may be pertinent for duplicated lab environments for ongoing testing and evaluation.

Cross-vCenter NSX Configuration

vCD 9.5 does require a standard Cross-vCenter NSX configuration implemented between the resource/payload vCenters before we can do any configuration at the vCloud Director level. Below is what we will accomplish in this section –

There are many guides out there, but here’s a link to the official VMware documentation on setting up cross-vCenter NSX. 

This can be a single or multi-SSO domain topology. In my environment, here’s what I’ve configured between my two sites: Site-A and Site-B.

1. From the Networking and Security plugin, I’ve assigned my Site-A NSX Manager while linking Site-B NSX Manager as the secondary instance
2. From there, I need to establish my Universal Segment ID pool and Transport Zone.
3. Keep in mind you do not want to overlap with an existing Segment ID pool, so pick a number that’s high enough (or out of reach from other pools) – 
4. From the Transport Zone screen, I’ve created my new Transport Zone named “Universal-TZ.” The same guidelines still apply for the control plane mode – if one utilizes Hybrid/Multicast be aware of the RFC1918 requirements for private IP’s to ensure there is no overlap. 
5. Now, I’m ready to connect it to my respective clusters on Site-A and Site-B. Keep in mind I need to hit the drop down for the NSX Manager and attach the respective cluster at your secondary (or additional) location.
6. That’s it! Onto the next configuration which is at the vCloud Director level.

vCloud Director Initial Provider Setup

In this step, we need to assign the correlated NSX Manager to each vCenter instance that’s participating in the Cross-VDC networking solution. I will be showing how I’ve done this for my two sites, Site-A and Site-B, while establishing a fault domain.

1. From my Site-A, navigate to System -> Manage & Monitor -> vSphere Resources -> vCenters
2. We are going to right click, go to Properties of this vCenter
3. From there, we need to go the NSX Manager tab. This is where we populate the following:
   1. Host/IP of NSX Manager
   2. Admin username
   3. Admin password
   4. The Control VM’s are correlated to the Universal Distributed Logical Router (UDLR) function. This is deployed on a specific resource pool just like tenant ESG’s and utilized to push routing updates to each kernel module (i.e. vSphere host).
   5. Control VM Resource Pool vCenter Path – The resource pool vCenter Path starts with the cluster and continues through the RP Tree. (Ex. TestbedCluster1/ParentResourcePool/ControlVMResourcePool)
      1. On each vCenter/NSX Pairing, if you want to use a dedicated resource pool for the Universal DLR control VMs, a resource pool must be created.
   6. Control VM Datastore Name – full name of the datastore in vCenter.
   7. Control VM Management Interface Name – again, full name of the Portgroup in vCenter.
   8. Network Provider Scope – now this is where we establish a fault domain. This Network Provider Scopes need to be unique across each vCenter/NSX Pairing across vCD instances.
4. 
5. Now, on my Site-B, I will configure my respective properties along with a Network Provider Scope of “region-b”
6. Great! Next step is to add the Universal Transport Zone as a new network pool on each vCD instance. This is purely importing the created Universal-TZ and moving on, so very easy –
7. That’s it – now we are ready to enable a specific orgVDC for cross-VDC networking.

Enabling an orgVDC for Cross-VDC Networking

This is a very simple process – really just enable it on a per orgVDC basis.

1. Go to your orgVDCs and right click on the orgVDC you want to enable cross-VDC Networking on. For example, I am enabling this on my Daniel oVDC’s – 
2. Click on the Network Pool and Services sub-tab and you’ll see a new box below the Network Pool that states, ‘Enable Cross VDC Networking (Using Network Pool “Universal-A-TZ” Check this box.
   1. 1. This still allows for local oVDC network creation using the traditional network pool as stated in the screenshot above. Only L2 stretched networks will use the Universal Network Pool.
3. Now, enabling this on my organization VDC in Site-B –
4. We now ready to create our first VDC Group inside of the H5 UI within the “Daniel” organization.

Permissions/Rights required for Cross-VDC Networking

As discussed in the previous blog post, there are specific rights and roles required for Cross-VDC networking that are not enabled by default for the organization administrator. Please review these before the tenant utilizes Cross-VDC networking.

1. VDC Group and Egress Point/Routing Management is tied to the VDC Group Configure Right.
2. Viewing a VDC Group and the Egress Points/Routing is tied to the VDC Group View Right.
3. Creation/Management of Stretched Networks is tied to the Org VDC Network Edit Right.
4. Viewing of Stretched Networks is tied to the Org VDC Network View Right.

Cross-VDC Networking Permissions Review

Moreover, if you want the organization administrator to create their own multisite pairing, they will need the Multisite permissions added –

Creation of the initial Cross-VDC Group

Now we are ready to test the creation of a new Cross-VDC group.

The concept is creating a logical entity that can span 2 or more organization VDC’s. In this example, I am taking a single oVDC from each instance and creating a datacenter group called “Daniel-VDC”

1. Let’s log into the Tenant UI and we should see the Datacenter Groups from the context switching menu
2. Now, I can create my first Cross-VDC group and start establishing my egress points. Awesome! 

Next up, we will review a high-level provider design and design considerations. Thanks!