Step 1: gather information
A. VSAN Nodes
Each site shall have two sets of MDS Switching fabrics. In our case, we are targeting the Florida data center that hosts FL-FABRICA and FL-FABRICB that are dedicated to VSAN 10 & VSAN 20, respectively. Each fabric may consist of multiple switches of various generations being chained together using specialized cables [to connect the back-planes]. Our use-case also includes a 3PAR branded SAN storage array with four (4) controllers. Each controller carries one (1) fiber optic connection toward FL-FABRICA, and one (1) toward FL-FABRICB. Thus, there are eight (8) paths from the SAN fabrics to reach the 3PAR SAN. Below are the IP addresses of these devices.
- FL-FABRICA: 10.10.8.1
- FL-FABRICB: 10.10.8.2
- FL-3PAR01: 10.10.10.1
B. Host to Fabric Connections
Before starting any configuration, it’s important to verify physical connections between a newly installed ESXi host toward each of the switching fabric. In this example, FL-ESX007 HBA port 1 is plugged into FABRIC-A fiber channel 4 port 1 (fabric-a fc4/1), and FL-ESX007 HBA port 2 is attached to FABRIC-B fiber channel 4 port 1 (fabric-b fc4/1). Here is the illustration.
FL-ESX007 HBA port 1 <==1 connection==> fabric-a fc4/1 <==4 connections==> FL-3PAR01
FL-ESX007 HBA port 2 <==1 connection==> fabric-b fc4/1 <==4 connections==> FL-3PAR01
C. Use Configuration Template to Generate Configs and Review for Accuracy
This can be done with a PowerShell Script here. You’re welcome.
############################# Configuration Script for FL-ESX007 #############################
##############################################################
## FL-FABRICA : VSAN 10 ##
config t
fcalias name fl_esx007_001 VSAN 10
member pwwn xx:xx:xx:xx:xx:xx:xx:xx
exit
zone name fl_esx007_001-fl_3par01_011 VSAN 10
member fcalias fl_esx007_001
member fcalias fl_3par01_011
exit
zone name fl_esx007_001-fl_3par01_111 VSAN 10
member fcalias fl_esx007_001
member fcalias fl_3par01_111
exit
zone name fl_esx007_001-fl_3par01_211 VSAN 10
member fcalias fl_esx007_001
member fcalias fl_3par01_211
exit
zone name fl_esx007_001-fl_3par01_311 VSAN 10
member fcalias fl_esx007_001
member fcalias fl_3par01_311
exit
zoneset name ZoneSet01 VSAN 10
member fl_esx007_001-fl_3par01_011
member fl_esx007_001-fl_3par01_111
member fl_esx007_001-fl_3par01_211
member fl_esx007_001-fl_3par01_311
exit
zoneset activate name ZoneSet01 VSAN 10
copy running-config startup-config fabric
########################################################
##############################################################
## FL-FABRICB : VSAN 20 ##
config t
fcalias name fl_esx007_002 VSAN 20
member pwwn xx:xx:xx:xx:xx:xx:xx:xx
exit
zone name fl_esx007_002-fl_3par01_012 VSAN 20
member fcalias fl_esx007_002
member fcalias fl_3par01_012
exit
zone name fl_esx007_002-fl_3par01_112 VSAN 20
member fcalias fl_esx007_002
member fcalias fl_3par01_112
exit
zone name fl_esx007_002-fl_3par01_212 VSAN 20
member fcalias fl_esx007_002
member fcalias fl_3par01_212
exit
zone name fl_esx007_002-fl_3par01_312 VSAN 20
member fcalias fl_esx007_002
member fcalias fl_3par01_312
exit
zoneset name ZoneSet01 VSAN 20
member fl_esx007_002-fl_3par01_012
member fl_esx007_002-fl_3par01_112
member fl_esx007_002-fl_3par01_212
member fl_esx007_002-fl_3par01_312
exit
zoneset activate name ZoneSet01 VSAN 20
copy running-config startup-config fabric
##############################################################
Step 2: Perform the Configuration
SSH into FL-FABRICA
Cisco Nexus Operating System (NX-OS) Software
TAC support: https://www.cisco.com/tac/
Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
https://opensource.org/license/gpl-2-0 and
https://opensource.org/license/lgpl-2-1
Before going any further, it’s important to confirm that FL-FABRICA is connected to FL-3PAR01
FL-FABRICA# show fcalias | inc 'fl_3par01'
fcalias name fl_3par01_011 vsan 10
fcalias name fl_3par01_111 vsan 10
fcalias name fl_3par01_211 vsan 10
fcalias name fl_3par01_311 vsan 10
The result above shows that there are four (4) paths to fl_3par01 (note: we use lowercase names in Cisco configs as a standard). Run the same command without filter to check pwwn associations of the FL-3PAR01 SAN. Be advised that this list will most likely be long. Have patience in the manual process of scanning through the values to derive at the desired information.
fabric-a(config)# show fcalias
-- Truncated for brevity --
-- Many records omitted --
fcalias name fl_3par01_011 vsan 10
pwwn xx:xx:xx:xx:xx:xx:xx:xx
fcalias name fl_3par01_111 vsan 10
pwwn xx:xx:xx:xx:xx:xx:xx:xx
fcalias name fl_3par01_211 vsan 10
pwwn xx:xx:xx:xx:xx:xx:xx:xx
fcalias name fl_3par01_311 vsan 10
pwwn xx:xx:xx:xx:xx:xx:xx:xx
Next, check for interface statuses with the focus of verifying port fiber channel 4/1
FL-FABRICA# show interface br
-------------------------------------------------------------------------------
Interface Vsan Admin Admin Status SFP Oper Oper Port
Mode Trunk Mode Speed Channel
Mode (Gbps)
-------------------------------------------------------------------------------
fc4/1 10 FX on down swl F 8 --
fc4/2 1 FX on up swl F 8 --
fc4/3 10 FX on up swl F 8 --
-- Truncated for brevity --
-------------------------------------------------------------------------------
Interface Status Speed
(Gbps)
-------------------------------------------------------------------------------
sup-fc0 up 1
-------------------------------------------------------------------------------
Interface Status IP Address Speed MTU
-------------------------------------------------------------------------------
vsan1 down -- 1 Gbps 1500
vsan10 up -- 1 Gbps 1500
-------------------------------------------------------------------------------
Interface Status IP Address Speed MTU
-------------------------------------------------------------------------------
mgmt0 up 10.10.8.1/24 1 Gbps 1500
If the target interface status is down (shutdown mode), then it will be necessary to change it to up (no shutdown mode) so that the directly attached HBA’s WWPN would register with the MDS fabric.
FL-FABRICA# conf t
Enter configuration commands, one per line. End with CNTL/Z.
FL-FABRICA(config)# int fc4/1
FL-FABRICA(config-if)# no shut
FL-FABRICA(config-if)# exit
FL-FABRICA(config)# exit
Check for Port Name (WWPN) association toward interfaces
# Checking the specific interface fc4/1
FL-FABRICA# show flogi database | inc 'fc4/1'
fc4/1 1 0xc70100 10:00:d0:67:xx:xx:xx:xx 20:00:d0:67:xx:xx:xx:xx
# Checking all interface associations
FL-FABRICA# show flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
fc4/1 10 0x8c0000 20:11:00:02:xx:xx:xx:xx 2f:f7:00:02:xx:xx:xx:xx
-- Truncated for brevity --
sup-fc0 10 0x8c0dc0 10:00:00:0d:xx:xx:xx:xx 20:00:00:0d:xx:xx:xx:xx
Total number of flogi = 109.
Check to see whether fl_esx007 has been configured on this MDS fabric.
fabric-a# show zoneset active | inc 'fl-esx007'
zone name fl_esx007_001-ca_3par01_011 vsan 10
zone name fl_esx007_001-ca_3par01_111 vsan 10
zone name fl_esx007_001-ca_3par01_211 vsan 10
zone name fl_esx007_001-ca_3par01_311 vsan 10
The result above indicates that zoneset associations for node fl_esx007 port 001 (a host in Florida) has been incorrectly configured with ca_3par01 (a MDS fabric in California). Thus, it will be necessary to delete these zones as a precursor to starting over.
FL-FABRICA# conf t
Enter configuration commands, one per line. End with CNTL/Z.
FL-FABRICA(config)# no zone name fl_esx007_001-ca_3par01_011 vsan 10
FL-FABRICA(config)# no zone name fl_esx007_001-ca_3par01_111 vsan 10
FL-FABRICA(config)# no zone name fl_esx007_001-ca_3par01_211 vsan 10
FL-FABRICA(config)# no zone name fl_esx007_001-ca_3par01_311 vsan 10
FL-FABRICA(config)# exit
FL-FABRICA#
Check to see whether WWPN has been associated with VSAN 10
FL-FABRICA# show fcalias vsan 10 | inc '10:00:ba:4e:xx:xx:xx:xx'
pwwn 10:00:ba:4e:xx:xx:xx:xx
The result above shows that FL-ESX007 WWPN has been configured to associate with VSAN 10. Thus, a repeat of re-association is unnecessary. For purposes of demonstration, we shall apply the prepared MDS Zoning template as shown in step 1C to observe any anomalies.
FL-FABRICA# config t
Enter configuration commands, one per line. End with CNTL/Z.
FL-FABRICA(config)# fcalias name fl_esx007_001 VSAN 10
FL-FABRICA(config-fcalias)# member pwwn 10:00:ba:4e:4e:d0:00:24
Duplicate member
FL-FABRICA(config-fcalias)# exit
FL-FABRICA(config)# zone name fl_esx007_001-fl_3par01_011 VSAN 10
FL-FABRICA(config-zone)# member fcalias fl_esx007_001
FL-FABRICA(config-zone)# member fcalias fl_3par01_011
FL-FABRICA(config-zone)# exit
FL-FABRICA(config)# zone name fl_esx007_001-fl_3par01_111 VSAN 10
FL-FABRICA(config-zone)# member fcalias fl_esx007_001
FL-FABRICA(config-zone)# member fcalias fl_3par01_111
FL-FABRICA(config-zone)# exit
FL-FABRICA(config)#
FL-FABRICA(config)# zone name fl_esx007_001-fl_3par01_211 VSAN 10
FL-FABRICA(config-zone)# member fcalias fl_esx007_001
FL-FABRICA(config-zone)# member fcalias fl_3par01_211
FL-FABRICA(config-zone)# exit
FL-FABRICA(config)#
FL-FABRICA(config)# zone name fl_esx007_001-fl_3par01_311 VSAN 10
FL-FABRICA(config-zone)# member fcalias fl_esx007_001
FL-FABRICA(config-zone)# member fcalias fl_3par01_311
FL-FABRICA(config-zone)# exit
FL-FABRICA(config)#
FL-FABRICA(config)# zoneset name ZoneSet01 VSAN 10
FL-FABRICA(config-zoneset)# member fl_esx007_001-fl_3par01_011
FL-FABRICA(config-zoneset)# member fl_esx007_001-fl_3par01_111
FL-FABRICA(config-zoneset)# member fl_esx007_001-fl_3par01_211
FL-FABRICA(config-zoneset)# member fl_esx007_001-fl_3par01_311
FL-FABRICA(config-zoneset)# exit
FL-FABRICA(config)#
FL-FABRICA(config)# zoneset activate name ZoneSet01 VSAN 10
Zoneset activation initiated. check zone status
FL-FABRICA(config)# copy running-config startup-config
[########################################] 100%
Copy complete.
Verify that FL-ESX007 has been associated with FL-3PAR01
fabric-a(config)# show zoneset active | inc 'rpsesxi02b'
zone name fl_esx007_001-fl_3par01_011 vsan 10
zone name fl_esx007_001-fl_3par01_111 vsan 10
zone name fl_esx007_001-fl_3par01_211 vsan 10
zone name fl_esx007_001-fl_3par01_311 vsan 10
fabric-a(config)# show zoneset active | inc 'rpsesxi02b_001-mph3pss001'
zone name fl_esx007_001-fl_3par01_011 vsan 10
zone name fl_esx007_001-fl_3par01_111 vsan 10
zone name fl_esx007_001-fl_3par01_211 vsan 10
zone name fl_esx007_001-fl_3par01_311 vsan 10
To view additional details, run the same command without filtering. Scroll toward the bottom of the output to view the latest entries
fabric-a(config)# show zoneset active
zoneset name Default_zoneset vsan 1
zone name Default_zone vsan 1
pwwn 50:01:43:80:xx:xx:xx:xx
pwwn 50:0a:09:84:xx:xx:xx:xx
pwwn 50:0a:09:84:xx:xx:xx:xx
-- Truncated for brevity --
-- Many records omitted --
zoneset name ZoneSet01 vsan 10
-- Truncated for brevity --
-- Many records omitted --
zone name fl_esx007_001-fl_3par01_011 vsan 10
* fcid 0x8c0000 [pwwn 20:11:00:02:xx:xx:xx:xx]
* fcid 0x8c1999 [pwwn 10:00:ba:4e:xx:xx:xx:xx]
zone name fl_esx007_001-fl_3par01_111 vsan 10
* fcid 0x8c0001 [pwwn 21:11:00:02:xx:xx:xx:xx]
* fcid 0x8c9999 [pwwn 10:00:ba:4e:xx:xx:xx:xx]
zone name fl_esx007_001-fl_3par01_211 vsan 10
* fcid 0x8c0002 [pwwn 22:11:00:02:xx:xx:xx:xx]
* fcid 0x8c9999 [pwwn 10:00:ba:4e:xx:xx:xx:xx]
zone name fl_esx007_001-fl_3par01_311 vsan 10
* fcid 0x8c0003 [pwwn 23:11:00:02:xx:xx:xx:xx]
* fcid 0x8c9999 [pwwn 10:00:ba:4e:xx:xx:xx:xx]
This is the final verification that the four paths are active as indicated by the asterisks “*” signs
fabric-a# show zoneset active | inc '10:00:da:3c:7b:00:00:00'
* fcid 0x0b0085 [pwwn 10:00:da:3c:7b:00:00:00]
* fcid 0x0b0085 [pwwn 10:00:da:3c:7b:00:00:00]
* fcid 0x0b0085 [pwwn 10:00:da:3c:7b:00:00:00]
* fcid 0x0b0085 [pwwn 10:00:da:3c:7b:00:00:00]
Summary of useful show commands
### Useful show commands ###
# show fcalias vsan 10 # Displays the long output of all VSAN 10 configs
# show zoneset active | inc '10:00:da:3c:7b:00:00:00' # Checks active zoneset for matches of a specific wppn
# show flogi database | inc '10:00:da:3c:7b:00:00:00' # Shows the Fabric Login database for matches of a specific wppn
# show zoneset | inc 'fl_esx007' # Check zonesets for matches of the new servername config
# show fcalias name fl_esx007_001 VSAN 10 # Check VSAN 10 for any entries of the specific fcalias
The asterisk symbol next to Fiber Channel ID (fcid) indicates that the connection is active. The newly configured zones should have their PWWN set as active. Also, the fcid 0x8c1999 [pwwn 10:00:ba:4e:xx:xx:xx:xx] must match the fcalias name fl_3par01_011 vsan 10 record gathered at the initiation step of this configuration procedure. Once these two things established, we may reasonably assume that the networking portion of ESX to SAN connectivity is “good to go” for ESXi HBA Port 1 of 2.
ESXi HBA Port 2 of 2 configuration would just be a repeat of “Step 2” (this section).