In this post and based on the below topology, we will elaborate with authentic configurations, on three main subjects:
- Virtual Port Channel
- Fabric extenders
- Fiber Channel over Ethernet to End Host Configuration
- Configure VPC domain 77 between N5K1 (Primary) and N5K2 using Port Channel 33 in active mode as vpc peer-link and use management interface as vpc peer-keepalive.
- Configure FEX 100 and FEX 199 in conjunction with vpc100 and vpc199 according to the diagram.
- Configure the C-220 servers port0 and port1- use vlan100, vlan199, vsan100 and vsan199 for the fcoe configuration.
Step1: Configure management IP addresses N5K1 (10.1.1.93/24) and N5K2 (10.1.1.94/24)
N5K1# show run int mgmt 0
!Command: show running-config interface mgmt0
!Time: Thu Dec 16 16:59:59 2010
version 6.0(2)N2(2)
interface mgmt0
ip address 10.1.1.93/24
N5K2# show run int mgmt 0
!Command: show running-config interface mgmt0
!Time: Thu Dec 16 17:17:53 2010
version 6.0(2)N2(2)
interface mgmt0
ip address 10.1.1.94/24
N5K1# ping 10.1.1.94 vrf management
PING 10.1.1.94 (10.1.1.94): 56 data bytes
64 bytes from 10.1.1.94: icmp_seq=0 ttl=254 time=0.922 ms
64 bytes from 10.1.1.94: icmp_seq=1 ttl=254 time=0.715 ms
64 bytes from 10.1.1.94: icmp_seq=2 ttl=254 time=0.71 ms
64 bytes from 10.1.1.94: icmp_seq=3 ttl=254 time=0.616 ms
64 bytes from 10.1.1.94: icmp_seq=4 ttl=254 time=0.708 ms
--- 10.1.1.94 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.616/0.734/0.922 ms
Step2: Configure Port Channel 33 LACP between N5K1 and N5K2 (e1/10-11 both sides)
N5K1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
N5K1(config)# feature lacp
N5K2# conf t
Enter configuration commands, one per line. End with CNTL/Z.
N5K2(config)# feature lacp
N5K2(config)#
Add interfaces e1/10-11 on both N5K1 and N5K2 to Port Channel 33
N5K1(config)# int e1/10-11
N5K1(config-if-range)# channel-group 33 mode active
N5K1(config-if-range)#
N5K1(config-if-range)# show port-channel database
port-channel33
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/10
Age of the port-channel is 0d:00h:07m:51s
Time since last bundle is 0d:00h:07m:51s
Last bundled member is Ethernet1/11
Ports: Ethernet1/10 [active ] [up] *
Ethernet1/11 [active ] [up]
N5K1(config-if-range)#
N5K2(config)# int e1/10-11
N5K2(config-if-range)# channel-group 33 mode active
N5K2(config-if-range)#
N5K2(config-if-range)# show port-channel database
port-channel33
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/10
Age of the port-channel is 0d:00h:01m:09s
Time since last bundle is 0d:00h:01m:09s
Last bundled member is Ethernet1/11
Ports: Ethernet1/10 [active ] [up] *
Ethernet1/11 [active ] [up]
N5K2(config-if-range)#
Step3: Configure VPC domaine 77 between N5K1 and N5K2
N5K1(config)# show run vpc
!Command: show running-config vpc
!Time: Thu Dec 16 17:43:52 2010
version 6.0(2)N2(2)
feature vpc
vpc domain 77
role priority 1
peer-keepalive destination 10.1.1.94
delay restore 150
interface port-channel33
vpc peer-link
N5K1(config)#
N5K2(config)# show run vpc
feature vpc
vpc domain 77
peer-keepalive destination 10.1.1.93
delay restore 150
interface port-channel33
vpc peer-link
N5K2(config)#
Check the VPC status
N5K1(config)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
N5K1(config)#
N5K2(config)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
N5K2(config)#
Step4: Configure FEXes with VPC100 and VPC199.
N5K1(config-if)# show cli history
feature fex
int e1/1
channel-group 100
int po 100
switchport mode fex-fabric
fex associate 100
vpc 100
int e1/3
channel-group 199
int po 199
switchport mode fex-fabric
fex associate 199
vpc 199
N5K2(config-if)# show cli history
feature fex
int e1/1
channel-group 199
int po 199
switchport mode fex-fabric
fex associate 199
vpc 199
int e1/3
channel-group 100
int po 100
switchport mode fex-fabric
fex associate 100
vpc 100
Check the status FEXes, VPC 100 and VPC199 status
N5K1(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2232PP-10GE SSI1751051F
199 FEX0199 Online N2K-C2232PP-10GE SSI175105D7
N5K1(config-if)#
N5K1(config-if)#
N5K1(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 66
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
100 Po100 up success success -
199 Po199 up success success -
101376 Eth100/1/1 down* Not Consistency Check Not -
Applicable Performed
101377 Eth100/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
101390 Eth100/1/15 up success success 1
................... [Omitted input]..........................
202752 Eth199/1/1 down* Not Consistency Check Not -
Applicable Performed
202753 Eth199/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
202766 Eth199/1/15 up success success 1
................... [Omitted input]..........................
202783 Eth199/1/32 down* Not Consistency Check Not -
N5K2(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2232PP-10GE SSI1751051F
199 FEX0199 Online N2K-C2232PP-10GE SSI175105D7
N5K2(config-if)#
N5K2(config-if)#
N5K2(config-if)#
N5K2(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 66
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
100 Po100 up success success -
199 Po199 up success success -
101376 Eth100/1/1 down* Not Consistency Check Not -
Applicable Performed
101377 Eth100/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
101390 Eth100/1/15 up success success 1
................... [Omitted input]..........................
101407 Eth100/1/32 down* Not Consistency Check Not -
Applicable Performed
202752 Eth199/1/1 down* Not Consistency Check Not -
Applicable Performed
202753 Eth199/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
202766 Eth199/1/15 up success success 1
................... [Omitted input]..........................
202783 Eth199/1/32 down* Not Consistency Check Not -
Applicable Performed
N5K2(config-if)#
Step5: Configure the FCOE link toward C-Series server.
N5K1(config-if)# show run | sec fex
fex 100
description "FEX0100"
fcoe
N5K2(config-if)# show run | sec fex
fex 199
description "FEX0199"
fcoe
N5K1(config-if)# show vsan
vsan 100 information
name:VSAN0100 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
N5K1(config-if)# show vsan membership
vsan 100 interfaces:
vfc100
N5K1(config-if)#
N5K2(config-vsan-db)# show vsan
vsan 199 information
name:VSAN0199 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
N5K2(config-vsan-db)# show vsan membership
vsan 199 interfaces:
vfc199
N5K1(config-if)# show run int e100/1/15
interface Ethernet100/1/15
switchport mode trunk
switchport trunk allowed vlan 100
N5K1(config-if)# show run int vfc 100
interface vfc100
bind interface Ethernet100/1/15
switchport trunk allowed vsan 100
no shutdown
N5K1(config-if)#
N5K2(config-vsan-db)# show run int e199/1/15
interface Ethernet199/1/15
switchport mode trunk
switchport trunk allowed vlan 199
N5K2(config-vsan-db)# show run int vfc 199
interface vfc199
bind interface Ethernet199/1/15
switchport trunk allowed vsan 199
no shutdown
N5K2(config-vsan-db)# show int vfc 199
vfc199 is trunking
Bound interface is Ethernet199/1/15
Hardware is Ethernet
Port WWN is 20:c6:00:2a:6a:7b:5e:3f
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 199
Trunk vsans (admin allowed and active) (199)
Trunk vsans (up) (199)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
541 frames input, 98764 bytes
0 discards, 0 errors
11 frames output, 1132 bytes
0 discards, 0 errors
last clearing of "show interface" counters Thu Dec 16 18:40:16 2010
Interface last changed at Thu Dec 16 20:38:41 2010
N5K2(config-vsan-db)#
Here are the configuration from CIMC
N5K1# show run int mgmt 0
!Command: show running-config interface mgmt0
!Time: Thu Dec 16 16:59:59 2010
version 6.0(2)N2(2)
interface mgmt0
ip address 10.1.1.93/24
N5K2# show run int mgmt 0
!Command: show running-config interface mgmt0
!Time: Thu Dec 16 17:17:53 2010
version 6.0(2)N2(2)
interface mgmt0
ip address 10.1.1.94/24
N5K1# ping 10.1.1.94 vrf management
PING 10.1.1.94 (10.1.1.94): 56 data bytes
64 bytes from 10.1.1.94: icmp_seq=0 ttl=254 time=0.922 ms
64 bytes from 10.1.1.94: icmp_seq=1 ttl=254 time=0.715 ms
64 bytes from 10.1.1.94: icmp_seq=2 ttl=254 time=0.71 ms
64 bytes from 10.1.1.94: icmp_seq=3 ttl=254 time=0.616 ms
64 bytes from 10.1.1.94: icmp_seq=4 ttl=254 time=0.708 ms
--- 10.1.1.94 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.616/0.734/0.922 ms
Step2: Configure Port Channel 33 LACP between N5K1 and N5K2 (e1/10-11 both sides)
N5K1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
N5K1(config)# feature lacp
N5K2# conf t
Enter configuration commands, one per line. End with CNTL/Z.
N5K2(config)# feature lacp
N5K2(config)#
Add interfaces e1/10-11 on both N5K1 and N5K2 to Port Channel 33
N5K1(config)# int e1/10-11
N5K1(config-if-range)# channel-group 33 mode active
N5K1(config-if-range)#
N5K1(config-if-range)# show port-channel database
port-channel33
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/10
Age of the port-channel is 0d:00h:07m:51s
Time since last bundle is 0d:00h:07m:51s
Last bundled member is Ethernet1/11
Ports: Ethernet1/10 [active ] [up] *
Ethernet1/11 [active ] [up]
N5K1(config-if-range)#
N5K2(config)# int e1/10-11
N5K2(config-if-range)# channel-group 33 mode active
N5K2(config-if-range)#
N5K2(config-if-range)# show port-channel database
port-channel33
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/10
Age of the port-channel is 0d:00h:01m:09s
Time since last bundle is 0d:00h:01m:09s
Last bundled member is Ethernet1/11
Ports: Ethernet1/10 [active ] [up] *
Ethernet1/11 [active ] [up]
N5K2(config-if-range)#
Step3: Configure VPC domaine 77 between N5K1 and N5K2
N5K1(config)# show run vpc
!Command: show running-config vpc
!Time: Thu Dec 16 17:43:52 2010
version 6.0(2)N2(2)
feature vpc
vpc domain 77
role priority 1
peer-keepalive destination 10.1.1.94
delay restore 150
interface port-channel33
vpc peer-link
N5K1(config)#
N5K2(config)# show run vpc
feature vpc
vpc domain 77
peer-keepalive destination 10.1.1.93
delay restore 150
interface port-channel33
vpc peer-link
N5K2(config)#
Check the VPC status
N5K1(config)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
N5K1(config)#
N5K2(config)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
N5K2(config)#
Step4: Configure FEXes with VPC100 and VPC199.
N5K1(config-if)# show cli history
feature fex
int e1/1
channel-group 100
int po 100
switchport mode fex-fabric
fex associate 100
vpc 100
int e1/3
channel-group 199
int po 199
switchport mode fex-fabric
fex associate 199
vpc 199
N5K2(config-if)# show cli history
feature fex
int e1/1
channel-group 199
int po 199
switchport mode fex-fabric
fex associate 199
vpc 199
int e1/3
channel-group 100
int po 100
switchport mode fex-fabric
fex associate 100
vpc 100
Check the status FEXes, VPC 100 and VPC199 status
N5K1(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2232PP-10GE SSI1751051F
199 FEX0199 Online N2K-C2232PP-10GE SSI175105D7
N5K1(config-if)#
N5K1(config-if)#
N5K1(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 66
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
100 Po100 up success success -
199 Po199 up success success -
101376 Eth100/1/1 down* Not Consistency Check Not -
Applicable Performed
101377 Eth100/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
101390 Eth100/1/15 up success success 1
................... [Omitted input]..........................
202752 Eth199/1/1 down* Not Consistency Check Not -
Applicable Performed
202753 Eth199/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
202766 Eth199/1/15 up success success 1
................... [Omitted input]..........................
202783 Eth199/1/32 down* Not Consistency Check Not -
N5K2(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2232PP-10GE SSI1751051F
199 FEX0199 Online N2K-C2232PP-10GE SSI175105D7
N5K2(config-if)#
N5K2(config-if)#
N5K2(config-if)#
N5K2(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 77
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 66
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po33 up 1
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
100 Po100 up success success -
199 Po199 up success success -
101376 Eth100/1/1 down* Not Consistency Check Not -
Applicable Performed
101377 Eth100/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
101390 Eth100/1/15 up success success 1
................... [Omitted input]..........................
101407 Eth100/1/32 down* Not Consistency Check Not -
Applicable Performed
202752 Eth199/1/1 down* Not Consistency Check Not -
Applicable Performed
202753 Eth199/1/2 down* Not Consistency Check Not -
Applicable Performed
................... [Omitted input]..........................
202766 Eth199/1/15 up success success 1
................... [Omitted input]..........................
202783 Eth199/1/32 down* Not Consistency Check Not -
Applicable Performed
N5K2(config-if)#
Step5: Configure the FCOE link toward C-Series server.
N5K1(config-if)# show run | sec fex
fex 100
description "FEX0100"
fcoe
N5K2(config-if)# show run | sec fex
fex 199
description "FEX0199"
fcoe
N5K1(config-if)# show vsan
vsan 100 information
name:VSAN0100 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
N5K1(config-if)# show vsan membership
vsan 100 interfaces:
vfc100
N5K1(config-if)#
N5K2(config-vsan-db)# show vsan
vsan 199 information
name:VSAN0199 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
N5K2(config-vsan-db)# show vsan membership
vsan 199 interfaces:
vfc199
N5K1(config-if)# show run int e100/1/15
interface Ethernet100/1/15
switchport mode trunk
switchport trunk allowed vlan 100
N5K1(config-if)# show run int vfc 100
interface vfc100
bind interface Ethernet100/1/15
switchport trunk allowed vsan 100
no shutdown
N5K1(config-if)#
N5K2(config-vsan-db)# show run int e199/1/15
interface Ethernet199/1/15
switchport mode trunk
switchport trunk allowed vlan 199
N5K2(config-vsan-db)# show run int vfc 199
interface vfc199
bind interface Ethernet199/1/15
switchport trunk allowed vsan 199
no shutdown
N5K2(config-vsan-db)# show int vfc 199
vfc199 is trunking
Bound interface is Ethernet199/1/15
Hardware is Ethernet
Port WWN is 20:c6:00:2a:6a:7b:5e:3f
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 199
Trunk vsans (admin allowed and active) (199)
Trunk vsans (up) (199)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
541 frames input, 98764 bytes
0 discards, 0 errors
11 frames output, 1132 bytes
0 discards, 0 errors
last clearing of "show interface" counters Thu Dec 16 18:40:16 2010
Interface last changed at Thu Dec 16 20:38:41 2010
N5K2(config-vsan-db)#
Here are the configuration from CIMC