Hi, I'm having trouble getting F-Port trunking to work with NPV and FCoE for multiple VSANs and hoping someone can shed some light. My topology is very straightforward, I have N5K1 set up as an fcoe switch with npiv enabled, my second switch is N5K2 with fcoe-npv enabled. Int e1/1 is my link. Now, when I setup my interfaces (ethernet and vfc), I have absolutely no problem with a single VSAN (VSAN10). Below are the salient pieces of code and the output (you will note I have provisioned vlans 20 and 30 also and respective vsan config, but not yet using them):
N5K1:
feature fcoe
feature npiv
vlan 10
fcoe vsan 10
vlan 20
fcoe vsan 20
vlan 30
fcoe vsan 30
vsan database
vsan 10 name "fcoe-vsan10"
vsan 20 name "fcoe-vsan20"
vsan 30 name "fcoe-vsan30"
interface Ethernet1/1
switchport mode trunk
switchport trunk allowed vlan 10,20,30
spanning-tree port type edge trunk
interface vfc1
bind interface ethernet 1/1
switchport mode f
switchport trunk allowed vsan 10
no shutdown
N5K2:
feature fcoe-npv
feature lacp
feature lldp
vlan 10
fcoe vsan 10
vlan 20
fcoe vsan 20
vlan 30
fcoe vsan 30
vsan database
vsan 10 name "fcoe-vsan10"
vsan 20 name "fcoe-vsan20"
vsan 30 name "fcoe-vsan30"
interface Ethernet1/9
switchport mode trunk
switchport trunk allowed vlan 10,20,30
spanning-tree port type edge trunk
interface vfc1
bind interface ethernet 1/1
switchport mode NP
switchport trunk allowed vsan 10
no shutdown
N5K1 Outputs:
N5K1(config-if)# sh int vfc1
vfc1 is trunking
Bound interface is ethernet 1/1
Hardware is Ethernet
Port WWN is 20:00:54:7f:ee:3c:85:ff
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode isTF
Port vsan is 1
Trunk vsans (admin allowed and active) (10)
Trunk vsans (up) (10)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
603 frames input, 68788 bytes
0 discards, 0 errors
701 frames output, 96936 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Mon Aug 12 09:07:02 2013
N5K1(config-if)# sh vsan 10
vsan 10 information
name:fcoe-vsan10 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:up
N5K2 Outputs:
N5K2(config)# sh int vfc1
vfc1 is trunking
Bound interface is ethernet 1/1
Hardware is Ethernet
Port WWN is 20:00:54:7f:ee:21:3f:ff
Admin port mode is NP, trunk mode is on
snmp link state traps are enabled
Port mode is TNP
Port vsan is 1
Trunk vsans (admin allowed and active) (10)
Trunk vsans (up) (10)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
164 frames input, 26088 bytes
0 discards, 0 errors
92 frames output, 13160 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Mon Aug 12 09:17:33 2013
N5K2(config)# sh vsan 10
vsan 10 information
name:fcoe-vsan10 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:up
Okay, so we're good for VSAN 10.
Now, when I add VSANs 20 and 30 to each of the VFC interfaces on N5K1 and N5K2, such that my config now looks like this:
interface vfc1
bind interface ethernet 1/1
switchport trunk allowed vsan 10
switchport trunk allowed vsan add 20
switchport trunk allowed vsan add 30
no shutdown
I get the following (on both switches):
N5K1(config)# sh int vfc1
vfc1 is trunking
Bound interface is ethernet 1/1
Hardware is Ethernet
Port WWN is 20:00:54:7f:ee:3c:85:ff
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 10
Trunk vsans (admin allowed and active) (10,20,30)
Trunk vsans (up) (30)
Trunk vsans (isolated) ()
Trunk vsans (initializing) (10,20)
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
625 frames input, 71556 bytes
0 discards, 0 errors
743 frames output, 104668 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Mon Aug 12 09:17:21 2013
N5K-p3-1(config)# sh vsan 10
vsan 10 information
name:fcoe-vsan10 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
N5K-p3-1(config)# sh vsan 20
vsan 20 information
name:fcoe-vsan20 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
N5K-p3-1(config)# sh vsan 30
vsan 30 information
name:fcoe-vsan30 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:up
I can't get all VSANs in the UP state, and for some reason, VSAN 30 has gone into 'UP' but VSAN 10 (which was previously UP) is now 'DOWN' and VSANs 10 and 20 just say 'initialising'.
I don't think it's something I have misconfigured, more that perhaps I'm missing something? Either that, or F-Port trunking (TF) is different to what I'm thinking, and that only a single VSAN can go between the switches?
Probably me and something very simple :-)
Thanks
Dominic