Tag Archives: Problems

vSAN ROBO – A Call to Arms! [Updated / Resolved]

Hey all,

We have a problem that we are stuck against a wall with. I am going to lay it out here, and any info that anybody can provide, or even some input, would be great! Here’ goes:

We have 3 DELL R710s for vSAN at a ROBO Location. The servers have the following networking configuration

 

Management/vMotion:

vSwitch     : vStandard Switch

IP Segment  : 172.18.18.1/26 GW 172.18.18.1

VMNics      : TWO on each servers connected to Switch S1 and S2

Switch Port : ACCESS Ports (vLAN 24)

 

vSAN

vSwitch     : Tried both vSS and vDS

IP Segment  : 10.149.12.1/28 GW 10.149.x.1

VMNic       : Single port on each server connected to S1 only

Switch Port : ACCESS Ports (vLAN 12)

 

Cisco Switch Details:

Cisco WS-C6509-E with 12.2(33)SXH5 IOS running on them.

 

IGMP Snooping is ON (but querier is disabled), switch info below.

 

Vlan12 is up, line protocol is up

Internet address is 10.149.12.3/28

IGMP is disabled on interface

Multicast routing is disabled on interface

Multicast TTL threshold is 0

No multicast groups joined by this system

IGMP snooping is globally enabled

IGMP snooping CGMP-AutoDetect is globally enabled

IGMP snooping is enabled on this interface

IGMP snooping fast-leave (for v2) is disabled and querier is disabled

IGMP snooping explicit-tracking is enabled

IGMP snooping last member query response interval is 1000 ms

IGMP snooping report-suppression is enabled

 

Note: The vLAN 12 was newly created just for vSAN traffic.

 

Issue:

Can’t ping the vSAN IP segment GW (10.149.12.1) from any of the ESX hosts, can’t ping the vSAN vmk ports between ESX hosts. Network guys says he is not learning the MAC either on the switch end. At the same time, we have a windows server in the same segment as ESXi management network and it can ping the vSAN GW (10.149.12.1) just fine. Tried multicast testing using this command “tcpdump-uw -i vmk2 -n -s0 -t -c 20 udp port 12345 or udp port 23451” it’s only returning UDP packets from the same ESX hosts not from other two which is in the cluster.

 

Anyone who could possibly help out, or even if you know anyone, please share this around. Any and all input would be greatly appreciated, and I would love to buy you a beer for your troubles at VMWorld this year!

-vTimD

 

 

UPDATE:

Thanks to @vkoolaid, @chuckhollis, @DuncanYB, and @leedilworth for all of the responses and info. After reading a troubleshooting document and then checking out a VMTN article listed below, I see that querier must be enabled if IGMP Snooping is enabled. We are going to need a code upgrade for this, so this is our next path. More updates to come.

https://communities.vmware.com/message/2367348

 

 

Update 2 / Resolution:

Thank you so much to every single person who pinged me, or read, or provided some form of input. Cisco was engaged and we were able to enable the querier without a code upgrade on the switch. The 3 hosts are now visible to each other, and the vSAN is passing traffic as it should!