We have a problem that we are stuck against a wall with. I am going to lay it out here, and any info that anybody can provide, or even some input, would be great! Here’ goes:
We have 3 DELL R710s for vSAN at a ROBO Location. The servers have the following networking configuration
vSwitch : vStandard Switch
IP Segment : 172.18.18.1/26 GW 172.18.18.1
VMNics : TWO on each servers connected to Switch S1 and S2
Switch Port : ACCESS Ports (vLAN 24)
vSwitch : Tried both vSS and vDS
IP Segment : 10.149.12.1/28 GW 10.149.x.1
VMNic : Single port on each server connected to S1 only
Switch Port : ACCESS Ports (vLAN 12)
Cisco Switch Details:
Cisco WS-C6509-E with 12.2(33)SXH5 IOS running on them.
IGMP Snooping is ON (but querier is disabled), switch info below.
Vlan12 is up, line protocol is up
Internet address is 10.149.12.3/28
IGMP is disabled on interface
Multicast routing is disabled on interface
Multicast TTL threshold is 0
No multicast groups joined by this system
IGMP snooping is globally enabled
IGMP snooping CGMP-AutoDetect is globally enabled
IGMP snooping is enabled on this interface
IGMP snooping fast-leave (for v2) is disabled and querier is disabled
IGMP snooping explicit-tracking is enabled
IGMP snooping last member query response interval is 1000 ms
IGMP snooping report-suppression is enabled
Note: The vLAN 12 was newly created just for vSAN traffic.
Can’t ping the vSAN IP segment GW (10.149.12.1) from any of the ESX hosts, can’t ping the vSAN vmk ports between ESX hosts. Network guys says he is not learning the MAC either on the switch end. At the same time, we have a windows server in the same segment as ESXi management network and it can ping the vSAN GW (10.149.12.1) just fine. Tried multicast testing using this command “tcpdump-uw -i vmk2 -n -s0 -t -c 20 udp port 12345 or udp port 23451” it’s only returning UDP packets from the same ESX hosts not from other two which is in the cluster.
Anyone who could possibly help out, or even if you know anyone, please share this around. Any and all input would be greatly appreciated, and I would love to buy you a beer for your troubles at VMWorld this year!
Thanks to @vkoolaid, @chuckhollis, @DuncanYB, and @leedilworth for all of the responses and info. After reading a troubleshooting document and then checking out a VMTN article listed below, I see that querier must be enabled if IGMP Snooping is enabled. We are going to need a code upgrade for this, so this is our next path. More updates to come.
Update 2 / Resolution:
Thank you so much to every single person who pinged me, or read, or provided some form of input. Cisco was engaged and we were able to enable the querier without a code upgrade on the switch. The 3 hosts are now visible to each other, and the vSAN is passing traffic as it should!