1

Hi,

I have a bit of an issue which I am really hoping someone has an idea what it be.

We Have Windows VCenter on 6.0 and 5 ESXI Hosts on 6.0

These have been ticking along with no issues and all of a sudden hosts start to disconnect and then auto reconnects.

Initially it started on one host which was the VSphere HA Master. I put this in maintenance mode and then gave it a reboot. The problem went away but then it started on another Host.

I did the same on this one (Reboot) and then the problem went away but now it started on the different one.

I thought it might have been something to do with us upgrading our Backup solution (VRanger) to the latest version including the cVirtual appliances.

I contacted Quest and they advised to turn of the Virtual Appliance and Backup using the VRanger Server. This was to rule the Virtual Appliances.

Just this morning, I noticed a host disconnected again for a few of hours and then reconnected backup again.

The host is still accessible and all the VM’s on the disconnected host are still active. Pings works and also you can RDP into the VM’s or SSH in if they linux.

I can try and reboot the remaining hosts and see if this resolves the issue but I wanted to ask if anyone else has had a similair issue.

Any help on this would be really appreciated.

Thanks for reading :slight_smile:

5 Spice ups
2

Wow, is vRanger still going, it’s been a while since i heard that name.

Also be aware that ESXi 6.0 is no longer supported, you may be spending time fixing an issue that turns out to be a known bug, however you’ve only told us about your ESXi version and not your hosts or hardware, it could be a network issue and nothing to do with ESXi

3

Hi Rod,

Thank you for your response.

You are right. We do have support with vmware but they not entertaining us as we need to be on at least 6.7. This is something I am scoping out but I really need to get to the bottom of this issue before doing the upgrade.

I am looking at going to Veeam and have something scheduled for November this year.

All hosts are on 6.0 build 4600944 which is a dell custom build. The make and model of the hosts are dell poweredge r730 with 2 cpu’s and 128gb each. We have all our datastores on a dell sc series san connected via iscsi at 1gb .

Let me know if you need any further Information.

Thanks again

4

The first thing you should do is patch your ESXi servers, whilst unsupported, they are still very out of date.

Your patch version is from 2016, the latest patch version is ESXi 6.0 EP 25 build 15517548 released on 20th Feb 2020

You’ll also want to run the latest FW patches for your hardware too otherwise you’ll end up down a rabbit hole and may never find the cause.

1 Spice up
5

Check which NICs your mgmt networks are on ? They shd be on a vswitch that have at least 2 active NICs else loads may cause a long host disconnect that may not affect VMs. This was one of the bugs that exists on ESXi 6.0, 6.5u1. Please use ESXi 6.7u3 and VCSA 6.7 as well.

This was in one of the bug list but its so long ago. I will try searching my email for the KB next week as I am on vacation. The fix is really not worth the time as you can upgrade to ESXi 6.7 with VCSA faster than applying the fixes to ESXi and vCenter.

6

Thanks guys for your comments. I will start planning upgrading the windows vcenter from 6.0 to 6.7u3 and I can see there is a direct upgrade/migration . I’ve seens a few YouTube videos regarding it and it looks pretty straightforward.

My concern is the rollback. We currently backup the physical windows vcenter to arcserve and I was think of doing a BMR to a physical server onto a spare dell box and leave it off network just incase things go tilt.

I can’t see anything in terms of KB about rolling it back.

Thanks again.

7

What edition of VMware are you using ?

If you are using HP or Dell servers, download the 6.7 ISO from HP or Dell else use VMware ISO for other servers and run the ESXi installer.

Install VCSA 6.7 instead of vCenter (save 1x Windows server OS as well). Then use that to migrate settings etc from vCenter.

2 Spice ups
8

Hi Adrian,

All the esxi hosts are vmware 6.0 using the dell iso custom build. I’ll get them downloaded.

I guess I have to upgrade vcenter first.
Is it best using the migration / upgrade tool for Windows to vsca.
Oh yes. Boss will be happy with one less windows license and a physical box in the server room :slight_smile:

9

While upgrading/migrating from Windows to VCSA 6.7U3 is pretty straightforward https://www.starwindsoftware.com/blog/how-to-migrate-vmware-vcenter-from-windows-to-vcsa-6-7 I would highly recommend you spin a fresh new VCSA alongside the current Windows-based vCenter and move ESXi hosts from the old vCenter to the new one nuking the old vCenter afterward. That would make sure you don’t inherit any old issues, back compatibility rubbish, and tons of logs and trash from the old instance. After that part is done, you should be able to upgrade your hosts using VCSA VUM. If you are brave enough, reinstalling ESXi hosts to 6.7U3 from scratch is an even better option.

1 Spice up
10

You need to tag a person by using @adrian_ych ​ (the @).

It depends how complex is your setup ? If Dell…then onr idiot-proof method is get the 6.5 Dell ISO & 7.0 Dell ISO, upgrade ESXi from 6.5 then 7.x, after the ESXi are done, just add VCSA 7.x as it will pickup most settings from ESXi.

11

i Agree with @rod-it ​ you really need to upgrade/update and also update to the latest patch on your Hardware. I believe it is the safest way to go befor doing anything else. that way the issue won’t become another, and another and yet another different or added issue.

12

@adrian_ych
I found the root cause of the problem.

A newly built vm had 2 physical nics on it. Our vmware environment didn’t like this.

I turned the vm off and I got no disconnections this morning. Panic over! I’ll just have to use 1 nic and assign it the 2 ip addresses required.

This doesn’t excuse the fact I need to upgrade. The compatibility matrix does show our Dell PE R730’s will accommodate up to version 7 with the latest update. Don’t need to buy more kit which is good news.

Thank you so much to you all for all your comments and suggestions.

It shows, you not alone when you hitting a brick wall :slight_smile: :grinning:

1 Spice up
13

Those symptoms look rather like a trivial IP address conflict, actually, since ESXi virtual machines are perfectly capable of running multiple network adapters without issues. Have you checked the IP addresses of the VMs and affected hosts?