We have several virtual windows 2012 R2 servers hanging, all the servers are running under ESXI 6.03. There are 6 virtual servers in 3 different hosts.

  1. When the server hanging I checked the vmware tools is “Not running (Current)” in the summary tab of the vSphere. after reboot server the server works again, but after 3 or 4 days the issue comes again.

  2. As I got information from web site that there suggest me to uninstall and reinstall vmware tools, I tried to uninstall vmware tools, after uninstalled vmware tools the OS crashing and can’t recoverable.

  3. I am also tried to install latest version vmware tools 10.2, but the same issue still there.

Thanks You!

Jack

6 Spice ups

Was this VM a P2V by any chance?

Hi Gary, Thanks for reply! There are virtual servers, I’ve migrated one virtual server to another host, but there is the same issue. Thanks Jack

Yes but were the vm’s created by doing a p2v - i.e converting a physical to virtual?

Actually I am not sure where is the vm came from, our global engineer created same vm and deploying to different site, I am not sure the vm is done by P2V or not, anyway other sites are not issue.

The reason that I’m asking is that the sorts of issues you’re describing are often the signs that a machine has been P2V’ed and is hitting issues because of stuff left beyond from what it was physical. You might want to spin up a fresh VM and migrate the apps and data.

Yes, that is a option, finally if we still can’t fix the issue we have to do it. But one thing very interesting is if one server have issue I do think this is reasonable, now six servers have same issue, and all of the server have worked more than one year already.

So what changed? Clearly something changed when those servers went from one year of up time to crashing every few days. Also, if they had a years up time it means that they weren’t patched so I immediately suspect that they’ve had a ton of patches installed.

1 Spice up

Do you have a resource issue? Are these severs all sharing a SAN?

Yes, we patch the server timely if Microsoft has new patch, I don’t know why the different sites they are doing same thing but no issue, in our site we have more than 20 servers they are also patching the same thing but no issue; As I checked that every time when issue comes up I can see the vmware tools is not running, so, I am not sure this is ESXI issue or OS issue.

We didn’t setup SAN. when issue happens I can see the CPU utilized 100%, but I believe that because the vmware tools stops to work.

Check what type of NIC is on the VM. If it’s the E1000 then you might try switching to the vmxnet type. I have had a few VMs loaded from OVA templates that came with E1000 that would go offline for no reason. Once I converted to the native VMware nic (vmxnet) the VM stays online fine.

It’s VMXNET already. Thanks.

Today six servers were hanging again and at the same time. Attached is a logger from vm, would some one help to analysis for what’s going on. Thanks you!

vmware-5.log (516 KB)

what is in the event logs?

The event logger was attached, but I don’t how to identify the issue. one thing interesting is in the event said: 2018-02-25T02:37:01.060Z| vmx| I125+ The VMware Tools package is not running in this virtual machine. The package might be necessary for the guest operating system to run at resolutions higher than 640x480 with 16 colors. The package provides significant performance benefits as well. To install it, choose VM > Install VMware Tools鈥?after the guest operating system starts.

I am not sure this related to the issue.

What about the system event log?

What process is running at 100% inside the VM?

System event is not thespecial,i cant found any related information.
Yes,when OS hanging I saw the CPU utilization in vSphere client there was 100%.

Kill the exe and then what happens?

Which’s exe you wanted to kill?