1

Hey all,

I am setting up a new esxi8 host on a Dell xc760 server. The build is 24674464. The installation went smoothly, and I attached it to my vCenter, then all of a sudden i started experiencing “no healthy upstream” errors. I put the host in maintenance mode, and rebooted it, but vCenter couldn’t “find” it. I tried to login to the host directly, and saw “no healthy upstream” when i tried to login directly.

I saw some posts about people seeing the error in vCenter. The fixes were to verify the DNS and NTP were setup correctly. i made some changes on my vCenter server to verify a correct configuration, and it connected!

I had to put my esxi host in maintenance mode and reboot it for another error I was getting, and now I am back to the same scenario.

There was a second xc760 that I setup at the same time, and it is not experiencing any problems.

Let me know if anyone has experience fixing this issue.

4 Spice ups
2

If there’s no VM’s on this troubled host yet, what’s the harm in starting over? I’m guessing that changing the DNS and NTP fixed it but maybe the reboot after reset it again, like they didn’t save, so you could also check there.

2 Spice ups
3

vCentre needs to be higher than your ESXi version or at least be able to support your ESXi version.

Your subject mentions ESXi but this isn’t an ESXi issue, it’s a VCSA issue, common causes are not enough resources or services are not running. (I’ve fixed your subject)

What spec is your VCSA and have you confirmed all services are up and running?

VCSA on reboot can take an excessive amount of time to be ready too, depending on spec, VM count and underlying host resources.

Your profile says you have a VCP, wasn’t this covered on the exam?

2 Spice ups
4

If the problem is that I cannot login directly to the esxi host, not involving vCenter, I’m not sure vCenter would be the issue?

1 Spice up
5

Yes, I could start over, but I am curious why this started in the first place, and what would keep it from happening again?

1 Spice up
6

The fact that this is only happening on one esxi host doesn’t point to a VCSA problem for me yet.

I haven’t rebooted the VCSA a this time. However, I did have some struggles upgrading from VCSA7 to 8. TD Synnex spent a long time figuring out why, and tweaking things to get the upgrade to work. It’s possible something buggy could be left over from that process.

1 Spice up
7

vCenter and esxi are on the latest version 8 builds.

1 Spice up
8

To clarify, are you saying you get this error on the host directly, NOT the VCSA?

I can’t say I’ve ever seen that in 25+ years, but I’d check your logs, either services are not starting properly, ports are not open or you may have an IP conflict.

2 Spice ups
9

If you’re running on the bleeding edge, are you surprised when you get cuts on your feet? Check known issues to see if there’s anything similar to what you’re experiencing here. Security reasons aside, why would you be on the newest build of anything?

1 Spice up
10

Yes, the error occurs attempting to login to the esxi host with no vCenter involved.

Security/hacking concerns from broadcom were behind the version decision.

1 Spice up
11

I checked the logs, but didn’t see anything yet. Is there any one logfile that might be the most beneficial to target?

1 Spice up
12

Yes, the error occurs attempting to login to the esxi host with no vCenter involved.
Security/hacking concerns from broadcom were behind the version decision.

1 Spice up
13

So either your services are not starting or they are stuck.

SSH to the host and check the following services are running

/etc/init.d/hostd status
/etc/init.d/vpxa status

Restart (or start) them if they are not

/etc/init.d/hostd restart
/etc/init.d/vpxa restart

Look at /var/log/hostd.log and /var/log/vpxa.log for errors.

2 Spice ups
14

I recently had this error a previous contract and it was related to a certificate issue. I don’t remember all the details. I do recall that the host CLI offers some ‘common’ certificate refresh options, but I ended up having to work with certs that were not one of those.

There were two scripts that I pulled from vmware support site that flushed and reapplied all the certificates, not just the 2-3 that the built-in tool handled.

I wish I had more details, all those notes stayed with that company when I left.

Also, as others have mentioned, definitely watch the timing on your attempt to connect. Once the host finishes booting you need to give it another couple of minutes for the services to fully come alive.

2 Spice ups
15

Certificates part, though if this is a new install, I can’t see how it would be certificates

/sbin/generate-certificates
/etc/init.d/hostd restart
2 Spice ups
16

I could not login to the host, so I could not enable ssh.

1 Spice up
17

Good news! Today I was able to connect to the host normally, without any errors. I was able to look at the hostd and vpxa logs, and identify when the host started booting up. I opened a case with VMware to see if they can give feedback on what they see, and how to keep it from occurring in the future..

Thanks everyone for your feedback !!

2 Spice ups
18

this is what idrac is useful for.

2 Spice ups
19

yep, definitely. Thanks for the reminder.

1 Spice up