1

Hi Everyone,
I am a bit stumped. I have a DHCP issue where new devices are struggling to obtain IP addresses from the DHCP server. I have checked connections and ensured that I can ping the DHCP server, and that the server and client are receiving ARP and DORA packets.

However, for some reason, multiple devices on my network are not able to obtain IP addresses. Instead, they are defaulting to APIPA addresses, and even releasing and renewing do not appear to work reliably. It feels a bit like playing a slot machine when getting an IP, leaving them without any connectivity.

The network is a flat /16. I do have a bunch of Bad_Address entries, but all of the Macs are invalid with only 4 segments, and none of them are static workstations since. I made every workstation a reservation once this issue popped up, which has proven a good workaround for static clients.

But for devices or NICs like laptops with docks that are not used every day, they need to attempt to grab IPs and fail 3-5 times before grabbing one that works with manual intervention.

Any assistance is appreciated.

In the meantime, I am going to try and grab some new Wireshark captures and see if I missed anything within the DORA handshake or ARP packets.

Let me know if more information is needed.

Thanks,
Jeremy

7 Spice ups
2

Let’s start with some system-wide info:
What server-level are your domain controllers running on?
How long ago did the problem start and what changed since it last worked?
What all have you attempted and what do your logs say?
DHCP is from the server, but what about DNS (it’s always DNS…)?

3 Spice ups
3

Hey @veremyth

I ran into a similar situation at a previous job, and we spent way too long chasing DHCP and DNS ghosts before realizing it was the Core that was acting up.

Everything seemed fine—DHCP server reachable, traffic flowing—but clients kept grabbing APIPA addresses and piling up Bad_Address entries. Eventually records would start hanging on DNS. We could see DNS from the endpoints, but it would not hand anything out. It hit mobile and docked devices first but then started hitting desktops especially after reboots. We had to create DHCP reservations for the devices that absolutely needed stable connections. That held thing together…mostly.

I brought in outside Engineers from our VAR, and they could not find anything either.

Eventually, we spotted weird behavior coming from the core that caused address conflicts. We tried clearing the ARP cash but that did not work either. We ended up rebooting our core switch. That completely cleared things up. After the reboot, DHCP started handing out leases like clockwork again.

In the end it was our Network Core playing middleman too aggressively with DHCP Helper. Let us know what Wireshark shows—I’m curious to see if you get the same result as we did.

6 Spice ups
4

Piggybacking a little on Jay’s post, what is serving DHCP addresses? I see no mention of whether it’s a Windows server, Linux, Router/Gateway, etc.

3 Spice ups
5

Is your DHCP scope’s address pool actually handing out only good addresses? There is no overlap with any static addresses, etc? Lease duration is set to an acceptable value?

If there is an overlap with static addresses, have you turned on “Conflict Detection Attempts” in Windows DHCP server? This will cause the server to ping an address before offering it - if it gets a response, it moves to the next IP to test/offer.

5 Spice ups
6

I have seen similar behavior due to various things:

  1. somebody put another device on the network that was trying to act as a DHCP server (a home router).
  2. Someone manually assigned a device the same IP as the DHCP server.
2 Spice ups
7

What server-level are your domain controllers running on?

Active Directory Domains and Trusts states that DC01(Primary, DHCP) is running on Windows2016Forest and Windows2016Domain. Both DHCP and DNS are Active Directory-integrated with replication set across all DCs in the domain in this case each DC is also a DNS server.

How long ago did the problem start, and what changed since it last worked?

Around April. To my knowledge, nothing was added or changed.

What all have you attempted, and what do your logs say?

  • Cleared bad entries and rebooted the domain controller to check for persistence.
  • Rebooted both DCs outside of normal maintenance reboots.
  • Compared ARP table to invalid entries.
  • Flushed the ARP cache, reconciled scope, and cleared bad entries again.
  • Double-checked exclusion scopes and local devices for static assignments within scope. Including Workstations, Printers, IoT (HVAC), and mobile devices. All are within expected exclusions if static or have DHCP enabled.

DHCP is from the server, but what about DNS (it’s always DNS…)?

DNS is operating as expected from what I can see. No errors are standing out on the primary DNS server.

1 Spice up
8

Oh boy, the Bat signal is alight!

While this issue is not resolved for me yet. I have been given several items to try. I have replied to some. But I wanted to say thanks everyone for the suggestions so far.

The Wireshark results are below.

I have not needed to use Wireshark to check packets often. But I can see ARP “Who has” packets are transmitting alongside a few “IP is at MAC” replies and even fewer Gratuitous ARP requests.

I do see a fair few duplicate requests within the ARP packets, but ping and Arp tables come up empty when looking for those MACs

The DORA packets look okay but out of order? I don’t know if its normal for DORA packets to repeat but Request and ACK packets far outnumber the Discover and Offer requests on the client machine. It does make some sense though… Instead of repeating the whole process just request again until a valid IP is found… In this case it is taking WAY too many requests to find a valid IP.

1 Spice up
9

Both of these are not impossible but I have checked all hardware on the premises that I am aware of and everything checks out.

Wireshark capture does not have any packets from other DHCP servers and the IP and MAC addresses match that of the expected DHCP server.

1 Spice up
10

Conflict detection is on and set for 3 attempts.

The ranges for exclusions and inclusions check out as well. DHCP is attempting to hand out valid IPs from the 50 or so packets I have glanced at they are all within proper scope.

1 Spice up
11

Going to schedule a reboot of the core tonight. Need to run through some red tape first as its going to send alerts off.

1 Spice up
12

What’s your monitor/alert solution? You should be able to schedule a maintenance window where it doesn’t alert anything for a given timeframe (including until manually started…just don’t forget to start it back up).

13

Yes , sometimes, remember that they are different packet types, the Discover message is a broadcast packet, so it really only needs to go out once, everyone sees it, the Offer packet is unicast, so it gets routed (hopefully) and only the sender should see it..

What you don’t want to see often is Offer followed by more than one Request+Ack, that means something collided between the time the DHCP server found a free IP and the time the client said “ok, gimme that one”
You also usually don’t want to see duplicate Offers, that means you have more than one DHCP server handing out things..

Repeated Discover packets from the same client usually mean the switch is not forwarding broadcast traffic reliably/properly, or it needs tags to route it through a multi level VLAN and doesn’t have them ..

2 Spice ups