AD Replication Woes

donmangiarelli4968 · 2024-07-08T23:37:42.222Z

Hi All,

I have 1 site out of 3 that is having replication errors. Seems to be a Kerberos issue with error -2146893022. Upon many Google searches over the last month I have come to the conclusion that all existing articles solutions on this error do not work in this situation. THe topology looks like the following:

Site 1: 1 Windows 2016 Domain Controller

Site 2: 1 Windows 2016 DC and 1 Windows 2012 DC

Site 3: 1 Windows 2012 DC (I had 1 2016 DC here but demoted it and have not been able to re-promote it)

Sites 1 & 2 are replicating AD no problem. Site 3 has an unreplicated copy of the AD and I cannot get the DC to replicate. It reports that the Windows 2016 DC in Site and Aite 1 are unreachable. I downloaded and ran the Port Query tool and it connects to both DC’s on ports 53, 88 & 389.

I had originally had a Windows 2003 server in Site1 That was the original AD server for this forest. It died and I have made sure that all of the remaining junk from that server was cleaned out of AD and DNS. Not sure what else to consider at this point. DCDIAG reports that RPC Bind fails due to Target Principal Name being incorrect and the password is bad. I have reset the password on both DC’s in Site 1 and Site 2 many times to no avail.

Each location is connected by Site-to-Site VPN

Thanks for any help,
Don

PatrickFarrell · 2024-07-09T04:47:38.866Z

How is DNS set up on your DC NICs? Is each DC pointing at another for primary and itself secondary?

Active Directory DNS Refresher Windows

It’s DNS. It’s always DNS. No, DNS is fine, it’s not DNS, trust me. It was DNS. I’m posting because in the last few weeks we’ve had a few posts that turned out to be misconfigurations with DNS in a domain environment. Things like Domain controllers pointing at themself first for DNS, or having a public DNS server on the NIC of the Domain Controller or the clients. Hopefully this will help some people get out in front of potential issues. The general rules for DNS in an AD E…

webmasterdan · 2024-07-09T19:29:14.054Z

Always check DNS on NICs as Patrick said, firstly.
Make sure that AD sites & services has the old servers removed as well.
What other errors are reported here: dcdiag /c /e

donmangiarelli4968 · 2024-07-10T06:56:05.102Z

DNS is configured correctly. The server can resolve the names of the other servers, but it still won’t replicate. I don’t want to promote the 2016 DC until I get the replication issue resolved. I have been working on this for about 2 months with no resolution. I have tried every Microsoft article and resolution that is published. For some reason it’s just this one site. I thought if I demoted the DC and re-promoted it, the issue might clear up. No dice…

Thanks for the reply. I appreciate that you took the time to reply, and I also have read those threads that you are talking about.

donmangiarelli4968 · 2024-07-10T07:01:10.294Z

Thanks Dan. DCDiag is tripping up on Kerberos errors (event ID 4), LDAP Errors 94 and 89, it fails Auth and BASC, and says that there are no host records on the working DC’s. The weird thing is that if I run DCDiag from the servers that are replicating in the other 2 sites against the same servers, all the tests pass, which leads me to believe that the problem is not with the working DC’s but this one lone DC.

PatrickFarrell · 2024-07-10T12:18:03.397Z

Any firewalls between the sites that are filtering ports or are you wide open between the sites?

donmangiarelli4968 · 2024-07-10T19:44:45.080Z

I have 3 Firewalls all connected via Site-to-site VPNs. I have tested with the Port Query Tool and all traffic is passing. All ports on remote DC’s are open an confirmed listening 53, 88 & 389

donmangiarelli4968 · 2024-07-10T19:57:47.542Z

I do have a ton of DNS errors, and dcdiag reports DNS errors as well on all 3 DC’s, so currently troubleshooting that. The weird thing is, when I run DCDIAG /Fix all of the Domain tests passed. The only command that doesn’t pass is DsGetDCName. I am perplexed, but think that DNS is hosed up on the two DC’s. BUT… DNS is resolving names to IP’s, so WTF?

So to recap:
Kerberos Errors Event ID 4 The DC is Site 3, since AD is not replicated, has an old password and the other two DC’s can’t decrypt its tickets.
DNS Errors 4015 - DNS Can’t connect to AD, Probably has to do with the Kerberos errors
I am getting some weird GP errors about Printers not being able to be applied. There is no GP trying to apply printers. When I try to open GP MC on the DC in Site 3, Access Denied. Probably has to do with Kerberos errors.
I am also getting a DCOM error saying that DCOM can’t communicate with the other DC’s, again probably has to do with the Kerberos errors.

I have spent 2 months trying to get all of this resolved. I have gone through every SetSPN article Microsoft has with no avail. I agree, this is all probably DNS related, but I am at my wits end. The only thing I can come up with is to remove DNS server from the DC and re-install it.

PatrickFarrell · 2024-07-10T20:46:31.734Z

There are a lot more required ports than that fyi.

image732×442 5.68 KB

learn.microsoft.com

Configure firewall for AD domain and trusts - Windows Server

Describes the ports that are used when you configure a trust relationship between domains.

PatrickFarrell · 2024-07-10T20:48:32.127Z

Usually when I see stuff like this it’s because domain controllers point to themselves and DNS kind of works, but doesn’t fully work. Just to explicitly confirm, your DCs point at another DC for primary and themself secondary or tertiary right? Sorry to be a stickler on asking that but when you’ve killed countless hours with people insisting DNS is fine when it wasn’t you want to absolutely rule that out.

donmangiarelli4968 · 2024-07-10T22:19:41.214Z

All DC’s point to a different DC in a different site and then to themselves 127.0.0.1

Ok some other weird stuff. If I enable IP v6 DNS works, I can resolve the domain name mydomain.loc

If I disable IP v6 *** UnKnown can’t find mydomain.loc and If I try to nslookup a computer such as the DC in another site same message. With IP V6 enabled, it works. Not sure what to make of that…

PatrickFarrell · 2024-07-10T22:27:37.883Z

Thanks for confirming

I can tell you that nothing good comes from disabling IPv6. That will certainly be a debated statement among windows admins here.

Unfortunately I can’t do testing on that one for you as I have it enabled in my environments and I don’t currently have a home lab set up for testing.

Curious as to if you leave IPv6 up for a day and see if it straightens out replication, then try disabling again and see if the problem persists?

donmangiarelli4968 · 2024-07-10T22:33:54.576Z

Another weird thing is if I disable IP v6 and point to each DC 127.0.0.1 as the DNS, it works fine too. I’m baffled…

PatrickFarrell · 2024-07-10T22:42:38.455Z

That is definitely bizarre. You always want a DC to point to another DC first. My preference is DC in same site first, DC in another site (if one exists) second, 127.0.0.1 third.

I’ve seen far too many problems to count when it’s done 127.0.0.1 first with DCs essentially falling off the domain unable to replicate.

Run this against each server and see if it spits out anything useful?

dcdiag /test:dns /s:mydc01

donmangiarelli4968 · 2024-07-10T22:44:06.990Z

SO weird that if DNS points to itself as the server, everything works, but point to another server (which is pointing to itself and resolving IP addresses) and it doesn’t work.

PatrickFarrell · 2024-07-10T22:53:42.337Z

Your zones are AD integrated right? Almost wondering if DNS didn’t replicate properly.

PatrickFarrell · 2024-07-10T22:54:44.856Z

Scratch what I said earlier about test:dns. You will need to run that from each DC itself or it will generate a lot of errors. Been a while since I’ve had to run it and just saw a ton of errors on mine when i ran it remotely. Runs error free locally.

dcdiag /test:dns

donmangiarelli4968 · 2024-07-10T22:57:48.230Z

Yup, AD integrated.

donmangiarelli4968 · 2024-07-10T22:58:53.539Z

Yeah, I get a Delegation error when running it locally, so TS that now. If I run it against the other DC’s it gets a buttload of errors

PatrickFarrell · 2024-07-10T23:21:10.789Z

Should not get any error locally. Can you share the screen shot (sanitize the private info)

This is what mine looks like:

image613×662 67.1 KB