The Story of Active Directory and the Round Robin

In my job of developing Admin Arsenal I have the privilege of constantly creating, deleting, changing, breaking, punching, smashing, seducing, WTFing, and generally abusing Active Directory domains. In this capacity I've seen problems that probably don't crop up for the average administrator, but sometimes I see problems that are probably fairly common.

Photo by foxypar4

One such problem has to do with DNS. Periodically, after changing my domains in some way, I would suddenly lose connectivity to AD. But connectivity wouldn't be lost for long, it would suddenly reconnect some time later and stay connected for a while, but then drop off again. Rebooting the clients or the server sometimes worked, sometimes it didn't. I could still authenticate and connect to servers with my AD credentials, but I couldn't connect using any management tool including Admin Arsenal (sometimes I could connect if I went straight to a domain controller, but not always.) Then I'd rebuild my AD servers to test something else and the problem would go away. I would just chalk it up to something I did to abuse AD. But it kept happening every few months, so I decided that it must be common problem and decided to delve in.

It took me several hours to track it down, but I was determined, dam-nit! I'm not sure how I finally figured it out (probably a lucky find on a forum somewhere,) but as I mentioned it was to do with DNS. At some point in all my AD thrashing I got a phantom A record in DNS for my domain. Since it was doing round robin name resolution, I would periodically get this phantom address when my machine tried to resolve lab.adminarsenal.local. As long as that address was either in my cache or kept being served up by DNS, my connectivity would be erratic.

It just goes to show how many pieces there are to a well run network. If one of them breaks, it can be difficult to find out exactly where it is. Well, I guess it keeps us all employed.

Reprinted from The Admin Arsenal Blog.