|
View Full Version : Oodles of 529 Logon Failures every 2:00 AM
Hello,
My Windows 2000 domain is getting an error every night at 2AM because it can't lock out the Administrator account. Yes, exactly; "why is it being told to lock out in the first place?" I don't think we're under attack because it is every night at the same time and because I have found some information which may shed some light on it.
It seems that at 2:00 AM some process happens that all of the local administrator accounts on the servers get a failed login to their local machine. The domain registers these logon failures I suppose because the machine itself is a member of the domain. The really weird thing is that the "logon type" shows as type 3, network. How can a local account have a network logon to its own machine?
More wierdness, wherever the local admin account of the server has been changed, _that_ name shows up with the failed 529. The domain name is _always_ the name of the local server, the AD domain is not referenced even once in all 200 of the 529's.
Something... is causing these failed local admin logins to happen every night at 2AM on servers. I think that's why the domain admin account is receiving a call to get locked out is; because the domain is confusing the local admin accounts with the domain admin account, and thinking that _it_ is the culprit.
The first thing we're going to do is rename the domain admin account (yes I know I should have done this a long time ago, but there are services, scheduled tasks, etc. running under that name that I have to track down and remediate before I change it).
The next thing I will do is I will check with our server team about nightly processes/tasks that may be occurring at 2AM, but I wonder if there is something in the undulations of AD itself that is triggering this, such as a master browser election.
If anyone can shed any light or has experienced something similar, I am open to any advice you could give.
Thanks a bunch!!
|
Wow, that is a massively confused situation (and large but mostly reasonable x-post I leave untouched)
I am inlining some comments that may shed some light, and hope (for your sake) that others add more.
-- Roger Abell Microsoft MVP (Windows Server System: Security) MCSE (W2k3,W2k,Nt4) MCDBA "-" <-@-.com> wrote in message news:eJybcSIjEHA.3664@TK2MSFTNGP12.phx.gbl... > Hello, > > My Windows 2000 domain is getting an error every night at 2AM because it > can't lock out the Administrator account. So you are saying that the domainname\administrator account is being (or rather the attempt is made to have it) locked out.
> Yes, exactly; "why is it being > told to lock out in the first place?" because the invalid login count threshold is reached within the time allowed, as you well know
> I don't think we're under attack > because it is every night at the same time and because I have found some > information which may shed some light on it. > Gut level feeling are often right, but sometimes wrong. Why do you not think it an attack?
> It seems that at 2:00 AM some process happens that all of the local > administrator accounts on the servers get a failed login to their local > machine. So, the process is attempting to log in with the domainname\administrator and with each machine\administrator account
> The domain registers these logon failures I suppose because the > machine itself is a member of the domain. No. The login attempts are logged where authentication is processed. For machine\administrator this is on machine, for domain\administrator this is on a domain controller
> The really weird thing is that > the "logon type" shows as type 3, network. ??
> How can a local account have a > network logon to its own machine? > Nothing strange here. If login is by use of a network based access. First, it sounds like at each machine, the process may be attempting to use in turn machine\admininstrator and also domain\administrator. This likely originates on some machine other than the one targetted, but it could orginate there and still be login type 3.
> More wierdness, wherever the local admin account of the server has been > changed, _that_ name shows up with the failed 529. This indicates that either you have not tightened the machines (and if the domain\administrator account falls into this camp, tightened the domain) so that it does not allow enumeration of accounts; or, that the process that is behind the behavior has access to a valid login so that it can enumerate account names non-anonymously. If you can query against the SAM of account info, it is not hard to know which are admins.
> The domain name is > _always_ the name of the local server, the AD domain is not referenced even > once in all 200 of the 529's. > I do not follow what that said, as it seems to say one thing and then say that it is not what was just said.
> Something... is causing these failed local admin logins to happen every > night at 2AM on servers. Yes. As they say on Mission Impossible, your task, should you choose to accept it, is . . .
> I think that's why the domain admin account is > receiving a call to get locked out is; because the domain is confusing the > local admin accounts with the domain admin account, and thinking that _it_ > is the culprit. Again, I got lost on what that was saying. "The domain admin account is receiving a call to get locked out is . . ." ??
> > The first thing we're going to do is rename the domain admin account (yes I > know I should have done this a long time ago, but there are services, > scheduled tasks, etc. running under that name that I have to track down and > remediate before I change it). > Not just the domain\administrator account, but each machine\administrator account (and, ideally not all to the same thing). Reset passwords while at it.
> The next thing I will do is I will check with our server team about nightly > processes/tasks that may be occurring at 2AM, Excellent idea, especially now that it is apparent that there are evidently admins of servers in your environment doing things of which you may have no awareness. Also, you may want to consider reviewing successful logins onto domain accounts, or onto the servers, at about the same time, or in the interval before the event begins. Do you have uplevel machines ? The event logs on uplevels will provide info on the originating IP for the failed attempts.
> but I wonder if there is > something in the undulations of AD itself that is triggering this, no, not that I can thing of, but it certainly could be programmed to do so, just not "as shipped"
> such as a > master browser election. > That is pre-AD, and is non-authenticated. My first thought is dumb backup software someone is trying out and did not configure, or something like Nessus that someone has decided would be good to turn loose at 2 am to scan about.
> If anyone can shed any light or has experienced something similar, I am open > to any advice you could give. > > Thanks a bunch!! >
Good luck. Collect the dominoes and the picture will point your nose in the right direction.
-- Roger
|
I think your best bet would be to try and track down what is happeneing at 2:00AM. See if the failed type 3 logons are originating from the same computer and then see what is happening on that computer checking Scheduled Tasks and AT tasks by entering AT on the command line and also enabling auditing of process tracking on it. It would have nothing to do with the browser elections. I beleive if the domain name and the server name are the same in the failed logon that means that the failed logon was against a local computer account rather than domain account. For domain account failed logons it may help to refer to the link below and use netlogon logging to find the computer or computers causing these failed logons. It also has a lot of good info on tracking down failed logons and common reasons why they happen. I would also suggest that you try http://www.eventid.net for Event ID 529 to see if you find anyhting helpful there. --- Steve
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/technologies/security/bpactlck.mspx http://eventid.net/display.asp?eventid=529&eventno=1&source=Security&phase=1 --- Eventid.net for ID 529.
"-" <-@-.com> wrote in message news:eJybcSIjEHA.3664@TK2MSFTNGP12.phx.gbl... > Hello, > > My Windows 2000 domain is getting an error every night at 2AM because it > can't lock out the Administrator account. Yes, exactly; "why is it being > told to lock out in the first place?" I don't think we're under attack > because it is every night at the same time and because I have found some > information which may shed some light on it. > > It seems that at 2:00 AM some process happens that all of the local > administrator accounts on the servers get a failed login to their local > machine. The domain registers these logon failures I suppose because the > machine itself is a member of the domain. The really weird thing is that > the "logon type" shows as type 3, network. How can a local account have a > network logon to its own machine? > > More wierdness, wherever the local admin account of the server has been > changed, _that_ name shows up with the failed 529. The domain name is > _always_ the name of the local server, the AD domain is not referenced even > once in all 200 of the 529's. > > Something... is causing these failed local admin logins to happen every > night at 2AM on servers. I think that's why the domain admin account is > receiving a call to get locked out is; because the domain is confusing the > local admin accounts with the domain admin account, and thinking that _it_ > is the culprit. > > The first thing we're going to do is rename the domain admin account (yes I > know I should have done this a long time ago, but there are services, > scheduled tasks, etc. running under that name that I have to track down and > remediate before I change it). > > The next thing I will do is I will check with our server team about nightly > processes/tasks that may be occurring at 2AM, but I wonder if there is > something in the undulations of AD itself that is triggering this, such as a > master browser election. > > If anyone can shed any light or has experienced something similar, I am open > to any advice you could give. > > Thanks a bunch!! > >
|
Hello, when this same thing was happening to us it was the backup agent for Veritas which was trying to authenticate a service account on all servers at once. It was a local service admin account which authenticated to the domain by creating an account with the same username / password in AD. When the AD password expired this same thing happened. We reset the AD password and set it to "never expire" and that fixed it. Hope this helped.
Lisa
>-----Original Message----- >Hello, > >My Windows 2000 domain is getting an error every night at 2AM because it >can't lock out the Administrator account. Yes, exactly; "why is it being >told to lock out in the first place?" I don't think we're under attack >because it is every night at the same time and because I have found some >information which may shed some light on it. > >It seems that at 2:00 AM some process happens that all of the local >administrator accounts on the servers get a failed login to their local >machine. The domain registers these logon failures I suppose because the >machine itself is a member of the domain. The really weird thing is that >the "logon type" shows as type 3, network. How can a local account have a >network logon to its own machine? > >More wierdness, wherever the local admin account of the server has been >changed, _that_ name shows up with the failed 529. The domain name is >_always_ the name of the local server, the AD domain is not referenced even >once in all 200 of the 529's. > >Something... is causing these failed local admin logins to happen every >night at 2AM on servers. I think that's why the domain admin account is >receiving a call to get locked out is; because the domain is confusing the >local admin accounts with the domain admin account, and thinking that _it_ >is the culprit. > >The first thing we're going to do is rename the domain admin account (yes I >know I should have done this a long time ago, but there are services, >scheduled tasks, etc. running under that name that I have to track down and >remediate before I change it). > >The next thing I will do is I will check with our server team about nightly >processes/tasks that may be occurring at 2AM, but I wonder if there is >something in the undulations of AD itself that is triggering this, such as a >master browser election. > >If anyone can shed any light or has experienced something similar, I am open >to any advice you could give. > >Thanks a bunch!! > > >. >
|
|
|
|