In-Depth

You Got Hacked! Now What?

Hacks are a fact in a connected world. After discovering and expelling the intruders, you have to clean up their messes.

Sunday, 7:30 p.m. You get a call from the help desk saying that no one can log on to the domain. You connect to the VPN and give it a look. For some reason, Active Directory is refusing requests. You start investigating. It turns out the server is out of disk space. How can this be? After all, you’ve just installed a larger hard drive.

You frantically start freeing disk space on the box by combing through Windows Explorer, hoping to find the files using all the space. You move the page file to another drive. You delete old service pack backups and all temp files.

Then you stumble across a folder using 35 GB of space. Looking inside, you find thousands of MP3s and hundreds of games. Checking the files’ properties to find the owner reveals that administrators appear to own the data. How is this possible?

Find Out What Really Happened
It turns out that someone’s been using your server as a storage bin for his or her music and games. The question: Is this a hack? Or is there another explanation?

That’s what you have to find out first. Remember: Not every situation that looks like a hack actually is. For instance, if you find a copy of pcAnywhere on your server and disconnect it from the network because you think it was hacked, you may do more harm than good. Maybe it’s there because your Web developer added it to do a mission-critical upgrade from home over the VPN. Note that I’m not recommending putting remote control software on your Web servers; I’m just saying to get all the facts before you take action and be sure to communicate with everyone involved.

Here are some of the warning flags that you may have been hacked:

  • Logs showing repeated failed login, FTP and telnet attempts.
  • Finding software you didn’t place on the server like Symantec’s pcAnywhere, or the open source Rconsole or AT&T’s Virtual Network Computing (VNC).
  • Running out of disk space on servers that shouldn’t be full.
  • Constant virus outbreaks despite having anti-virus software on all of your machines.
  • Periods of high network usage at odd times.

Get a Mop and a Broom
If you discover that your network’s been compromised, take immediate corrective action. That means first locking out the attacker. The easiest way to do that is by disconnecting your computer from the network. Then comes the process of finding out where the hacker’s been and cleaning up the messes he or she has made. Follow the footprints, which may have been left in the some of these places:

  • New user accounts.
  • Additions to group membership.
  • Changed user rights.
  • Programs configured to start automatically.
  • Altered, added or deleted file, share and registry permissions.
  • Altered, added or deleted Web permissions.
  • Added files and features.

We’ll go through these areas one by one. As you progress, remember to make copies of all logs and to document everything that you find. In addition to electronic notes, I recommend keeping a physical notebook for all your servers; this way your notes can’t be easily erased.

Change Passwords!
Before doing anything else, change the passwords for all user accounts on the machines in question, starting with the administrator account. If a hacker has gained administrative access to your system, assume he or she has compromised all the system’s passwords. Tons of easily accessible tools exploit local and domain databases once you have administrative rights.

Look for New User Accounts and Additions to Group Membership
Most companies change their passwords every 30 to 90 days. This makes a stolen password good for a limited time only, so the first thing hackers will do is create user accounts. These accounts must be deleted. You should have a copy of your user list in your security notebook. (If not, make one.) Print out a new list and compare the two.

Verify all new accounts, especially accounts that appear to be service accounts. Hackers create user accounts that appear as service accounts (such as iusrcomputername instead ofiusr_computername or svcveritas), hoping administrators will overlook them.

The first groups to check are the built-in groups like Administrators and Server Operators. If an attacker created user accounts, chances are good he or she gave one of them elevated privileges. Remember to document everything as you go.

Check for Programs Configured to Start Automatically
Always look for programs that are set to start automatically. These are dangerous, as you could be running a program and not even know it. Always check the following places:

  • AT Scheduler and Task Scheduler
  • Services
  • Startup Folders
  • Registry

Verify that all scheduled jobs should be there and are configured correctly. Anytime you schedule a job, note the details, including date and business reason for implementing the job, in your server notebook. What appears as a normal scheduled task could be a password-gathering tool scheduled to run every night at midnight under the system’s credentials.

Always check your services. Check the account they’re using and what executable they’re starting. Manually start the service to verify that it works correctly. Hackers will replace the executable used by the service with a new executable. At first glance, everything appears to be normal. However, when the service starts, the damage begins.

Programs can also be configured to start when a user logs in or opens a certain program. Be sure to check the startup folders for all of the profiles on your system. Also check out the following keys in the registry to look for run entries:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\Run

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\RunOnce

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\RunOnceEx

HKEY_CURRENT_USER\Software\Microsoft\Windows\Current Version\Run

HKEY_CURRENT_USER\Software\Microsoft\Windows\Current Version\RunOnce

HKEY_CURRENT_USER\Software\Microsoft\Windows\Current Version\RunOnceEx

Check File, Share, and Registry Permissions
Always assume a hacker has given himself or herself permissions to everything. Check your local file system. Look for new shares and check permissions on existing shares. If a hacker has rights to a user’s directory, he or she could use it to upload harmful scripts or programs. Thus, when a user clicks on that new file in the directory, the hacker’s tool is launched. Also check registry permissions, especially the keys listed above. If the hacker sets these keys to Everyone Full Control, he or she can easily configure them to run programs later.

Check for Files and Features Added by the Hacker
Look for any files that shouldn’t be on your computer. It’s not a bad idea to print your PC or server’s directory and store it in your notebook. This way you can easily print out another directory (dir > lpt1) and compare the differences. Even better is to use a third-party program such as Tripwire (www.tripwire.com) to check data integrity. Tripwire creates a snapshot of your system and stores it in a database. Then, later, you can run another snapshot and compare the changes.

Also check for normal features of Windows that may have been installed by the attacker such as IIS, RAS, FTP and Telnet Server. If these features aren’t needed, they shouldn’t have been installed; uninstall them. Hackers always try to leave themselves several back doors into the system. What easier way to do that than with the built-in features provided by Microsoft?

Should I Rebuild?
From a security standpoint, it’s always better to rebuild your system and restore your data from clean backups. This way you know your server’s clean. I recommend using a third-party imaging product like Norton’s Ghost or PowerQuest DriveImage to create an image of all of your servers when they’re built. Then you can rebuild a server in less than 10 minutes.

Minimizing Successful Attacks
There’s no way to guarantee you’ll never be hacked again. There are only best practices you can follow to limit the likelihood of being hacked. Apply all service packs and security updates; most compromised systems aren’t fully patched. Install virus protection. Lock down your computers so that only the services needed are available. You may want to install a third-party firewall product on your server to help with this. Thoroughly locking down a server can be a time-consuming process but probably less so than cleaning up after an attack.

comments powered by Disqus

Reader Comments:

Wed, Sep 25, 2002 Sean Anonymous

Consise and accurate.

Thu, Sep 12, 2002 Anonymous Anonymous

rather basic but good!

Thu, Sep 12, 2002 Monte Washington

Simple and good

Wed, Sep 11, 2002 Ghostz Anonymous

i dun understand!

Wed, Sep 11, 2002 Mohammed Marhoon Kingdom of Bahrain

It's Great, to start.....

Mon, Sep 9, 2002 Anonymous Anonymous

Rather basic but good!

Tue, Aug 27, 2002 rotund1 baltimore

great job... thanks.

Tue, Aug 27, 2002 mrchase Cali Side

Jackie since you're a versed veteran. I would assume that you've recently looked at a number of web server transfer logs within the last year. Wouldn't you agree that with bugs Unicode and CodeRed attackers and their worms typically try to execute cmd.exe or some other binary that would allow them to possibly download netcat or some other remote execution command line tool in order to get a foothold on the system? Wouldn't you agree that with Worms Like SQL Snake being able to check your firewall logs and correlate them with your windows systems logs that just might have a vulnerable version of SQL from a single system isn't valuable? You're telling me that the MCPMag Microsoft based community isn't interested in information like this? How can that be? When I e-mailed Chad Todd he acknowledged that there were deficiencies in his article.

Tue, Aug 27, 2002 mrchase Cali Side

The points that I make in my comments regard a central syslog server are have been used in Unix environments for years. With the advent of companies like rippletech.com who has been in business for a great deal of time you have to wonder why this was not covered.
Any seasoned IT Security PRO Lance Spitzner maybe would tell you about the value of a "secure" central logging server and doing IIS ODBC logging. Please don't discount valuable advice.

Tue, Aug 27, 2002 mrchase Cali Side

You know its really hard to write in pretty little paragraphs when you don't have small things like additional spacing, or line breaks.

Tue, Aug 20, 2002 Umer Pakistan

Its a good approach but it still misses some critical points that hackers may use to interogate our systems.....Good effort though

Tue, Aug 20, 2002 Jackie Assprin East Side

mrchase,
Get a life. Write your own article if you have something to contribute. Don't waste mine and MCP mags time with mindless garble. This article is well written and to the point. It is not written for SANS, but for a Microsoft based audience. If you need to critique, be constructive not immature. And for God's sake, learn the English language! Step outside, it a beautiful world.

Avid MCP Reader and well versed IT veteran
-JA

Tue, Aug 20, 2002 mrchase Cali Side

Its kind of hard to mop up the "hackers" mess and prevent them from coming back in if you are doing it from 10,000 feet! You guys should look at giac.org and look at the posted practicles they are much better.

Tue, Aug 20, 2002 Farooq Anonymous

GOOD

Mon, Aug 19, 2002 Anonymous Anonymous

okay, only covered the issues from a high level.

Sun, Aug 18, 2002 mrchase Cali Side

I'm not happy with Chad Todd. Scripts, Worm, Tools or people don't break into systems and use things like PCanywhere to get a GUI on a machine that they hacked I would much rather use something like ANITX to have full unix like shell from the command line that I can use netcat redirects so that you'll never know what's going on because its happening over an encrypted session using BLOWFISH. READ BELOW FOR YOUR OWN GOOD!!! What he is saying that you need to use some kind of file integrity checking system that is piped off to a central location. Use something like NTSyslog look on sourceforge.com or the thing from intersectalliance.org and either use a UNIX or NT central syslogging server msyslog.sourceforge.net or kiwi syslog which works on windows. You can make your routers also log to that syslog server. If someone broke into your web servers which is what most likely happend then you're an idiot but that is another story. Look at your central syslogging server for events from the server that was compromised for unsuccessful / successful logins. Make sure that everything is being sent know what is normal so that you can verify what happened. Most of these web attacks need to somehow execute arbitrary code on the vulnerable server so that they can get in to do all of the bad things that crackers are known for. Make sure that you employ process separation you should not run a service under a user context with any more privileges than it needs which means that your exchange server, your sql server should run under a normal user account. Since you create a new user for IUSR_COMPNAME that user shouldn't have access to any application like at, or cmd. Use ODBC logging for IIS so that you can look at the web requests and determine what is not normal. On that central system that you use to login to other systems from there should be process accounting so that you know what people are doing on the machine. Not allowing everything out of from the DMZ is a good idea which would mean that only requests for HTTP, HTTPS etc. should come into the web servers and only traffic with a source port of 80 or 443 etc.. with a destination port of whatever should leave which would make something like PCanywhere impossible to use. Any system allows remote logins from should be on a procted subnet away from others screen by at least one firewall and if you dont have the Firewall feature set on your cisco router it does NOT COUNT AS A FIREWALL. When you setup new servers run a file integrity checking tool that also checks for ALTERNATE DATA STREAMS and creates a crypto hash, every time that you do a product update run this again. Make sure that you're information security policy that you have signed from the CEO has included in it INCIDENT HANDLING. I'm disgusted that this was not talked about in the article. If you have more questions please refer to the good people at online.securityfocus.com and the microsoft mailing list is a good place also into the archives for the INFOCUS there are a bunch of articles regarding things like IIS and securing windows.

Thu, Aug 15, 2002 James Anonymous

A good start

Thu, Aug 15, 2002 Anonymous Anonymous

rather basic but good!

Thu, Aug 15, 2002 Anonymous Anonymous

rather basic but good!

Thu, Aug 15, 2002 Hanan Israel

Sounds simple... too simple. You left out the frantic feeling that you have been invaded.

Thu, Aug 15, 2002 gfreeman London, UK

A good start on what to do - sound basis for a department policy tweaked to your own needs.

Wed, Aug 14, 2002 Fred Delaware

It is always positive to have knowledge about where and what to look for. I think having software that monitors uncommon ports is a good tool to see if someone might be trying to create a back door as well. When using remote software I would also recommend using extreme passwords and changing the ports that the software communicates on if possible. Software updates have become more critical than ever and will help with possible flaws. Keep up the good work.
Thanks

Wed, Aug 14, 2002 Anonymous Anonymous

interesting

Wed, Aug 14, 2002 Anonymous Anonymous

Great and well organized

Add Your Comment Now:

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Comment:
Please type the letters/numbers you see above

Redmond Tech Watch

Sign up for our newsletter.

I agree to this site's Privacy Policy.