In-Depth

It's 10 O'Clock. Do You Know Where Your Hacker Is?

IT pros have reservations, but ethical hackers are becoming a fact of life.

• By Joel Shore
• 02/01/2007

"You want the good guys to find the security holes before the bad guys do," says Jack Koziol, program manager at the InfoSec Institute, an organization that certifies security professionals. "If your people are not doing it, someone else will -- and that someone won't be on your side," he says.

It's not just about keeping that nefarious "someone" out. Nabbing a successful perpetrator -- or even simply knowing that a break-in has occurred or is being planned -- is too often well beyond the technical scope of many IT departments. Even worse, when a break-in is discovered, most IT professionals don't know how to secure and preserve the evidence needed for the forensic analysis and prosecution.

Enter the Good Guys
Paul H. Luehr is a former computer crimes prosecutor with the U.S. Dept. of Justice and the Federal Trade Commission. After the Sept. 11 attacks, it was Luehr who oversaw the initial forensic investigation into computer evidence related to convicted terrorist Zacarias Moussaoui. He has also prosecuted computer crimes perpetrated against eBay, Best Buy and the corporate parent of Saks Fifth Avenue.

Today, Luehr is deputy general counsel at Stroz Friedberg, a law firm that specializes in using forensic investigations to prosecute computer crimes. "IT departments often jeopardize prosecution because they are unfamiliar with the use of the procedures and forensics required to catch and put the bad guys behind bars," he says.

Matt Hillman, founder of the Legion of Ethical Hacking (LEH), believes the concept is valid. He classifies any authorized break-in as ethical hacking, which eliminates some of the confusion. "Hacking is essentially neutral. It's just a thing that you do. What you do it for is a whole other matter," he says. Of course, Watergate was an authorized break-in, and we all know how that turned out.

So these days, we certify the good guys, or the people we hope are good guys. For InfoSec's Koziol, that means running extensive background checks before students are taught advanced techniques such as DNS host identification abuse, cache poisoning, password cracking, spoofing, SSL session hijacking and malicious log editing, just to name a few things in a hacker's bag of tricks. He notes that students are typically experienced IT professionals from larger corporations taking the next step toward protecting their companies' crown jewels.

Though thousands have taken these tests and become certified hackers, even Koziol acknowledges the exams test only technical aptitude -- not one's underlying ethics. He's quick to note, however, that InfoSec has never heard of one of its certified hackers going bad.

It can be a fine line, however, that separates the hackers wearing white hats from those wearing black, says Oliver Friedrichs, director of Symantec Security Response. "Checking for a criminal record or prior abuse gets you only so far. After all, a successful hacker is someone who has managed to remain invisible," he contends.

The stereotype of the invisible hacker -- someone sitting in his or her apartment surrounded by computers -- is "largely true," says Gunter Ollmann, director of IBM Internet Security Systems' X-Force Threat Analysis Service.

His team invests their own personal time to improve their skills, he adds. Most belong to groups that meet regularly where they exchange new tools and techniques. "There's a lot of alcohol involved and the burnout rate for these people is very high, typically five to eight years before they become alcoholics or burn themselves out," he says.

When hiring for X-Force, Ollmann insists on candidates with a technical degree. Most come from the physical sciences, instead of computer science. His candidates must have three to five years of multinational experience in dealing with large infrastructures and a breadth of attack types, or as security researchers with a detailed understanding of how large institutions develop and deploy systems.

Not surprisingly, hiring a "hacker gone straight" is generally frowned upon. "The idea of a bank hiring a convicted robber as a security consultant because he knows where the money is just doesn't make sense," says Luehr. "It's not the image any reasonable corporation should project. After all, the guy got caught."

Cheryl Currid, a former IT director with a Fortune 100 company and current president of Currid & Co., an IT strategy consultancy, takes a more reticent position. She's still cautious, however, about recruiting from the dark side. "It's possible to learn a lot from these guys," says Currid. "I would hire only on a short-term project basis. Bring him on full time and he'll get bored." And the trouble with boredom, she adds, is that it breeds curiosity, which in turn breeds trouble.

Whether you call it ethical hacking or penetration testing, the underlying philosophy of proactively finding weaknesses before the bad guys do is very much alive at IBM's Global Services unit. In current online marketing materials for its Ethical Hacking service, IBM states that its team members simulate a real intruder's attack in a controlled manner and "tell you what they find and how you can fix it." The service comes at a steep price, though. One stand-alone IBM Ethical Hack will set you back as much as $45,000.

Is it worth the price? If the testers discover a damaging vulnerability, then it's practically priceless. By their very nature, though, any security test can find only that which it is assigned to look for.

"You've got to keep in mind that no matter how good any tool is, it can do only what it's designed to do," says Michael Howard, senior security program manager in the Security Engineering Group at Microsoft and a world-renowned expert on software security. "The nature of threats is constantly changing and the people behind those threats are more sophisticated than ever. The tests will always be one step behind," Howard says.

Testing, Testing
In his landmark 2001 white paper on ethical hacking, Charles Palmer, manager of the Network Security and Cryptography department at the IBM Thomas J. Watson Research Center, identifies six key areas of testing. Published years before the rise of social-networking platforms like MySpace and YouTube and the thriving music and video downloading industry, Palmer's target list seems positively clairvoyant today:

• Remote network: Simulate an Internet attack by hitting perimeter firewalls, filtering routers and Web servers.

• Remote dial-up network: Targeting authentication schemes, this was originally conceived to attack modem pools. It has been updated to include any channel providing external access to the internal network, including a VPN.

• Local network: This tests employee or other authorized access from within the perimeter. Targets include intranet firewalls, internal Web servers, server security measures and e-mail systems.

• Stolen laptop: Choose a key company employee, then take his or her laptop computer without any advance notice and give it to the testers. Targets include passwords stored in remote-access software, corporate information assets, personnel information and customer data (whether it's encrypted or not). This is a favorite way to leapfrog perimeter security and gain access to the corporate intranet with full privileges.

• Social engineering: These are not technical tests, but rather evaluations of staff behavior. Tests include calling tech support and asking for remote-access assistance or going on-site, looking lost, and asking where the computer room is located. Updated for life online, other tests include how employees respond to e-mails from impostors, whether or not they click links that may lead to sites with malicious software, and if they download multimedia that may contain embedded malware.

• Physical entry: This test gauges on-site security, security guards, access controls and monitoring, and security awareness by attempting to gain access to the premises. A hacker might try this by digging through trash cans to find documents with the company logo.

Palmer concludes that "regular auditing, vigilant intrusion detection, good system administration practice and computer security awareness are all essential parts of an organization's security efforts." Just one failure, he says, can expose an organization to cyber-vandalism, embarrassment, loss of revenue, and/or litigation. As for the ethical hackers themselves, while Palmer says they will help any IT director better understand the organization's needs, they should be carefully watched as well.

Simple Solution, Zero Cost
Besides testing the stolen laptop scenario, Luehr also recommends choosing servers at random and testing whether logging functions are on and that firewall functions are operating correctly. "One of the biggest problems we see is IT directors who carry the old habit of not turning on enough logging functions," he says. He ascribes this practice to a time when storage was expensive and logging tactics and tools, like mainframe CICS journaling or NetWare's Transaction Tracking System, slowed system performance.

Today, he says, if you can log it, then turn it on and do it. "In any security investigation, whether in a preventative mode or reactive mode after a crime has occurred, those logs can prove invaluable."

Logging functions contain a goldmine of potentially useful forensic information, often including IP addresses, open port activity or even vectors of attack that investigators can analyze for patterns. "You can often tell whether the attack is coming from a domestic source, a former employee or from overseas hackers with more nationalistic goals in mind."

CSI: Data Center
While the purpose of ethical hacking is to minimize the possibility of an actual attack, no scenario is perfect. Consequently, quickly securing the crime scene following an attack is essential. This doesn't involve stringing yellow crime-scene tape across the data center, but it does involve taking any compromised systems out of service, assuming you can determine which ones they are.

"If a system is compromised and forensics are needed, locking the machine in a closet is far smarter and more effective than allowing an IT department's bright minds and curious fingers to poke away at it," says Luehr.

If you can't take the system out of service because it's running mission critical software, you can use specialized forensic tools to go after live data. It will take longer that way, and may hinder the prosecution of the offender. At Stroz Friedberg, Luehr's forensics examiners and penetration testers use several commercially available scanning and testing tools, as well as an arsenal of proprietary testing and data-reconstruction tools developed in-house.

As the need for penetration testing continues to grow, so too has the assortment of available tools. Among the best known is Internet Scanner from ISS, which IBM acquired in October. Others include Impact from Core Security Technologies, and Paraben Corp.'s software for analyzing e-mail, instant messages and handheld devices.

The New York-based EC-Council provides training that leads to certification as a Computer-Hacking Forensics Investigator, which is similar to InfoSec's ethical hacking certifications. The EC-Council course teaches participants to identify intruders' traces and to gather evidence needed for prosecution. The list of companies that keep at least one CHFI on staff reads like the Fortune 100.

While these tools and the investigators who use them find the vulnerabilities, InfoSec offers a CD-ROM containing more than 750 tools to exploit them. The list of tools includes keyloggers, password crackers, rootkits, router hacking, Trojans and password cracking dictionaries in 163 languages.

Never a Certainty
Unfortunately, plugging every security hole, shutting down every unused open port, changing default passwords on routers and running quarterly penetration tests still takes you only so far. Too often, the bad guys find their way.

"We see an increasing number of content-borne threats, such as scripts embedded in word-processing files," says ISS's Ollmann. A newer technique, prized by hackers for its elegant simplicity, is placing a keylogger or other malware program on inexpensive USB thumb drives handed out by the thousands as promotional items. "The moment you plug it into a USB port, you are in serious trouble," he contends.

No amount of training can prevent such threats. And penetration testing, by definition an attempt to break in from the outside, is unlikely to help in those cases. For that reason, most security auditing firms recommend frequent and comprehensive internal testing.

It's a fact of life that increasing percentages of IT budgets are being allocated to security. That provides a sad commentary on the times in which we live. Using ethical hackers and penetration testing to maintain network and data integrity, and forensic tools to analyze breaches and find the perpetrators has become an essential part of any IT security protocol. Jay Bavisi, president of the EC-Council, sums it up best. "To beat a hacker, you need to think like a hacker."