IT's Security Dilemma: To Patch or Not To Patch
Security administrators faced a familiar if uncomfortable position: Just one
day after Microsoft released an out-of-band patch to fix vulnerabilities in
several versions of Windows, exploit code appeared in the wild.
The latest vulnerability -- with its near-same-day exploit code -- was too
serious to go unpatched, experts warned, but it put administrators in a bad
spot: Damned if they patched and damned if they didn't.
Industry watcher Gartner, for example, urged IT administrators to bite the
bullet and patch their systems. "Microsoft made the right decision to issue
an out-of-cycle patch for this vulnerability, given the evidence of active attacks
against the Windows Server service and the ease at which exploits can spread
across the majority of Windows systems," wrote Gartner analysts John Pescatore
and Neil MacDonald. "Although Microsoft reports that these privately reported
attacks have been limited and targeted, [it] is being very aggressive in pushing
these patches out to prevent large-scale downtime for businesses around the
Gartner argued that the stakes were just too high for IT administrators to
IT pros have been in this situation before. For example, during the summer
of 2001, Microsoft's Windows franchise was rocked by the first of the truly
blockbuster exploits: Code Red. In June of that year, media outlets alerted
users to a dangerous new vulnerability that affected Windows NT 4.0 and Windows
2000 systems running Microsoft's IIS 4.0 and IIS 5.0 Web servers. Microsoft
did its part, e-mailing alerts to the more than 250,000 subscribers of its security
mailing list and urging them to install a patch to fix the problem. Redmond
also announced that it had dispatched representatives to many of its largest
corporate customers to urge them to install the patch, too.
Despite Microsoft's efforts, the result, as everyone now knows, was Code Red:
a malware worm that infected tens of thousands of vulnerable, unpatched systems.
Just why did so many systems remain unpatched?
A closer look at Microsoft's history, particularly when it comes to software
patches, helps answer that question. Earlier that same summer, Microsoft required
three attempts to successfully patch a serious vulnerability in its Exchange
2000 Outlook Web Access Component. Prior to that, Microsoft had botched at least
two service pack releases (Windows NT 4.0 SPs 2 and 6), more than a few NT 4.0
hotfix updates, and a variety of maintenance releases (Office 97 SR-1, Office
2000 SR-1). In many cases, Microsoft software patches have severely damaged
systems; in some cases (Windows NT 4.0 SP2), they've rendered certain systems
This helped inculcate a prejudice among IT managers (who, understandably, tend
to be risk-averse). Many viewed the prospect of installing an untested Microsoft
software update as too risky.
Much has changed since that summer. To its credit, Microsoft has significantly
improved its software patching process. Gartner conceded as much, citing "Microsoft's
investment in a secure software development life cycle process" along with
the increased use of network intrusion protection software in many enterprise
shops as two factors that have "greatly reduced the success rate of attacks
trying to exploit these types of vulnerabilities."
All the same, risk-averse IT administrators probably still approach the prospect
of installing an untested Microsoft patch with some degree of trepidation. More
to the point, the systems management and software development lifecycles in
most shops simply aren't designed with rapid (that is, untested) patch deployments
A Certified Information Systems Security Professional (CISSP) with consultancy
and services firm Booz Allen Hamilton put it best: "The threat that comes
from the outside is an unknown. In a case like this, it might never materialize.
[So] if your systems go down because of an external threat...you're not off
the hook, but it's more understandable than if you install [a patch] and it
takes them down.
"If in the commission of updating and applying a patch you broke your
own system, you're kind of stuck on the hook for that. Chances are that [management]
won't buy it if you said you were only doing it to protect your systems."
That's why this security pro said it's still important to test -- even in cases
where exploit code is in the wild.
"The best you can do is compress your testing schedule. The most immediate
threat is that the patch could break your application, so you have to test it,
even if the testing is compressed," the CISSP said.
It's a rule for which few, if any, exceptions are made. "When SQL Slammer
appeared [in January of 2003], that was the only time in my life when I know
that a patch was installed immediately. Other than that, the management process
of the application system in question dictates how quickly a patch will be applied,"
the CISSP said. "You have to weigh the risk of the patch taking down your
system versus whatever damage could be done by the exploit."
Approaches to patching can vary from organization to organization, too. Shops
that are certified according to the rigorous capability maturity model index
tend to be much charier about how or when they install patches.
"A CMMI organization would very much prefer to run a full regression test
rather than just throw it on the server and hope for the best," the CISSP
said, "so the typical approach [in CMMI shops] is to thoroughly test it
and then to apply it -- but to apply it not right away but during the next scheduled
Stephen Swoyer is a Nashville, TN-based freelance journalist who writes about technology.