Or, "The Vacation That Wasn't."
As I was leaving for vacation I provided my backup with all the instructions necessary to install an anti-virus management and monitoring tool for Exchange 2003 on his workstation. The tool was needed to administer the 30-plus, statewide Exchange 2003 servers providing e-mail for more than 10,000 mailboxes.
Unfortunately, he populated the "install-to" list instead of the "servers-to-monitor" list with the 30-plus Exchange servers. The program wasn't intelligent enough to understand it was already installed, so the reinstallation began to run. I was paged shortly after my plane landed 5,000 miles away in a place with no broadband and spotty dial-up connectivity.
Once the reinstallation had completed on all of the servers, the version wasn't patched, some servers' anti-virus scanners wouldn't run, some Exchange services wouldn't start or the program wouldn't update. Opening a support ticket with the vendor resulted in a new Knowledge Base article and a solution from them reading: remove the product and reinstall.
The Knowledge Base article was used as the resolution to our problem. The biggest downer is this took place during business hours on the same day as the annual IT holiday luncheon, so the usual IT folks weren't around and messaging was a mess. Removing the product, cleaning up remnants and reinstalling did do the trick, and the company was only without e-mail for half of a day.
Scott Chimner is a security consultant with Convergent Computing in Oakland, Calif.