Never Again
Trusting to a Fault
A consultant learns the hard way to verify information with his own eyes.
In 2006, I was employed as a consultant for a small IT firm. That firm's major client, a startup insurance brokerage, was going through a massive expansion and opening up remote offices in multiple locations. I was given the assignment of traveling to those offices to set up their servers, NAS devices and the data side of the networks. Things were going fine -- at least until the last office I visited.
Worst Nightmare
This place proved to be something right out of the big book of "IT Professionals' Worst Nightmares," even though this particular office was located in the beautiful town of Hamilton, Bermuda. I spent weeks procuring the required electrical circuits, servers, racks, Cisco switches and routers, VoIP phones and Nortel PBX in preparation for this trip.
The Bermuda office had a local office manager who was in charge of the IT operations there. As it turns out, I made the grievous error of assuming she knew what she was talking about. I took her word that everything was in place and ready for me on the data side of the system deployment.
I arrived in Bermuda, settled into my hotel room, grabbed my laptop and other equipment and made my way to the office. After just a cursory glance around the server room, my buoyant mood quickly soured.
Disaster Area
The same person who assured me everything was in order had hired some local IT contractor to set up the Cisco 3750 layer 3 switches to support both the data and VoIP side of the network. We had our own local Cisco engineer back home. Due to Bermudian laws, however, he was not allowed to perform any onsite work on the Cisco equipment. This seemed to justify hiring a local IT contractor.
When I looked behind the racks, I realized that all he did was rack-mount the switches. There were no cross-connect cables, no power cables, no patch cables to the panels, and the switches had no configuration information whatsoever. Yet the contractor was more than happy to bill the company for his "work."
| What's Your Worst IT Nightmare? |
Write up your story in 300-600 words and e-mail it to Editor Ed Scannell at [email protected]. Use "Never Again" as the subject line and be sure to include your contact information for story verification.
|
|
|
Lesson Learned
It was then that I started to sweat profusely and screamed every profanity I could think of. I called my manager back home to lay out the situation for him, much to his shock and dismay. To make matters worse, the office had an unreliable and very slow residential-class DSL Internet connection. I was forced to use this slow connection to let my manager and the local outsourced Cisco provider back in the states remotely connect to my laptop and configure the switches through a serial console cable.
To make a long story short, I pulled through with the help of my home team. I even got the VoIP phones working with the PBX, despite not knowing anything about this technology. After all, I was there only to set up the servers and NAS devices.
I made a vow right then and there that in the future I would be more diligent and hold people more accountable for their actions, or, in this case, their lack of actions. I know I need to trust people. I also need to verify their statements to avoid such a hellish experience again.
About the Author
Roger Zan is an independent consultant.