Never Again

Sometimes You Have to Say No

One admin realizes too late that he might have taken more than he could handle.

I was hired by a midsize IT department to be the lead Windows engineer. There was no one in this position before. My primary job was to consolidate and unify a decentralized and what I'd call an "everyone-did-what-they-wanted" type of environment. There were multiple Windows NT domains mixed in with Netware and multiple levels of authentication. The goal was to have one internal corporate domain and single sign-on authentication that leveraged Active Directory. I also had to handle data migration and consolidation, mixed in with putting out daily fires and resolving many issues before I could even begin a migration. This project had been underway for two years and run by others with no migration experience, so my timeline was about eight months.

Disaster Strikes
The real horror story began with the testing phase. I was attempting to put all the pieces together to achieve my goals. I had set up parallel Windows-based development environments, but made the error of setting up synchronization to the production Netware platform. This in and of itself was fine, until I decided to break down the development environments.

My blunder was that I forgot that I set up full bidirectional synchronization between my development AD domain and Netware. I deleted the OUs and user accounts in AD before stopping the synchronization. The result was the Netware user accounts started deleting from the tree.

I didn't realize this until we started getting reports from users that they were getting messages regarding their account deletions. At first, I didn't put two and two together. By the time I had, roughly half the 900-plus Netware accounts had been deleted.

To compound the issue, we discovered that our backups had been failing for months with no one noticing. We were forced to restore from months-old data. Needless to say, it was a full 26 hours before we had systems backed up and users logged in and working.

Older and Wiser
The moral of this story is twofold: First, learn to say no and push back when your plate is full and spilling over. I was new to the organization and was trying too hard to prove myself. Consequently, I took on too much and made the rookie mistake of costing myself and others on my team days of work. Second, I should have more regularly monitored the backups. If we had better backups, the impact from this mishap would have been softened.

I finally got through the migrations, and on time, no less. Now, I oversee administration of a single Active Directory domain that provides single sign-on functionality to all corporate resources. Management let me stay on and finish what I started, but having to admit I was the cause of that fiasco was one tough pill to swallow.

About the Author

Brian Mangiaracina is a systems engineer.

Featured

  • How To Configure Windows 10 for Intel Optane Memory

    Intel's Optane memory technology can significantly improve the performance of your Windows 10 system -- provided you enable it correctly. A single mistake can render the system unbootable. Here's how to do it the right way.

  • Microsoft and SAP Enhance Partnership with Teams Integration

    Microsoft and SAP this week described continuing partnership efforts on Microsoft Azure, while also planning a Microsoft Teams integration with SAP's enterprise resource planning product and other solutions.

  • Blue Squares Graphic

    Microsoft Previews Azure IoT Edge for Linux on Windows

    Microsoft announced a preview of Azure IoT Edge for Linux on Windows, which lets organizations tap Linux virtual machine processes that also work with Windows- and Azure-based processes and services.

  • How To Automate Tasks in Azure SQL Database

    Knowing how to automate tasks in the cloud will make you a more productive DBA. Here are the key concepts to understand about cloud scripting and a rundown of the best tools for automating code in Azure.

comments powered by Disqus