Never Again

Please Tell Me That Was a DDS-2 Tape

Of all the times to have a backup crisis, it had to be the eve of Y2K.

It was December 1999 -- just a few days before the big Y2K New Year's Eve bash. I was working at a privately owned company that ran an AIX-based Fixed Assets system on an IBM RS/6000. The database was Oracle and the backup system was ARCserve. We also ran ARCserve on our Novell and Windows NT 4 systems. The AIX system used DDS-2 tapes; NT used DDS-3.

We had recently moved an application from the RS/6000 to a new AS/400 server. The move freed up some badly needed space, which let us move Fixed Assets to a larger drive. The plan was relatively simple: Stop Oracle, back up the drive, mount the file system on the new drive and restore the tape.

I had done something similar a year earlier when we purchased an external local storage cabinet. I loaded a new tape and went through the process of backing up the system. Then I removed the file system and mounted the new 9GB drive with the same name.

As I kicked off ARCserve and started the restore process, the tape ejected and the job canceled. I put the tape back in and kicked the restore off. Again, it ejected and the job canceled. That's when I started sweating.

Next I dug through the backup logs, and found that my evening backup jobs had been canceling consistently due to the lack of hard drive space. After digging through tape libraries, the earliest successful backup of Fixed Assets I could find was a full two weeks old.

At this point, I was going through everything I could think of, including rebooting AIX and recycling power on the tape drive. We even pulled out the old hard drive and shipped it to a company that specializes in restoring data. No dice.

What's Your Worst IT Nightmare?

Write up your story in 300-600 words and e-mail it to Editor at Large Jeff Schwartz at [email protected]. Use "Never Again" as the subject line and be sure to include your contact information for story verification.

As I was examining the tape I had used for the current successful backup, I found to my horror it was a DDS-3 tape. I tried restoring it to an NT drive, but NT ARCserve wouldn't recognize it. I called ARCserve and explained the situation, but they were not able to come up with a solution.

So there I sat, at 11:30 p.m. on that epic New Year's Eve, restoring a two-week-old tape. Fortunately, our Fixed Assets accountant had hard copies, but had to re-enter everything from the lost two-week period. To this day, I don't think she has ever really forgiven me.

We replaced the Fixed Assets system two years later and stopped using ARCserve. I'll never use it again.

About the Author

Derryl Steib is a manager with Ergon Inc.


comments powered by Disqus