Posey's Tips & Tricks
Rethinking My Air Gapped Backups
Like anyone else, I perform regular, automated point-in-time backups of my production Hyper-V virtual machines. In addition to those backups, I also regularly create air gapped backups.
Like anyone else, I perform regular, automated point-in-time backups of my production Hyper-V virtual machines. In addition to those backups, I also regularly create air gapped backups.
For those who aren't familiar with the concept of an air gapped backup, it's really just a backup copy that is written to removable media and then ejected from the system. The reason why air gapped backups are important is because if a system were to become infected with ransomware, then the ransomware could potentially harm the backup as well, thereby leaving you with little choice but to pay the ransom.
It is worth noting that not every ransomware infection inflicts damage that makes it impossible to restore a backup. Depending on the type of ransomware, the system that is infected, the permissions associated with the user who is logged on to that system at the time of infection, and numerous other factors, ransomware may or may not be able to compromise your backup.
Since there is always a chance that a ransomware infection could damage any available backups, I like to hedge my bets by creating an air gapped backup copy. Because the air gapped backup is physically disconnected from the system, ransomware cannot touch it (unless the air gapped backup is inserted before the infection is removed).
Before I move on, the one thing that I really want to stress is the idea that in most cases an air gapped backup exists as a failsafe. It is only used in the event that the primary backup becomes damaged. In my own organization, for example, I create hourly disk based backups and replicate those backups to a separate storage array. At least once a week however, I also create an air gapped backup that I can use as a last resort in the event that my primary backups become damaged.
Lately however, I have been thinking about the role that my air gapped backups play in my disaster recovery strategy and about the way in which I create the air gapped backups.
As previously noted, my entire reason for creating the air gapped backup is to act as a last line of defense against data loss. If my primary backup and its replica were to fail, then the air gapped backup represents my only realistic option for recovering the bulk of my data. As such, it likely makes sense to use the air gapped backup as more than just a protective mechanism that can be used following a ransomware attack. Ideally I should also be able to use the air gapped backup to protect against other types of data loss events that end up impacting my backups. So what types of incidents might such a backup protect against?
Let me shift gears a bit and tell you about a problem that I encountered several years ago. For whatever reason, the data cable attached to the storage array that I was using at the time started going bad. The cable problem was intermittent, so it did not impact all write operations. But I did began to notice with increasing frequency that files were being corrupted. Unfortunately, much of the corruption initially went undetected, which meant that I was backing up corrupt data. Hence, my primary data copy was bad and so were my backups. Thankfully, that particular incident only led to me losing a relatively small number of files. Better still, I happened to have copies of most of those files on other devices and was able to get my data back.
So what does a bad data cable have to do with an air gapped backup? Remember when I said that an air gapped backup should ideally help you to be able to recover from a variety of situations? When my data cable went bad, it caused individual files to become corrupt. The process happened slowly enough that at first I did not even notice it.
Think about that in the context of the way that I am creating air gapped backups today. Since the data that I need to protect exists within Hyper-V virtual machines, I am simply copying the virtual hard disk files to a removable hard disk. The problem with this (from a last resort data recovery standpoint) is that a virtual hard disk is really just a big file. All of the individual files that I use every day reside within those virtual hard disk files. If a hardware related data corruption incident like the one that I described a moment ago were to happen today, it would likely damage the virtual hard disk file. As the corruption progressed, it would probably eventually reach the point where it became impossible to mount the virtual hard disk, let alone access the files within it. This would be especially true if the virtual hard disk file were encrypted.
That being the case, I am beginning to seriously consider writing individual files and folders to my air gapped backup rather than just copying a virtual hard disk file. The idea is that this approach would likely make it possible to recover the bulk of the data even if low-level corruption exists within the virtual hard disk file. The method is essentially removing one layer of complexity thereby improving the odds that the air gapped backup will be useful as a last resort recovery tool.
About the Author
Brien Posey is a 22-time Microsoft MVP with decades of IT experience. As a freelance writer, Posey has written thousands of articles and contributed to several dozen books on a wide variety of IT topics. Prior to going freelance, Posey was a CIO for a national chain of hospitals and health care facilities. He has also served as a network administrator for some of the country's largest insurance companies and for the Department of Defense at Fort Knox. In addition to his continued work in IT, Posey has spent the last several years actively training as a commercial scientist-astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space. You can follow his spaceflight training on his Web site.