Posey's Tips & Tricks
Did Microsoft Finally Fix Hyper-V Replication?
A persistent thorn in Brien's side -- Hyper-V replication falling out of sync for no good reason -- seems to have fixed itself out of the blue.
As someone who spends a lot of time writing about Hyper-V, it is hardly surprising that I have several Hyper-V servers running in my home. My production environment consists of a single virtualized file server that is replicated across two Hyper-V servers.
A couple of years ago, I found I had outgrown my aging Hyper-V servers and decided to perform a hardware refresh. As part of that process, I purchased two external storage arrays that I attached to the two servers via 10GbE Ethernet.
For the most part, everything has been working smoothly since then. However, there was one incident that occurred roughly a year ago when a severe storm went through the area where I live.
The storm caused me to lose power for a few hours. Normally, this wouldn't have been a big deal because I have backup batteries on absolutely everything. However, I hadn't bothered to replace those batteries as part of my hardware refresh because they were hardly used; I thought that they were still good. At any rate, several of my backup batteries failed, including ones the external storage arrays were connected to.
Although I didn't end up suffering any data loss, the power failure and subsequent battery failure caused my replicated Hyper-V servers to fall out of sync. Additionally, I was unable to manually restart the synchronization process and had to delete the replica and create a new one from scratch.
The whole process only took about a day-and-a-half to complete, but being without a replica during that time made me a bit nervous. After all, I use the replica as a redundant system in case anything were to happen to my primary system.
A couple of weeks ago, I had another unwelcome surprise. I heard a loud beeping noise coming from my office and soon discovered that the battery backup connected to my replica Hyper-V server had died. The battery was under a year old, so I'm not really sure why it failed, but there it was, alarms blaring, deafening everyone within an earshot. I pressed a button on the battery backup to silence the alarm, but upon doing so, the battery completely shut down. Needless to say, my Hyper-V server lost power, as well.
Replacing the battery -- and all of my other batteries in case any of them were defective -- proved to be challenging. Because of where I live, I don't really have the option to go to the store and buy a backup battery. Ordering online was my only option. However, backup batteries were in short supply, presumably because of the blizzards that have been plaguing so much of the country. Eventually, though, I was able to get the batteries (and a spare in case this happens again). By the time the new batteries arrived, however, my Hyper-V replica server had been offline for more than a week.
Given what happened to me last spring, I was positive I was going to have to rebuild the virtual machine replica from scratch. After all, before my hardware refresh, there were times when replication had fallen out of sync for no good reason and I was left with no choice but to reseed the replica. The need for a complete resynchronization after more than a week offline was a given.
Much to my surprise, though, when I replaced the failed battery and powered up the Hyper-V replica server, replication resumed instantly, as though nothing had ever happened.
I have spent some time trying to find any reference online to changes that Microsoft might have made to the Hyper-V replication engine. My search came up empty, but I believe Microsoft must have quietly fixed the synchronization issues that have plagued Hyper-V replication for so long. If that is the case, then it is a very welcome fix and I want to take the opportunity to publicly say thank you to Microsoft for getting it done.
About the Author
Brien Posey is a 22-time Microsoft MVP with decades of IT experience. As a freelance writer, Posey has written thousands of articles and contributed to several dozen books on a wide variety of IT topics. Prior to going freelance, Posey was a CIO for a national chain of hospitals and health care facilities. He has also served as a network administrator for some of the country's largest insurance companies and for the Department of Defense at Fort Knox. In addition to his continued work in IT, Posey has spent the last several years actively training as a commercial scientist-astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space. You can follow his spaceflight training on his Web site.