Posey's Tips & Tricks
What Happens When You Use a Virtual Reality Headset in Space?
And now for something (almost) completely different, Brien sees how well a HoloLens-like headset weathers the unique conditions of a zero gravity flight.
You may recall that last year I had the opportunity to try out a Microsoft HoloLens in zero gravity. As fun and unique as that experience might have been, it was really just the first step in exploring how the tech tools that we use every day might be used in space.
Recently, I was in Ottawa, Ont., where several of my colleagues and I have been performing an absolutely insane number of experiments in zero gravity onboard the infamous vomit comet -- the jet that is used for zero-gravity flight. Although I was responsible for several of those experiments, none of them had much to do with tech, at least not within the scope of my Posey's Tips & Tricks column. For instance, I tested an Egyptian CubeSat (a micro-satellite) to make sure that its solar panels would deploy properly in space. I also performed an MIT fluid cell experiment, and another experiment designed to determine how water would behave on the moon (we flew a few parabolas designed to create lunar gravity).
Although the space nerd in me was fascinated by all of these experiments, especially the CubeSat, the experiment that really caught my attention was one operated by my colleague and close friend Heidi Hammerstein. Some of you might remember Hammerstein from the presentation that the two of us gave at Microsoft Ignite last year.
The experiment that Heidi flew was a virtual reality-based radiology suite designed by Luxsonic Technologies. The idea behind the experiment was that VR technology might make it possible to send things to space that would otherwise be completely impractical. It can cost in excess of $10,000 per pound to send an item into orbit. Launch vehicles also have a very limited amount of volumetric capacity available for carrying cargo. As such, the odds of someone sending a full radiology suite into space are pretty slim. Luxsonic's founder, Dr. Mike Wesolowski, wanted to see if it was possible to create a virtual radiology suite that would be far smaller and lighter than the physical models that hospitals normally use. The other goal, of course, was to design the virtual environment in a way that would allow it to function in space.
The thing that really intrigued me about this particular experiment was that Luxsonic did not base its tests on the use of proprietary hardware, but instead created an entirely software-based solution that was designed to run on the Oculus Quest (although Wesolowski did not rule out building custom hardware in the future).
Even though I was responsible for performing other unrelated experiments, I got a chance to try out the virtual radiology suite before the flight. The software placed me in a room standing in front of three large monitors that had been configured to display images from a CT scan. I was able to transition from one image to the next, and to zoom in when necessary.
As curious as I might have been about what the application actually did, I was far more interested in finding out how well the Oculus Quest would perform in zero gravity -- we also tested the headset in lunar gravity -- especially given my experiences with the HoloLens. Once we were safely back on the ground, I pestered Hammerstein and Wesolowski with a million questions regarding how well the device had worked.
One of the first things that Hammerstein told me was that wearing the device in zero gravity did not cause her to feel sick, which was surprising given that parabolic flights are notorious for causing motion sickness. I experienced a significant amount of nausea when I used the HoloLens while weightless a year earlier, and I really expected the Oculus Quest to do the same thing.
It's hard to say for sure, though, if the Oculus Quest headset provides a more comfortable experience than the HoloLens, or if Hammerstein just handles the environment better than I do. Hammerstein and I have flown together on a number of parabolic flights, and she seems to be a lot less prone to motion sickness than I am.
One of the things that I was the most curious about was how the device would handle the hyper-gravity portion of the flight. To achieve zero gravity (or lunar gravity, for that matter), the aircraft must fly a series of parabolic arcs. Although you are weightless during the descent portion of the parabola, you experience hyper-gravity during the climb. During this part of the parabola, everything in the plane feels significantly heavier than it normally would.
The HoloLens didn't handle hyper-gravity very well. Sometimes the display would spin. Other times the colors would fade from view. Hammerstein said that she kept her eyes closed during the hyper-gravity portion of the parabolas in an effort to prevent motion sickness, but took a few quick peeks at the display just to see what it was doing. Unlike the HoloLens, the Oculus Quest's display didn't fade or spin, at least not vertically. There was, however, a slow, constant horizontal spin during zero gravity, lunar gravity and hyper-gravity.
The radiology software itself worked exactly the way that it was supposed to, aside from the headset losing track of which direction was forward. Hammerstein said that in each parabola, she had to look around to find the interface before she could begin performing the various tests. The interface, which started out in front of her, might be shifted to her right or left side, or possibly even behind her.
Wesolowski speculates that the reason why the headset is unable to maintain its orientation isn't because of the changing gravitational conditions, but rather because of the way that the device's room-tracking sensors work, and because of the way that parabolic flight works.
During the flight, each parabola begins with a dive that is intended to help the aircraft build speed. Once the jet reaches a speed of 340 knots, the pilot pulls back on the stick and initiates a 2G climb. At that point, the jet is ascending at a 40-degree incline, although it feels much steeper than that. Once the aircraft reaches the top of the parabola, the pilot puts the jet into a very steep dive.
The whole parabola from start to finish lasts just under a minute. During that time, the aircraft's position relative to the sun changes dramatically. Inside the aircraft, sunbeams cast rapidly changing shadows at the beginning of the 2G pull and again at the top of the parabola. These lighting changes may very well be the thing that causes the headset to lose its tracking.
All things considered, the Oculus Quest performed much better in zero gravity than I expected it to. In the future, there may be another test in which we cover the aircraft's windows in an effort to provide the device with consistent lighting. My guess is that consistent lighting will probably help the device to maintain its room tracking, thereby eliminating the slow horizontal spin. It's purely speculation at this point, but I think that the Oculus Quest could actually work in space.
If you would like to know more about the VR experiment, then check out the interview that I along with Dr. Shawna Pandya did with Hammerstein and Wesolowski just after the flight.
About the Author
Brien Posey is a 22-time Microsoft MVP with decades of IT experience. As a freelance writer, Posey has written thousands of articles and contributed to several dozen books on a wide variety of IT topics. Prior to going freelance, Posey was a CIO for a national chain of hospitals and health care facilities. He has also served as a network administrator for some of the country's largest insurance companies and for the Department of Defense at Fort Knox. In addition to his continued work in IT, Posey has spent the last several years actively training as a commercial scientist-astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space. You can follow his spaceflight training on his Web site.