Posey's Tips & Tricks
What's Next for Windows Mixed Reality?
HP's next-generation mixed reality headsets, with their focus on cutting-edge biometric features, offer a glimpse into where the technology is headed.
It's been a while since I wrote about Windows Mixed Reality. At times, it seems as though the technology is stagnating. After all, Microsoft has continued to do big things with HoloLens 2, but it's been a while since the company has released anything new related to Windows Mixed Reality.
Even so, third-party vendors have been hard at work on the next generation of mixed reality headsets.
The most obvious ways in which mixed reality headsets could be improved is with better optics, better immersive sound and improved motion-tracking. HP is doing all of that with its forthcoming Reverb G2 Omnicept Edition headsets, but the company has also introduced something completely unexpected: biometrics.
If you are like me, you probably hear "biometrics" and think of security and authentication. However, that isn't what HP is doing. The new headsets include four sensors that are not typically found in mixed reality headsets. These sensors are being used for some really creative purposes.
The first of these sensors is the eye-tracking sensor. Admittedly, this one isn't entirely new; some of the higher-end Windows Mixed Reality headsets already include eye-tracking sensors. These sensors allow you to interact with virtual objects using eye gestures rather than having to use the controllers.
The second new sensor is the pupillometry sensor. Pupillometry is the science of measuring pupil size and responsiveness. In medicine, testing pupil responsiveness is a way of testing for various neurological conditions.
The third sensor is a heart rate sensor. This one is self-explanatory in that it measures the wearer's pulse.
The fourth sensor is a face camera. HP has not yet published very much information about this particular sensor, but from what I can gather, it seems that it will be used to examine facial expressions.
So why all the new sensors? Once again, HP has yet to release a lot of specifics, but it does present two use cases.
First, HP is planning to use a machine learning system to process the collective sensor input and use all of that information to determine the wearer's cognitive load. In other words, the headset will be able to determine how much brain power the wearer is using as they do whatever it is that they are doing.
Consider the implications of such a system. From a gaming standpoint, developers could create new experiences that dynamically adjust to the player's cognitive abilities. If, for example, the system detects that someone is really struggling with a game, it might dynamically adjust the difficulty to make the game easier. Conversely, if a game finds that a player isn't expending much effort, the game could become more difficult.
Personally, I think these new abilities will be much more useful for engineers, because it will allow them to evaluate other types of complex systems. For example, if a company were designing a new airplane, it might virtually try out several different cockpit designs to see which one is the least demanding of the pilots.
The other use case that HP laid out for its new headsets is something that I will call "GPU prioritization." Rendering a virtual environment through a mixed reality headset is a graphically intensive process. Depending on how realistic and complex the environment is, the rendering process can put a tremendous strain on a computer's GPU. What HP has figured out is that because its headsets can tell where a user is looking, it can focus the GPU processing effort specifically on that area. That way, the user gets the best possible graphical experience while areas of the virtual environment that are not actively being viewed receive less GPU time.
It remains to be seen how well this technique will work in the real world, but my guess is that this type of graphical rendering prioritization will allow for much richer experiences, even on somewhat modest hardware.
Of course, even after the HP headsets have been released, software will be required in order to take full advantage of their capabilities. HP is envisioning the headsets being used for training, well-being, creation and collaboration. I personally think that the headsets will ultimately prove to be useful in other areas such as engineering, medicine and scientific research. We'll just have to wait and see.
About the Author
Brien Posey is a 22-time Microsoft MVP with decades of IT experience. As a freelance writer, Posey has written thousands of articles and contributed to several dozen books on a wide variety of IT topics. Prior to going freelance, Posey was a CIO for a national chain of hospitals and health care facilities. He has also served as a network administrator for some of the country's largest insurance companies and for the Department of Defense at Fort Knox. In addition to his continued work in IT, Posey has spent the last several years actively training as a commercial scientist-astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space. You can follow his spaceflight training on his Web site.