News

Microsoft Ups Its Windows 10 App Compatibility Assurances

• By Kurt Mackie
• 01/16/2019

Microsoft gave assurances this week that organizations adopting Windows 10 likely won't face application compatibility issues.

Application compatibility has been "one of the most important considerations" of Microsoft's customers in shifting to the "modern desktop" and Windows 10, according to Brad Anderson, corporate vice president for Microsoft 365, in an announcement this week. Microsoft depends on its machine learning models to detect the effects of Windows code changes on applications before the OS gets released and it conducts various testing programs as well, according to an announcement this week by Mete Goktepe of Microsoft's Windows application compatibility team.

The app compatibility stakes are ratcheted up for organizations this time around because the Windows 10 operating system gets new "feature updates" twice per year, in the spring and fall. New feature updates essentially are new OS releases that replace the underlying OS, typically via "in-place upgrades." The increased frequency of OS upgrades potentially should entail greater application compatibility testing work by organizations, it would seem.

Organizations also face "Windows as a service" update deliveries with Windows 10. Windows as a service means that organizations will have to replace the underlying OS every 18 months in some cases, following Microsoft's schedule. In contrast, older Windows clients, such as the rapidly expiring Windows 7 OS, typically got replaced by service packs after about two or three years, and those replacements happened more in accordance with IT department timelines.

The service pack world is long gone with Windows 10, though. While Windows 10 should require more frequent testing by organizations, Microsoft has argued that organizations running Windows 10 should instead be using their end users, and the Windows Insider Program, to test its new OS releases. Microsoft also argues that its application compatibility record with Windows 10 is at the high mark.

Microsoft's App Compatibility Testing
Goktepe outlined the stages involved in Microsoft's application compatibility testing. Microsoft works with independent software vendors on any needed changes. The applications Microsoft tests are based on a "risk-based recommendation model," he noted.

Microsoft "validates thousands of apps internally each week." These apps are chosen based on popularity, risk level and partner feedback. Automated tests are conducted in the early stages of the testing, with Microsoft running "more than 280,000 individual app compatibility tests each day." Later, Microsoft will run tests on "around 2,500 apps" to detect compatibility issues. A manual testing phase occurs later to catch things that may be visually spotted by end users.

Microsoft gets "telemetry" data from Windows 10 machines "to detect and predict potential regressions impacting the app ecosystem." This information gets collected from Windows Insider Program test releases, as well as new Windows 10 releases.

When Microsoft needs to issue a Windows 10 fix, it's done via shims to the OS as well as by setting blocks on upgrades. Microsoft is now starting to publicly disclose when it is deploying a block via its Windows 10 Update history page, Goktepe noted.

Microsoft has described its application compatibility testing process before with Windows 10. A year ago, Microsoft outlined efforts in which it tested 1,650 "applications in business unit portfolios." This number got further winnowed down to just 250 "business critical applications," per the earlier description.

Desktop App Assure Program
To help organizations with application compatibility issues on Windows 10, Microsoft has its Desktop App Assure partner effort, which is now a globally available assistance program, according to Anderson's announcement. Its availability comes one month earlier than originally estimated when the program was first announced back in September.

Anderson claimed that the Desktop App Assure Program is really there as an assurance to customers, rather than being a preventative measure. Based on the program's test results, "only 0.1 percent of all the apps that customers who have worked with the Desktop App Assure team to evaluate have had a compatibility issue," he said.

Microsoft argues that organizations are actually "overtesting apps" with Windows 10 because of its small app compatibility failure rate, according to a description of the Desktop App Assure program by Chris Jackson, a principal program manager at Microsoft's Experiences and Devices Group. He admitted that the overtesting has some justification because small failures can blow up and become difficult to fix later for organizations.

Jackson said that the Desktop App Assure Program can help organizations when they have problems with home-grown line-of-business applications, commercial apps, Web apps and even Office plug-in applications.

However, the Desktop App Assure Program isn't for every organization. It's a FastTrack partner program. As such, the only organizations that can use the Desktop App Assure Program are those that have bought "at least 150 licenses from the listed eligible [Microsoft 365] plans." That particular detail is explained in this Microsoft document.