Foley on Microsoft

Scorecarding: How Microsoft Measures Its Own Success

Mary-Jo Foley takes you inside the metrics Microsoft uses to judge itself.

• By Mary Jo Foley
• 03/01/2011

Observers measure the success of Microsoft with a variety of methods. Customers have their own unique evaluation criteria, as do Wall Street analysts, shareholders and Microsoft partners. But the process by which Microsoft execs evaluate their own company has long been murky to outsiders.

Ever since Kevin Turner became COO in 2005, measurement has become less of a black art and more of a science. The former Walmart exec brought to Redmond the ubiquitous "scorecarding" system the 'Softies love to hate.

The trend data for these measurements plays a big role in determining where Microsoft invests and where it doesn't, which products and strategies it evangelizes and advertises, and how the company's brass views and responds to customers.

Clearly Microsoft executives consider common external metrics -- say, the worldwide market share of Internet Explorer -- important enough to track constantly. But other less-obvious and less-tangible metrics also appear on the Microsoft scorecard watch list.

Overall Net Customer Satisfaction (NSAT): "NSAT is an index of net satisfaction scores across a number of segments and audiences as measured by the Global Relationship Study, which is conducted twice a year with goals being set against the H2 results," Microsoft explains internally. The senior leaders tout NSAT as the best measurement for setting customer and partner priorities.

Business Online Services (BOS): BOS are the services that Microsoft and its partners sell as complements to on-premises products, including Windows. Because Microsoft is so heavily dependent on the health of the PC market, the company established the BOS metric to monitor the overall state of the PC market and of Microsoft OS market share trends.

Windows 7 Enterprise Edition Deployments: Windows 7 Enterprise Edition is a Software Assurance annuity licensing benefit. Microsoft considers 10 percent as "the tipping point" in terms of infiltrating enterprise sales. The 'Softies also measure Windows Presentation Foundation (WPF) and Silverlight usage, as well as Internet Explorer 8, Windows Vista and Windows 7 targeting.

Office and Business Productivity Online Suite (BPOS): The 'Softies keep close tabs on the percentage of PCs running Office. Microsoft monitors both the deployment rates for Office copies sold via volume licenses and subscription agreements, as well as "unmanaged" -- essentially consumer -- Office sales. The company also is paying increasing attention to the number of new BPOS customers as a way of keeping track of the growth of competitors such as Google and other Software as a Service vendors.

Server Applications: The operations folks watch like hawks the number of seats sold of Exchange, SharePoint and System Center. They're tracking everything, including year-to-date new and renewal on-premises licenses. Of particular interest are the all-important Client Access License (CAL) sales, plus the net new seats from BPOS products. Top priorities here include securing and expanding the Exchange base, and -- on the SharePoint front -- paying close attention to the unwanted growth of "naked CALs," or CALs sold to customers with no SharePoint servers.

In studying the internal priorities of Microsoft, I find curious the seeming lack of concern about tablet competition and the relatively light focus on Windows phones. Anything else you think the 'Softies should -- or shouldn't -- be watching more closely, based on the information here?