Foley on Microsoft

What Does User Feedback Mean?

Microsoft is shifting its gears when it comes to incorporating feedback into its products.

• By Mary Jo Foley
• 10/01/2008

Microsoft has taken heat over the years in this very area, accused of not listening to customers. This complaint is somewhat unearned, given all the focus groups, on-site research, surveys and other ways the 'Softies use to determine how customers use Redmond's products.

Until now, it's been hard to argue that Microsoft doesn't listen to its beta testers. Unlike Apple and Google, which roll out new products that very few individuals have tinkered with through pre-beta or betas, Microsoft has gone to great lengths to incorporate tester feedback into Windows, Office and other wares.

Microsoft continues to run some of the biggest beta programs in the business. Windows Vista and Microsoft Office 2007 were tested by thousands before release. But what's shifting -- in a way that many Microsoft customers still have yet to understand -- is when and how user feedback is incorporated into Microsoft's products.

Take Windows 7, for example. The majority of testers probably won't get their hands on the first test build of Microsoft's new OS until this fall -- around the time of the Professional Developers Conference. However, by the time these testers get their first glimpse of Windows 7, the window of opportunity for input will have long since passed. With the way Windows is being developed these days, Windows 7 is basically already done.

Some PC makers got to see and share input on earlier Windows 7 milestone builds, which Redmond began releasing in a very controlled way in December 2007. I hear that a very select group of external testers also had a chance to interact with those milestone builds. But if you're thinking you can suggest a new feature or fix for Windows 7 once you get a test build, you're sorely mistaken. If Microsoft deems your Windows input worthy, it most likely won't see the light of day until Windows 8 or later.

The same is true of Office 14. Technical Adoption Program (TAP) testers aren't expected to get a first build of Office 14 until November 2008. Of course, a few chosen outsiders have gotten to play with internal milestone builds, but the vast majority of Office users are going to see the final Office 14 product the first time they get to play with a beta.

Ditto Internet Explorer 8, which even the most technically savvy developers didn't see until March 2008, when Microsoft rolled out beta 1.

From Microsoft's standpoint, releasing test builds that are nearly set in stone has a few advantages, the biggest being ship schedules. As the Office team, which keeps its early pre-beta programs very small and private, has learned, rolling out near-final public betas of new products helps to ship on time.

I personally predict that, by emulating this model, the Windows 7 team will ship in late 2009, less than a year after delivering this fall's test build. It's also the reason the Office 14 team will roll out the final version of its product in the latter half of 2009. It's also likely that the IE team will release to the Web the final version of IE8 in November 2008.

Finding bugs is unquestionably useful, but it's nothing like having input into the features that come together to form a new release or a whole new product. Once the great, unwashed masses of Microsoft beta testers realize they aren't influencing the development of new Microsoft products, there may be less interest among them to try "the next big thing."