In-Depth

Windows Server 2003's Critical Deadline Is Fast Approaching

In less than five months, Microsoft will discontinue support for Windows Server 2003, meaning millions of systems could pose security risks and fail to meet compliance requirements. Are you ready?

• By Jeffrey Schwartz
• 02/27/2015

In the midst of planning the rollout of an electronic medical records (EMR) system throughout its network of hospitals and outpatient facilities, the IT organization at MaineHealth discovered numerous old servers scattered throughout the network that were running Windows Server 2003. Those servers must be decommissioned by July 14 of this year. That's not a deadline set by the powers that be at MaineHealth -- it's the last day Microsoft will support the server OS.

Microsoft announced the end-of-service date many years ago, consistent with the lifecycle of its OSes. In keeping with the sunset dates for its OSes, after July 14, 2015, Microsoft won't issue security patches for Windows Server 2003. For many organizations governed by industry or legal regulations such as the Sarbanes-Oxley Act (SOX) for publicly traded companies, the Payment Card Industry Data Security Standard (PCI for short) for those processing payments or the Health Insurance Portability and Accountability Act (HIPAA) for health-care providers like MaineHealth, that means those systems will no longer be in compliance.

It's unknown how many systems running Windows Server 2003 are still in production throughout the world, but the most recent estimates as of last month ranged from 8 million to 20 million -- and that could mean instances or physical systems. Even with the largest guestimates, that may sound like a paltry number compared to the staggering amount of PCs running Windows XP that IT organizations had to remediate last year, which were likely in the hundreds of millions last April.

The expiration of Windows Server 2003 won't be as high-profile to the general public as the demise of Windows XP (see "This Is the End," April 2014), but experts say it's as important if not more so that IT upgrade them. Unpatched servers have the potential to do more harm given the number of devices, network nodes and client devices the servers touch experts say. Moreover, like Windows XP, many IT managers don't see a need to upgrade those systems, many of which either perform perfunctory functions, while others run mission-critical systems or hardware that can't easily be upgraded to newer server OSes.

"Windows Server 2003 was a very popular product for us and our partners," said Mark Linton, senior director of portfolio and product management within the Microsoft Worldwide OEM division, speaking at a media gathering in New York organized by Dell Software. "The challenge with that is having folks migrate off a popular operating system. We've had that Windows XP discussion and it's a similar story here."

Clock Is Ticking
Just like many organizations let last year's Windows XP end-of-service deadline come and go, many will do the same with Windows Server 2003, though many have it on their agenda for this year. Nearly one-third or about 32 percent of Redmond readers indicated that upgrading old Windows Server 2003 systems is top priority this year, as noted in last month's cover story ("Marching Orders"). It ranked fourth and was closely behind related activities that include virtualizing servers, replacing aging server hardware and upgrading network infrastructure.

Nevertheless, many organizations may not beat the clock. A survey of 500 IT professionals conducted throughout last year by application remediation tool vendor AppZero Inc. found that 65 percent will not complete their Windows Server 2003 migrations by July 14. Some, 29 percent, will complete those upgrades by the end of the year, while 10 percent said sometime in 2016 and 27 percent claim they don't know. Nearly one-third (31 percent) have no upgrade plan while 16 percent said they didn't even know that Windows Server 2003 comes out of service on July 14. Many are still researching their options.

Rory McCaw, managing principal consultant with Toronto-based Infront Consulting Group, says his clients started to get more serious about the pending deadline at the beginning of the year. But he warns some with larger Windows Server 2003-based systems may not get them all mitigated in time. Those facing that situation will have to prioritize, McCaw says. "We have a few customers that have 3,000-plus systems on 2003 and there's no way they're going to get 100 percent of those by the deadline," he says. "They're really looking at the mission-critical apps, what they can decommission, and maybe even the very easy workloads, like Web workloads that can migrate pretty easily, and getting those converted."

Migration Planning
Microsoft offers its own tool, called the Microsoft Assessment and Planning (MAP) toolkit. "That's where we recommend customers go to first," Linton explained at the New York media event. "It allows you to do a basic profile of your network and look at your Windows Server 2003 boxes or instances and understand what workloads you're running. That's the first step. Are you doing identity, branch office solutions, is it a SQL Server box running a departmental line-of-business app, or is it more complex? Being able to categorize what the servers are doing is really the first step in saying ‘where am I at, are these basic workloads?' because that's a faster migration. If I have a bunch of custom code, 32 bit apps that I wrote 15 years ago, that's more complex."

In addition to Dell, there are a variety of partners that offer a variety of Windows Server 2003, application and Active Directory migration tools including AppZero, Binary Tree, BlueStripe Software, Hewlett-Packard Co., Flexera Software and Lakeside Software Inc.

Linton said depending on the complexity of an infrastructure, number of applications and hardware that requires migration or mitigation, it can take anywhere from eight to 12 months to complete and that presumes they already have the budget and needed go-ahead to do so. Like anything, though, McCaw says it depends on the infrastructure. "We try to break it down per application or per server, so we're looking at an average of between five and 30 hours per server or per app," McCaw explains. "That's a pretty big range, I understand, but it depends how complicated and complex the application is."

Often the Windows Server 2003 systems deployed are on old towers, predating the introduction of server blades, as well as converged systems now available. It's not uncommon for them to be scattered and it's typical that they're not virtualized. "In most cases what we're finding is that customers are taking this opportunity to virtualize those systems," says Michael Tweddle, with BetterCloud.

Taking those old systems out of service in many cases requires a number of key decisions, including what version of Windows Server to move to and whether to move the applications and data to a cloud service rather than deploying new servers. Among the key issues, many applications and hardware interfaces developed for Windows Server 2003 and its predecessors are 32-bit and in a good number of cases won't simply run on a newer version (Windows Server 2008 and higher is 64-bit-only). The biggest issue facing those who need to move applications or data running on Windows Server 2003 systems is Active Directory remediation because the domain controller structure on newer versions of Windows Server aren't compatible. Overall the effort can be costly.

"It's very expensive to move," warns AppZero CEO Greg O'Connor. "If you're assuming the application doesn't have source code, and you're going to migrate, some of the system integrator partners that we have will charge you somewhere between $2,000 and $5,000 per machine. If you have 10,000 machines, it can cost $20 to $50 million and probably two or three times that if you have to buy new software and a landing pad. You can quickly get into a $100 million budget. Even though these are large banks with lots of money, that's expensive."

Many organizations will have no choice but to bring in outside help, given their already burdened IT staffs. In the case of MaineHealth, Paul Caron, supervisor of information services, said at the Dell Software event his IT team, while engaged in a Windows Server 2003 and Active Directory DC consolidation effort to enable access to the EMR system from any location, had only so much bandwidth for the project. "I supervise a team of 10 that deals with a lot of things including VMware, Citrix, SQL Server, physical servers, virtual servers, and I don't want my staff to become migration experts," he said. "We have so much in the hopper."

Caron said his team initially tried doing it themselves using the various free migration tools from Microsoft including the MAP tool for Windows Server 2003 for an e-mail migration. "We had a poor experience with the data tools before we engaged with Dell," he said. "We were able to piece together what we needed and we made it work, but there were too many planets lined up for that migration. Things worked well, but we really doubted we would ever get that again. We had great partners on the opposite side," referring to Dell Services and Microsoft.

Like many embarking on the migration will find during the discovery phase, MaineHealth found it had many more Windows Server 2003-based machines than it realized. That was the result of shadow IT, where people with limited skills were putting boxes under their desks. "In the one organization that had one domain controller, we asked how many servers do you have, they responded, ‘Two. Well... three, really. Well... really four.' Of those servers, two were server-class hardware, two were running on PCs. They didn't know."

Legacy Systems
Once the discovery is made, Active Directory must be reconfigured when running on a newer version of Windows Server (see "Modernizing Your Active Directory Domains,"). Because MaineHealth wanted to consolidate its Active Directory domains irrespective of the Windows Server 2003 deadline, that became an opportunity for Caron. MaineHealth decided to collapse the 21 Active Directory domains covering all of the hospitals in the network to just one to provide common authentication and security of the patient records stored in the new EMR system. "Our biggest business driver was one patient, one record," Caron says. "If anybody has had to stop into the emergency room and has to fill out the 27 pages worth of medical history, it's a pain in the behind. With our vision of one page, one record, if you're at another hospital that's part of the MaineHealth family, they would have access to your ecords in a secure fashion."

Like many IT organizations in Caron's shoes, MaineHealth didn't have the internal resources to take on that project so it brought in Dell Services, the consulting and systems integration division of the Round Rock, Texas, computer and systems provider. Not surprisingly, Dell Services used a variety of tools from Dell Software, primarily those it gained in its 2012 acquisition of Quest Software, such as ChangeBASE for compatibility testing and Migration Manager for upgrading the Active Directory domains.

Besides consolidation of the domains, there are numerous other disparities between the iteration of Active Directory in Windows Server 2003 and the current release, says Alan West, founder of XMS Solutions Inc., a Henderson, Nev.-based provider of migration services.

"There are also authentication security differences between the two operating systems from an Active Directory standpoint and Kerberos authentication differences with NTLM," West says. Among other authentication-level differences are Server Message Block (SMB) signing, he adds. "It was supported in 2003 but it wasn't the default. Things like just moving a file from the file server to someplace else, now in a modern OS requires that you authenticate I am who I say I am, this file actually coming to me and not that you're spoofed with SMB signing."

Overall, updating Active Directory schema to migrate to newer versions of Windows Server isn't a risky or difficult undertaking, says Rick Claus, a senior technical evangelist at Microsoft, who spoke about Windows Server 2003 migration at the most recent TechMentor conference in Orlando, Fla., which is produced by Redmond parent company 1105 Media Inc.

"I will let you know from personal experience, I have yet to find an issue where a schema has been unable to be upgraded or is damaged after the fact, unless you're running a very old version of Exchange still and that was the biggest gap that we had, or unless you had some developers who were cowboys and decided to go off and randomly change object IDs inside Active Directory, and just screw something up," Claus explains.

Key Choices
Microsoft and others recommend organizations needing to rid themselves of any number of Windows Server 2003 systems (and in some cases Windows 2000 and even Windows NT, which Microsoft stopped supporting long ago) to upgrade to modern server architectures or consider a hybrid or even public cloud alternative. In instances where the IT decision makers have opted to replace it with on-premises-based systems, Microsoft and most experts advise choosing the most current version of the OS -- Windows Server 2012 R2. Don't bother waiting for a new version of Windows Server this year -- ­Microsoft last month announced the new version, dubbed vNext, won't arrive until 2016.

Microsoft will selectively offer large enterprises that have migration strategies in place some refuge. Through custom support agreements, the company will continue to offer specialized updates, though the price is high: anywhere from $1 million to $2 million per year and the fee will double every year (Microsoft wouldn't officially confirm those figures). But after three years, even that option goes away. "The big challenge is after three years there is no more support agreements and some of these enterprises have 20,000 machines and you can only move 3,000 a year, so I think that's going to be a big problem for a lot of people," says AppZero's O'Connor.

File and Print Servers
In many organizations, especially branch offices, file and print servers are still running on Windows Server 2003-based systems. Many organizations can use that as an opportunity to modernize their network infrastructures with bandwidth acceleration hardware or services from the likes of Akamai Technologies, Cisco Systems Inc., F5 Networks Inc. or Riverbed Technology. By doing so they can centralize their file services now while getting the same performance available when they were stored locally.

Alternatively, others are using the Windows Server 2003 end-of-service deadline as an opportunity to eliminate local servers by moving to cloud services such as Office 365. By moving all these features off-premises, often the only time a server is even needed in a small or branch office is if it's running a legacy phone system or, perhaps, routing jobs to network printers.

One way to eliminate print servers is to use third-party tools such as PrinterLogic, which creates printer objects and uses the DNS settings on the local network to route the print job to the user's printer of choice. The system can be managed by an administrator off-site, if the location doesn't have one, says Jarrrett Taylor, founder and CTO of PrinterLogic. "It's a real fertile time to say, ‘Do we need these print servers?'" he says. "With our platform, you just get rid of them, and you put an agent on the desktop and you use our administrative console to manage the printer objects. And then our agent takes care of it on the desktop and then they don't need to have those servers at the branch locations."

Expect Surprises
Regardless of size, undergoing a Windows Server 2003 migration effort will inevitably result in some unexpected findings along the way. Caron at MaineHealth said the best approach is to push back with management when unrealistic expectations are set.

"Expect the unexpected," he said "We faced a variety of challenges, whether it was senior management tapping us at the door saying we need this done by June, or servers crashing while we were in the process of just looking at them," he said. "We never found high-quality systems that were well designed that we could take over rather than keep them in place. We've had to demolish and push out as quickly as we could so we could get rid of some of these older clunkers."