In-Depth

Troubleshooting Software Update Services

With Microsoft issuing a service pack for its new updating tool, here's what I've learned since writing my original article on implementing and troubleshooting Software Update Services.

• By Jeremy Moskowitz
• 05/01/2003

With that in mind, this troubleshooting guide assumes:

• Your clients are running Windows 2000 SP3 or Windows XP SP1, so you didn't need to deploy the client .MSI piece (it's included in the Service pack). The rest of this article should work if you’re deploying the .MSI client piece to, for instance, to a Win2K machine with SP2, as described in the original article to—this isn't how I did my testing, so your mileage may vary.
• Your SUS server has SUS with SP1. Either you loaded a server with a fresh copy of SUS SP1 or upgraded your SUS server with SP1 (with the same installer). You’ll find SUS SP1 at http://www.microsoft.com/downloads/details.aspx?FamilyId=
  A7AA96E4-6E41-4F54-972C-AE66A4E4BF6C&displaylang=en.
  (Tip: SUS with SP1 allows installations on domain controllers and Small Business Server. SP1 has a bunch of other new features, which are detailed in Microsoft’s whitepaper “Applying Service Pack 1 to Microsoft Software Update Services version 1.0” at http://www.microsoft.com/windows2000/windowsupdate/
  sus/sp1relnotes.asp.)
• Downloaded the newest WUAU.ADM template file from Microsoft. If you haven’t already done so, you’ll need it to follow along with this article. WUAU.ADM is not included in SUS SP1; it’s an additional download at http://microsoft.com/downloads/details.aspx?FamilyId=
  D26A0AEA-D274-42E6-8025-8C667B4C94E9&displaylang=en

Installation and Verification
SUS installation is straightforward, especially if you have the first article handy: Create a new GPO, and load the WUAU.ADM file. You’ll likely not want to create the GPO at the domain level, because if you do, you’re telling all computers (servers and workstations) to automatically download updates (and possibly reboot). So, create a test OU called “SUS Clients” and move about 10 computer accounts into it. Then link your new GPO to that OU.

Once created, be sure to use the latest WUAU.ADM. When you do, you’ll get even more policies to play with. I detail the new policies later, when I discuss SUS modes.

Figure 1. The newest .ADM file loaded in a properly configured GPO. (Click image to view larger version.)

Is It Working Yet?
At this point you should have the server side set up correctly and performed an initial synchronization to the Microsoft mothership. You should also have approved updates applicable for your Win2K and Windows XP clients and servers.

Are your clients getting the updates? No? Try these basic steps:

1. First, make sure the Automatic Updates service is running on the client. If not, start it up—and figure out why it didn’t start! Questions to explore: Did you actually distribute the .MSI (for SP2), or is SP3 really installed? The fastest way to start and restart the service is to use the NET STOP/NET START WUAUSERV command.
2. Next, determine if clients are getting the GPOs you deployed. From the client computer, run GPresult (this utility is in the Windows 2000 Resource Kit). Do the same thing with XP, even though XP (with SP1) has a working GPresult—the one in the Win2K Resource Kit is better for registry-related troubleshooting. You could also use Windows XP’s Resultant Set of Policies (RSoP) information in the Help and Support Center, which will display the Registry entry in a readable fashion.

Running gpresult /v /c (v=verbose, c=computer) on a client machine should return the following:

The following settings were applied from: {the name of your GPO here}

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate
ValueName: WUServer
ValueType: REG_SZ
Value: http://Vmserver1

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate
ValueName: WUStatusServer
ValueType: REG_SZ
Value: http://Vmserver1

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate\AU
ValueName: NoAutoUpdate
ValueType: REG_DWORD
Value: 0x00000000

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate\AU
ValueName: AUOptions
ValueType: REG_DWORD
Value: 0x00000004

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate\AU
ValueName: ScheduledInstallDay
ValueType: REG_DWORD
Value: 0x00000000

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate\AU
ValueName: ScheduledInstallTime
ValueType: REG_DWORD
Value: 0x00000000

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate\AU
ValueName: UseWUServer
ValueType: REG_DWORD
Value: 0x00000001

KeyName: Software\Policies\Microsoft\Windows\WindowsUpdate\AU
ValueName: RescheduleWaitTime
ValueType: REG_DWORD
Value: 0x00000005

XP Caution (times two): Running Windows XP’s own GPResult doesn’t display the registry punches shown here. You may have to dip into the registry to make sure the changes are made on the correct keys. Still don't see it? If you're on an XP box, remember that XP takes multiple restarts to get the latest GPOs.

Another way to verify that the client has downloaded the Registry changes is to simply examine the Registry’s contents. In the HKLM|Software|Policies|Windows key, look for a subkey called WindowsUpdate. If this key is present, it will contain values that point at the SUS server and subkeys that contain the schedule information.

At this point, you can analyze what's in each Keyname, Valuename and Value above.

Of note is the Value of the first two settings, which specifies the server. The rest correspond to the other policies. To test this, change a value and make sure your client responds. On the client system, you'll likely need to do a secedit /refreshpolicy machine_policy for the change to take effect immediately (on XP, the command is gpupdate).

What’s Really Going on Under the Hood?
There's another magic place inside the client computer’s registry that SUS cares about. Group Policies and their registry punches dictate how the client will respond to our SUS server, but the Automatic Updates service has its own “internal” places that describe the current “state” of the client. Those registry locations can be found in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\ WindowsUpdate. If you inspect this place in the registry, you can usually determine the current state of the client. Notice the DetectionStartTime entry in Figure 2? Here, the client should check for new updates and start to download them at this time. Several seconds later, LastWaitTimeout should appear (see Figure 3).

Figure 2. Peering into the Registry, you should now see DetectionStartTime, which indicates to the client when it should check for new updates. (Click image to view larger version.)

Figure 3. Same key, only this time LastWaitTimeout appears, indicating how much time should lapse before checking the SUS server for updates.(Click image to view larger version.)

LastWaitTimeout tells Automatic Update “If you don’t find anything to download now, wait for the specified amount of time before checking for updates.” According to the SUS deployment documentation (it’s not specifically spelled out in the Policy help), the number in the data column indicates that “every 22 hours minus a random offset, your Automatic Updates client computers will poll the server running SUS for approved updates to install." If the client doesn’t find any new updates at the DetectionStartTime, it will check again at the LastWaitTimeout time.

Figure 4. If no new updates are found at the DetectionStartTime state, Automatic Update reverts to the time shown in LastWaitTimeout to check for updates. (Click image to view larger version.)

In either case, the client will not look for new updates until after the installation of anything that was downloaded. This is important: If there are new critical updates to be downloaded, they simply won't be downloaded or installed until this first batch is processed.

If you want to "reset" the client and have Automatic Updates think it’s at square 0 again, open the client system’s Control Panel and select the Automatic Updates icon. Uncheck the “Keep my computer up to date…” option (see Figure 5) and click Apply. Then, select the same checkbox and click Apply again.

Figure 5. To reset the client to immediately check for updates, go to Control Panel | Automatic Updates, then click the “Keep my computer up to date…” option.

If you’re using Group Policy and specifying settings via the WUAU.ADM template, your checkbox will be automatically filled in and, hence, you won’t be able to unselect and reselect it. Instead, you’ll need to disable the GPO that houses WUAU.ADM and refresh the client with secedit for Win2K machines or gpupdate for XP machines. Then, you can actually watch the registry punches in this part of the registry get “reborn” while reading the log file and watching the event viewer to see that what’s going on under the hood is really happening.

Modality: Mode 3 vs. Mode 4
The ADM template we loaded has settings for specifying how SUS clients will obtain updates. The most critical policy to the entire operation, Configure Automatic Updates Properties policy (see Figure 6).

Figure 6. Use the Configure Automatic Updates Properties dialog to specify how SUS clients will obtain updates.

From the client’s perspective, SUS simply does three things and three things only: Detect and download updates from an SUS server; install them on the client machine; and—potentially—reboot the machine. In each of SUS’ modes, it performs the download and the update differently. Let’s examine each SUS mode.

Mode 3
When modifying the Configure Automatic Updates Properties policy, we need to first distinguish between each setting's function. Mode 3 says “Download the updates whenever. Once they’re downloaded, prompt for the initiation of the install.”

The key factor of Mode 3 is that it doesn’t prompt the user. Instead, it prompts the local administrator. Why would you want to do this? First, you might have an environment in which every user is an administrator of their local machine. This allows them to download updates automatically, then decide for themselves when they're ready for the install. Mode 3 can also be set up on your servers. You'd have a separate GPO for servers, set up to react to it. Each server downloads the updates, then, once an administrator logs on, she can simply install the update and reboot the machine if necessary at an appropriate time.

To repeat: Once the updates are downloaded, they are not installed until an administrator logs on to that machine and accepts the updates. I believe this causes the majority of people’s woes with SUS. They're using Mode 3 and not actually getting their updates installed on the client systems!

Mode 3 has nothing to do with scheduling whatsoever. The computer downloads the updates from the server whenever it feels like it (up to 22 hours), then just holds the updates until the local admin completes the install. If you look really closely at Figure 6, you'll see a note that states: "The following settings are only required and applicable if 4 is selected." So, let’s talk about Mode 4.

Mode 4
Mode 4 says “Download the updates from the server whenever the computer feels like it. However, once all the downloads are done, I want the update installation to happen at a specific time (presumably when no one is around.) Prompt whoever is logged on that the machine is going to reboot.”

Tip: While you’re configuring Mode 4, ensure you've set the policy for the time you actually want it to commence. Midnight is 00:00; if you set the policy to run at 10:00 AM, you might irritate a few users. Note that the updates should start installing directly on the stroke of that hour (or within about five minutes of that hour.).

If there are updates available (and the computer is on), it’s a crapshoot as to when the client will actually install the updates—assuming you’re using Mode 4. It could perform the updates today at the specified time, or it could be tomorrow at the specified time. The next scheduled time is determined when the client completes the download of the update.

But what happens if the computer is on, and the user is working? Will it just reboot the machine with the user’s data unsaved? This is where one of the new WUAU.ADM policies comes in. If the "No auto-restart for scheduled Automatic Updates" policy is enabled, the target computer won't reboot until the user on that machine says to. This is a major improvement over the original version of SUS, which would kick off all users when it was time to reboot.

But what happens if the computer's off and it misses the time it should have installed the updates? That’s where another one of the new WUAU.ADM policies comes in. If the "Reschedule Automatic Updates scheduled installation" policy is enabled, you can force it to install the updates the next time the computer boots. Once started, it will wait the assigned number of minutes before performing the installation. Set it to a low number, like one minute. This will force the computer to install the stored updates, and reboot if necessary before the user can even log on! If you set to a number like 20 minutes, the user will be interrupted by a prompt.

So now, assuming the updates are downloaded and scheduled (based on the ScheduledInstallDate registry entry; see above) you’ll get an Event 18 (in Mode 4) in the event log telling you when the installation will occur.

Figure 7. A computer in Mode 4 will show Event 18, specifying the installation procedure for updates and the scheduled time when the updates will occur. In Mode 4, the machine will install the updates as soon as the user tries to log on to that machine and reboot if necessary.

Figure 8. Successful updates on computer in Mode 3 requires local Administrator to be logged on that client. The machine will download the updates as scheduled, but it’s up to the local administrator to install them.

SUS Q&A Q: "[Before loading SUS], why are end users able to install applications or run Windows Update on their machines at all?" A: You're totally right. My brain was still in Windows 98-land. Only admins can run Microsoft Windows Update. Q: "I followed your instructions on your original article, but I can’t seem to be able to get my “automatic updates” download from my server. It keeps pointing to http://v4.windowsupdate.microsoft.com/en/default.asp A: As I just stated in this article, admins can still go to Windows Update, but users will get turned away. Even so, in both cases Windows Update from within the browser points to that Microsoft site. Performing this SUS addition doesn't change that; you just bring additional scheduling functions to the table. Clearly, you don't want admins going to that site anymore for updates. You can use another policy to stop this from happening. Do so by enabling the policy by clicking on User Configuration | Admin Templates | Start Menu and Taskbar | Remove links and access to Windows Update. When you do, users and admins will get an Access Denied error if they perform a regular Windows Update (see Figure). If users try to use Windows Update rather than Software Update Services, they'll be denied. (Click image to view larger version.)

Troubleshooting SUS
If you’re not getting this far, you might check out the Client’s Windows Update log file at %windir% \Windows Update.log. However, the client-side log doesn’t specifically tell you where in the process the client might be failing, nor does it tell you which patches’ .cab files from the server were successful and which ones bombed. At this point, the client-side log isn’t all that useful. However, I've seen some logs show specific error codes to signify specific problems (i.e. it can’t reach the server).

Those known error codes are detailed in the excellent SUS-SP1 deployment paper (http://www.microsoft.com/windows2000/windowsupdate/sus/ susdeployment.asp). It also has some server-side log reading tricks to help with troubleshooting. It even has some tricks I didn’t know about when I first wrote my original article. If you're successfully getting updates and event log entries in Mode 3, switch to Mode 4 and wait a day or so. At that point the client will automatically download the updates (within 22 hours) and install them at the time specified in the policy.

Hopefully, understanding the registry pieces of SUS clears up how and when SUS does its thing. And understanding the modes clears up why SUS it does its thing. Together, you can now deploy SUS with SP1 and do your thing.

Thanks to Bill Boswell for his technical input and troubleshooting assistance. Thanks to the SUS team at Microsoft for answering my questions and contributing feedback for this article.