In-Depth

Systems Engineering: Stressing Out over IIS

Test with Microsoft Web Stress Tool to tune IIS <i>before</i> it hits the wall. Then use Performance Monitor to measure the improvements.

• By Pat Filoteo
• 01/01/2000

Microsoft Internet Information Server (IIS) 4.0 is one of the fastest and most scalable Web solutions available. For most activities, such as running intranets or informational-type Web sites, the out-of-the-box configuration needs little optimization. On the other hand, a high stress (load or availability) environment, such as an e-commerce, search, or mission-critical intranet, will probably require tuning and benchmarking to be successful. If you’re eyeing a Windows 2000 and IIS 5.0 installation any time soon, be assured that the stress testing process and tools I describe here will remain the same.

How can you create an environment that allows your solution to grow with increasing traffic or complexity using IIS? This article discusses how to tune IIS, along with why these changes are necessary. I describe the use of Performance Monitor in conjunction with the Microsoft Web Stress Tool, so you can measure the improvement. The article primarily targets IIS administrators, rather than developers. Developers looking for improved performance (and curious administrators) should review the MSDN documentation available at www.msdn.microsoft.com (no subscription necessary).

Before tuning IIS, it’s important to first understand how it’s set up. When IIS was first released, a Pentium Pro 200 (single processor) server was considered very fast. So, most of the settings for IIS are optimized for a less capable machine. Faster hardware and processors hide this inefficiency; but to maximize the potential, you need to make certain modifications.

Before touching your keyboard, read the complete article and come up with a plan prior to implementing changes. The order of change doesn’t matter, and some rebooting (and time) can be saved with some forethought.

Change the Services

The first setting to change is the Server Service, located in the Network Properties dialog. When you right-click and choose Properties, there are three options: Network Services, File Services, and Mixed. Select Network Services and reboot to initialize the changes. This setting controls the optimizations of the Windows NT operating system regarding network listeners and memory for caching.

Change the Application

Next, modify the performance settings from within Microsoft Management Console Internet Services Manager by right-clicking on the Web site and choosing Properties | Performance. Slide the performance bar to the far right (100,000-plus hits). If the Web server is a dedicated machine, there’s no reason to “tune it down.” This setting directly affects the memory, threading, and listener threads that IIS allows to be in use.

Next, make sure that the “keep-alives” box is checked. This setting is one of the least understood performance enhancements available. Without this setting, every GET request requires a TCP handshake. Needless to say, that’s a lot of overhead for your clients (not to mention the server). Keep-alives are an HTTP 1.1 specification, so older proxies and browsers will ignore it.

Close the dialog and right-click on the Machine Name. Choose Properties | Edit WWW Master Properties | Home Directory | Configuration | Process Options. You need to modify two settings. First, change the default script engines cached from 30 to 250. For future reference, Microsoft increased this default setting in IIS 5.0 to 250. Under a heavy stress, this allows IIS to keep more interpreters available without unnecessary loading or unloading. Next, change the “templates cached” value from unlimited to a value equal to 25 percent of the number (count) of ASP pages. A template is the machine code version of an ASP page—basically, a precompiled version of the ASP page, analogous to the caching of SQL Server Stored Procedures.

The rest of the settings aren’t exposed via the interface and will need to be modified in either the Metabase or the Registry. It’s important to back these up in case something goes wrong.

First, modify the Metabase with the following changes:

cscript adsutil.vbs set w3svc/
AspQueueTimeout 30
cscript adsutil.vbs set w3svc/
ServerListenBackLog 1000
cscript adsutil.vbs set w3svc/
MaxEndPointConnections 1000

If the default installation location was used, these need to be run from the c:\winnt\system32\inetsrv\adminsamples directory.

The AspQueueTimeout is set to infinity by default. This code changes it to 30 seconds. Because very few users will wait more than 30 seconds for a response without hitting “Refresh,” there’s no reason to keep a page in the queue for any longer. ServerListenBackLog and MaxEndPointConnections must be set to the same value, which controls how many simultaneous client requests IIS can provide. Keep in mind, this doesn’t limit you to 1,000 clients—they won’t all be accessing the machine at precisely the same time. Instead, it allows for several thousand simultaneous users (depending on content and network latency). Content latency is the time it takes content to be served from the disk or cache to the wire; network latency is the time it takes a request to travel from the client to the server (or vice versa).

By default, IIS allows 10 ASP worker threads per processor. So, in a dual-processor machine, only 20 ASP pages may execute at a time. This was done when IIS was first released to keep it from saturating weaker machines. A Registry parameter needs to be added to the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\
W3SVC\ASP\Parameters:
Dword: ProcessorThreadMax
Value: 14 (hex) (20 in decimal)

This doubles the maximum worker threads. Raise this value further only after extensive testing—increasing the value too high could actually slow the server down.

If your Web site makes connections to a database (and how many big sites don’t?), the next setting might provide some significant improvement. Microsoft Data Access Components (MDAC), by default, allocates 7M of contiguous memory for every recordset. The memory is released as soon as the recordset is closed, but that’s still a big chunk of memory to give up on a busy server.

HKEY_CLASSES_ROOT\CLSID\
{c8b522cb-5cf3-11ce-ade5-00aa00
44773d}\Flags
Dword: MaxBlock
Value: 00004000(hex)

This setting modifies the default memory allocation to 16K. Needless to say, there are many more 16K blocks than 7M blocks available.
Another database performance setting is the MaxPoolThreads. The setting is actually for network requests from IIS (among other things) and can, therefore, limit the number of simultaneous links (connections) to a remote database. The default setting is eight.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\
InetInfo\Parameters
Dword: MaxPoolThreads
Value: 00000020 (hex, 32 in decimal)

A Note on Service Packs

Finally, the latest NT Service Pack (SP5 at the time of this writing) should be applied. Service Pack 4 included some significant improvements in the throughput and IP stack in general. SP5 includes these improvements, plus a new optimization for the ASP cache, memory handling (user layer) by the operating system, and other sundry bug fixes. Pursuant to Microsoft’s new policy regarding service packs, SP6 released after most of this article was written. SP6 has been designated an “optional” installation. I would strongly recommend that you review KnowledgeBase article Q241211 (SP6 list of fixes) before deciding to not apply it. As always, you should test any software before placing it on the production servers.

Unfortunately, most IT shops have a relatively delayed SP rollout schedule due to the sheer number of environments (DB, Web, Mail, and so on) that must be tested. I suggest testing the Web servers separately to minimize delay because the SPs also contain security updates as well as fixes.

Time to Test

Now that all the settings have been changed, it’s time to stress test and measure how much traffic the server can really handle. The tools to use include Performance Monitor and the Microsoft Web Stress Tool (formerly known as “Homer”). If you already have a stress tool, feel free to use it.

For the first run, start Performance Monitor with these counters, which I recommend you log and chart, to make sure you don’t miss something:

From the ASP object:

• Requests per second
• Requests executing
• Requests queued
• Request execution time

From the Processor object:

• Percentage of processor time

From the Process object:

• Private bytes (inetinfo.EXE)

From the Web object:

• Current connections

After these settings are set up, they should be saved to speed up later testing. Make sure to change the timing on the chart to be at least six seconds between ticks. A six-second delay makes the chart show the last 10 minutes of activity. This should be plenty for most testing runs. However, as a pre-rollout, you should perform a 12-hour (or longer) test. After a few runs, you can remove the unneeded counters to clear out the clutter. After testing becomes a habit, it’ll become fairly obvious when a change makes a difference in performance (beneficial or otherwise).

You can obtain the Web Stress Tool at http://homer.rte.microsoft.com, where you’ll also find documentation on its use. The tool creates a Web stress service, and if you’re in the administrators group of multiple machines that have the tool installed, you can create a load using many machines.
After the service is installed, record a script by browsing your site. The more pages you record, the more realistic the test. There will be times, such as during a troubleshooting session, when a particular page will need to be stressed. This is OK too, but in this article, you’ll be performing a general test. After the script is prepared, start the stress test.

With the stress test running, watch the counters to see where potential bottlenecks might be occurring. Each counter relays this information a little differently. Here’s a generic interpretation of the counters and what a given result might mean.

• Requests per second: This is used as a gauge of how much stress the machine is under. If this value is too low for the test you’re attempting (say, one where you want to answer the question: How many requests can this server handle?), you might need to modify the script or stress engine settings.
• Requests executing: By default, only 10 requests can execute simultaneously. Earlier you changed this value to 20 (14 hex). If the counter is maxing out at 20, but the processor isn’t reaching 100 percent utilization, then increasing the count might help. Anther suggestion is to streamline the ASP code so it executes faster, having fewer lines to interpret (include files add greatly to the length). If the processor is already pegged at 100 percent, increasing this value won’t help.
• Requests queued: If this value is constantly increasing, the stress test is overwhelming the server. It isn’t unusual for a request to queue, but a constant increase means that the server isn’t keeping up. This indicates a poor execution time or a script that artificially generates more requests than is realistic. Real-world usage typically shows that it takes 10 simultaneous users to keep one ASP page executing.
• Request execution time: This value is measured in milliseconds. If it’s more than 2K (2 seconds), an improvement needs to be made. ASP pages (or any request type) should ideally finish in less than 1.5 seconds. Perhaps the database isn’t keeping up or the logic in the ASP is too complex for a quick interpretation. This indicates that more in-depth testing is needed (single page, execution of query outside IIS, and so on).
• Percentage of processor time: This shows the CPU load. If the CPU is maxed out at 100 percent, increasing the load won’t make the server display more pages. Most test boxes perform double duty as database servers, lacking the horsepower of the production machines. Try to minimize differences between actual production and testing situations to improve the correlations. Changes that degrade performance will still show up, but it’s easier to get a one-to-one comparison if the environment is the same.
• Private bytes (inetinfo.EXE): This value dances around as objects are loaded and unloaded. The important item to note is whether there’s an upward trend, which indicates a memory leak. I just completed a 1.5 million hit stress test against IIS 4.0 (SP5), and no leaks showed. If it’s leaking, find out what else is installed and try to isolate the problem.
• Current connections: Like the pages executing, this is a general reference to the load you’re placing on the Web server.

Consider this stress testing primer as a place to start. Other considerations not discussed include the network, I/O, and Web farm scenarios (stress the firewall, DNS, load balancer, etc.). As I said earlier, with IIS 5.0, which will ship with Windows 2000, the process (and tools) will remain the same. Microsoft will be making some additional improvements, such as removing all settings from the Registry (100 percent Metabase), improved SMP support, and additional intrinsic objects. Performance Monitor is getting a facelift to become more user friendly. However, the testing procedures covered here (although not the actual settings) will be the same and should provide good insight into what works and what doesn’t.