Service Level Agreements Too IT-Centric, Forrester Report Warns
Service level agreements (SLAs) are a popular tool for measuring how well IT
is providing service to its customers, the organization's business users.
new Forrester Consulting report based on a survey of 389 global technology decision-makers
who work with service management finds serious deficiencies with current SLAs.
The study sheds light on how SLAs are currently being used, the degree of end-user
dissatisfaction, and what IT can do to correct the problem.
At the heart of the matter is the IT-centric nature of the service level metrics,
most of which have little or no relevance to business end users.
[T]here is one important question that IT executives should consider
in these projects: Whose services are we talking about? Do people -- especially
colleagues on the business side -- really care about how well IT performs
their IT processes, such as fixing incidents, making configuration changes,
and provisioning server environments? Isn't it much more relevant to discuss
and measure the performance of the services that the business cares about
-- such as processing orders, dealing with customers, and making and shipping
The report, Managing IT Services From The Outside In, commissioned by
Compuware Corporation, explores the effectiveness of an organization's processes
as well as what technologies they use. The survey asked about service management
processes, how content for service-level agreements is developed, and how service
quality is measured.
Although 81 percent of respondents are involved with formal SLAs, they meet
those expectations only three quarters (74 percent) of the time. "The major
reason cited for failure to meet the agreed-upon service levels was that 'The
business had higher expectations than we could meet.'" Forrester says this
is a "clear demonstration of poor business and IT alignment."
The majority of respondents use IT-centric metrics (including network and server
availability, number of incidents, and number of failed transactions). Of the
321 respondents using SLAs, 56 percent establish a fresh metrics baseline and
33 percent use the previous year's performance at a guide. The baseline often
involves only IT personnel.
Business users understand that these measures may be needed by IT, but Forrester
points out that systems and networks are "extremely reliable" and
"managing the applications portfolio is becoming increasingly complex with
end users, inside and outside the enterprise, [as is] running their apps on
different devices and over different networks."
Communication Is Key
Communication and coordination between IT and business users are two areas Forrester
says IT must examine; 41 percent of the respondents don't even "collect
the right service quality data together for their IT and business executives,
so they do not even attempt to provide these reports at all," the report
Poor communications may also explain why SLAs may not be "agreements"
They are not negotiated with the business partner because there is no
dialogue process and because the business does not understand terms like "percentage
availability" or "amount of storage." So when reviewing the
IT service at the end of a budget period, business managers can say, "It
was not good enough for me to do my business," even if the IT service
levels were met on a mathematical basis from the IT point of view.
Forrester reports that the focus of metrics will move from IT to the business
as communication increases between the two groups.
The survey found that although 87 percent of the respondents said they use
tools to monitor the end-user experience (EUE) for some their business-critical
apps, nearly two-thirds (64 percent) acknowledge that they learn about problems
only when users contact the help desk.
There's no question about the impact of poor application performance: 57 percent
say it increases business costs. Lost revenue and negative impacts to the satisfaction
of external customers were each cited by 48 percent. Even so, not all organizations
are using SLAs or believe they're needed at all. The study reports that 17 percent
of respondents don't have formal SLAs, and half of these acknowledge the need
but have not instituted a formal SLA program, and 37 percent of these respondents
said "the business hadn't asked for SLAs." Nearly a quarter (24 percent)
said SLAs were "overkill."
Two movements are affecting IT, according to Forrester. IT service management
(ITSM) "supports the change of an IT organization from a support group
into a service group." ITSM includes introducing ITIL or reorganizing IT
operations into process groups (instead of tech silos), as well as "the
introduction of account management, and even marketing concepts within the IT
organization." ITSM is inward-looking and only indicates "how well
the IT shop is being run." It does not address the need to know how the
business is being run from the perspective of the business unit and the end
Business Service Management (BSM) looks at a different set of metrics, including
the business transaction volume, transaction speed, and the processing backlog
IT operations must establish business-centric SLA metrics, and to do that, IT
must "understand their services from the outside in." Forrester recommends
...spend time with business users and understand how they, the users,
perceive the service and document what is important for business success or
failure. Companies in this phase of service management usually focus on the
quality of service around business applications. They need to analyze and
document the dependencies between business applications and the details of
IT infrastructure in order to better triage incidents or understand true business
impact of pending performance or capacity issues.
Communication is key. The report also claims that many SLA reports lack information
relevant to executives; 40 percent of survey respondents agreed. Forty-one percent
of respondents said they don't provide regular reports to executives.
The report also strongly suggests that leveraging ITIL best practices should
be the first methodology chosen because it specifically addresses IT processes.
"ITIL is best for documenting and standardizing the IT processes themselves."
It notes that quality-improvements programs (including Six Sigma) can help IT
improve its processes. Together, the two methodologies provide "a powerful
combination for continual IT service improvement."
Failure for IT to change its approach to SLAs may have serious consequences.
The study notes that "IT organizations that consider their service management
projects only at the IT service level will continue to disappoint their business
users in the longer term, and the age-old CIO problem of being equipped to demonstrate
value to the business will prevail."
James E. Powell is the former editorial director of Enterprise Strategies (esj.com).