Best Practices in Service Level Management - Communications

Executive Summary

  • Communications respondents generally assessed services that are larger than typical for the assessments from other industries. Almost 46% of the Communications respondents had over 10,000 users for their profiled service, compared to 31% for other industries.
  • Communications respondents operate complex multi-platform environments. However, they do not differ significantly from the total population of respondents in terms of their overall SLM assessment rating. Communications rates 68.7 out of a possible 100, versus 69.6 for the other industries.
  • In terms of Predictive Service Assurance (PSA), only 14.5% of Communications respondents indicated that they used automated tools for IT planning to respond to business events such as mergers. Only 10.4% had fully automated capacity planning processes.
  • Based on their SLM profiles, Communications respondents can be classified into Stars and Low Performers. The two groups differ in the degree to which they have achieved solid basic infrastructure and their Predictive Service Assurance processes. The two groups are at different stages of SLM development and therefore have different needs.
  • Key points of interest about the Communications industry respondents from a Service Level Management and Predictive Service Assurance perspective:

    - A propensity to assess e-business and custom applications operating in multi-platform, complex environments that indicates the need for service management solutions that are interoperable and applicable across applications and services.

    - The high cost of downtime for the service they profiled (estimated at $140,000 per hour) indicates the business importance of the services profiled by Communications respondents. In light of their roughly equal overall ratings and lower overall usersatisfaction it appears that Communications respondents are not getting the return on their infrastructure management investments that other industry groups are getting.

    - A tendency for Communications industry respondents to have achieved a fairly accomplished level in their SLM discipline development, but not to have taken the step to process automation, particularly in PSA efforts such as IT planning and capacity planning.

  • Communications respondents are positioned to take the next step to automate their services and leverage the quality service management work they are doing.

- To do this they will need highly interoperable, robust and scaleable solutions for automation. Based on the high levels of complexity of the services they profiled, Communications industry respondents likely need highly customized Service Level Management solutions.

- Based on the needs they indicate, Communications industry respondents probably should seek a provider with deep expertise in the Communications industries, and in complex SLM. A broad and deep services provider will be required to met the needs of the largest and most sophisticated respondents.

Introduction

  • This paper presents a comparative analysis of the patterns of Service Level Management (SLM) practices of the Communications industry relative to other businesses and organizations. It is one of a series of papers on practices in Service Level Management. The other papers in this series are available on this Internet site.
  • More than three thousand middle and executive management respondents from large organizations have taken the on-line Service Level Management assessments. The information presented in this paper is based on their input.
  • The on-line Service Level Management Assessment evaluates the impact of 22 service level measurements on user satisfaction. The factors include key measurements of availability, performance and recovery. Also included are Service Level Management features, such as Service Level Agreements, reporting, measurement, monitoring, and documentation, resource use, trends in help desk use and software costs. A Service Level Management Rating (scale: 0 to 100) is calculated based on the relationships between these 22 factors and user satisfaction.
  • Four new measures have been added to the on-line Service Level Management Assessment since the last paper in this series, Service Level Management Profiles - Finance, Banking and Insurance, was written. These measures assess predictive service assurance practices.

    Predictive Service Assurance (PSA) enables proactive prediction of future system behavior. PSA translates systems-monitoring data into knowledge that helps optimize both systems behavior and service levels by constructing a sophisticated application and resource system model. The model can aid in capacity planning, consolidation planning and application lifecycle development support.

  • The assessment tool can be found at www.nextslm.org/benchmark.

Key Findings


Communications vs. Other Industries

  • Communications respondents generally followed the overall response with respect to the role of the respondent. Communications had slightly less representation of management titles with the exception of technical support management. Technical support management and consultant titles were somewhat more prevalent in Communications.
  • Communications respondents showed a slightly higher propensity to profile e-business oriented services (36.3%) compared to respondents from other industries (30.4%). A smaller proportion of Communications respondents (25.5%) profiled custom applications, than other businesses (34.7%).
  • Slight differences were found between the computing environments of Communications sites and those of other businesses. Communications had a larger proportion of sites that relied on distributed systems only (54.5% for Communications vs.51.4% for other industries). Only 3.0% of Communications and 5.0% of the other business sites reported using mainframes only.
  • In general, Communications industry respondents described larger user communities when compared to other businesses, both in terms of number of internal users and number of external users.
  • The Communications industry tended to show greater sophistication, relative to other industries, in predictive service assurance (PSA). Use of automated tools to perform IT "business" planning in response to mergers/acquisitions, etc. was more prevalent (14.5% vs. 9.5%), as was use of automated capacity planning tools (45.5% vs. 42.1%) and use of automated configuration tools for workload balancing (47.4% vs. 39.4%).
  • Downtime cost was higher for Communications. Downtime cost per hour in excess of $100,000 was reported by 42% of Communications respondents vs. 29.8% of other respondents, and at $400,000 per hour, Communications reported 11.5% vs. 5.8% for other businesses. Extrapolated average downtime cost in the Communications industry was $140,000/hour compared to $106,000/hour in other industries.
  • Communications industry respondents had an average SLM rating of 65.9 vs. a rating of 67.9 for all respondents. Interestingly, the communications industry score is rising as more data is collected while the other industries' overall score is declining. (In the initial research project communications scored 63.5 against an overall respondents' score of 70.7).
  • Lower Performance status (acceptable vs. unacceptable) was the most significant factor in the lower SLM rating for the communications industry.
  • Seventy-six percent of the communications industry respondents reported having service agreements vs. 75% of all respondents.
  • Availability & Performance Status: 80% of communications industry respondents reported having an "acceptable" level of availability vs. 86% in other industries; 74% reported having acceptable response times vs. 84% in other industries. Since this measure is based on the respondent's perception of "acceptable" levels it may be that the communications industry respondents are simply holding themselves to a higher standard of performance vs. other industries.
  • Communications industry respondents reported having slightly lower customer satisfaction compared to other industries: 43.4% of communications industry respondents reported having neutral or lower customer satisfaction for the service profiled vs. 31.3% for other industries.
  • A more granular look at the Service Level Management (SLM) rating measures suggests that the respondents from the Communications industry represent two subpopulations of sites - a high performing (Stars) group and a lower performing group (Low Performers).
  • The proportion of North American and Latin American sites was similar in both the SLM performance groups, with more than 55% in each. Europe, the Middle East and Africa (EMEA) accounted for 23.1% of the Stars and 34.7% of the Low Performers. The Asia Pacific region accounted for 21.2% of the Stars compared to 8.2% of the Low Performers sites.
  • The Stars respondents described larger user communities than Low Performers respondents. Sites with at least 10,000 users comprised 51.9% of the Stars compared to 40.2% of the Low Performers.
  • The Stars tended to report greater financial exposure to IT downtime. Forty-eight percent of the Stars reported downtime cost of at least $200,000, compared to 34.7% of the Low Performers sites. Fourteen percent of the Stars reported downtime cost of at least $400,000, compared to 8.6% of the Low Performers sites.
  • A significantly greater proportion of Stars respondents (78.5%) gave their site user satisfaction ratings of satisfied or better, than Low Performers (33.4).
  • On a 100-point scale, Stars were rated significantly higher, on the average, than Low Performers (75.9 compared to 55.6).
  • The Stars showed significantly greater sophistication in the area of predictive service assurance (PSA). Stars (22.5%) were four times as likely to use automated tools for IT planning as were Low Performers (5.4%). Eighty percent of the Stars did some form of IT planning, compared to 62.2% of the Low Performers sites. A significantly greater proportion of Stars (69.2%) used automated tools for workload balancing than Low Performers (24.3%). Furthermore, 84.6% of the Stars did some form of workload balancing, compared to 64.9% of the Low Performers sites. Trial and error was the workload balancing approach reported most frequently for the Low Performers (40.5%).
  • A significantly greater proportion of Stars (64.1% vs. 26.3%) used automated tools for capacity planning. Furthermore, 92.3% of the Stars did some form of capacity planning, compared to 64.9% of the Low Performers sites. The capacity planning approach reported most frequently by the Low Performers was manual processes (42.1%). A significantly greater proportion of Stars (50.0% vs. 9.1%) rated their site's capacity planning as mostly or completely effective.

Service Level Management Profiles within the Communications Industry

The analyses presented in this section were based on assessments from the Communications industry only.

A more granular look at the 22 Service Level Management (SLM) measures suggests that the respondents from the Communications industry represent two subpopulations - a higher performing group and a lower performing group. For example, while Communications industry respondents reported higher representation in the more sophisticated options for alert methods, reporting practices, and better than average restore times (for system outages, resource conflicts and software outages) - they also reported a higher percentage of respondents in the lowest level of sophistication for those measures.

The chart profiles these two groups on the 22 SLM measures. All 22 measures are presented on the same scale, where "0" is the worst and "1" is the best possible score.

Stars and Low Performers in the Communications Industry

For the most part, the Communications industry high performers exhibit the SLM profile of the "Stars" discussed in Best Practices in Service Level Management - Baseline Research Report. The Stars ("High SLM" on the charts below) distinguish themselves from the Low Performers ("Low SLM" on the charts below) by doing many things well.

The two profiles outline stages of development in SLM implementation and provide evidence that meeting basic user needs should be everyone's first step in the evolution. The elevation of the IT organization to business leadership is then achieved through extending consistency and stability into high quality customer care.

Underlying Infrastructure

Stars provided a solid infrastructure and used more sophisticated SLM methods. Stars had faster recovery times from hardware and software failures and failures due to resource allocation. Stars used more sophisticated methods to alert them when system problems were developing, and a larger proportion of them (80% vs. 40%) applied root cause analysis to problems that did arise.

The solid infrastructure Stars provided resulted in meeting user availability and performance requirements at a very high level. The proportion of Stars that had acceptable performance ratings was 90.4%, compared to 49.0% for Low Performers. The proportion of Stars that had acceptable availability ratings was 92.3% vs. 62.7%.

Customer Care

Not only did Stars provide high availability and performance, they also provided a better "customer care" approach to SLM relative to the Low Performers. However, that level of care was not exceptionally high nor, for the most part, differed dramatically from that of the Low Performers. The Low Performers distinguished themselves from Stars through very low levels of communications with their customer-users. This is clearly seen in low quality reporting to their users and the lack of formal measurement of user satisfaction.

Relating Availability and Performance to Satisfaction

Meeting availability and performance expectations appears to be prerequisite for high levels of user satisfaction. It is the "basic" requirement (i.e. need) for user satisfaction to be achieved.
For all Communications respondents combined, 62.5% of the sites with acceptable availability had satisfied or better satisfaction ratings, compared to 31.6% of those with unacceptable availability.

For all Communications respondents combined, 63.4% of the sites with acceptable performance had satisfied or better satisfaction ratings, compared to 39.2% of those with unacceptable availability.

User Satisfaction Rating

One implication of the above argument is that Stars should have higher satisfaction ratings than Low Performers.

A significantly greater proportion of Stars (78.5%%) than Low Performers (33.4%) had satisfied or better ratings. Furthermore, none of the Low Performers received user satisfaction ratings of extremely satisfied, compared to 15.7% of the Stars.

Overall Rating

In addition, on a 100-point Overall SLM scale, Stars were rated significantly higher, on the average, than Low Performers (75.9 compared to 55.6). This was to be expected, since the Overall SLM scale is a summation of the individual SLM measures.

Stars vs. Low Performers - Firmographic Characteristics

Sites were almost even divided between Stars with 50.5%, and Low Performers, with 49.5% of the Communications industry sites. They did, however, differ on other factors.

Geographic Region

The proportion of North American and Latin American sites was similar in both the SLM performance groups. For the Stars, 55.8% were North American or Latin American sites, vs. 57.8% of the Low Performers sites.

The groups differed in the proportion of sites from Europe, the Middle East or Africa (EMEA) and the Asia Pacific regions. EMEA accounted for 23.1% of the Stars and 34.7% of the Low Performers sites. The Asia Pacific region accounted for 21.2% of the Stars compared to 8.2% of the Low Performers sites.

IT Service or Application Group

Respondents selected the set of services or group of applications to profile in the assessment. Stars showed a slightly lower propensity to profile e-business oriented services (34.6%) compared to the Low Performers (38.0%). A slightly larger proportion of the Stars (11.5%) profiled Enterprise Resource Planning (ERP) applications than Low Performers (8.0%). Likewise, a larger proportion of the Stars (26.9% vs. 24.0%) profiled custom applications.

Number of Users

The Stars described larger user communities than Low Performers. Sites with at least 10,000 users comprised 51.9% of the Stars compared to 40.2% of the Low Performers. Most of this difference was attributable to sites with at least 15,000 users. For the Stars, 19.2% of sites had at least 15,000 users, compared to 7.8% for the Low Performers.

Downtime Cost

IT downtime cost per hour is an indicator of an organization's financial exposure. The Stars tended to report greater financial exposure to IT downtime. Forty-eight percent of the Stars reported downtime cost of at least $200,000, compared to 34.7% of the Low Performers sites. Fourteen percent of the Stars reported downtime cost of at least $400,000, compared to 8.6% of the Low Performers.

Predictive Service Assurance (PSA)

Predictive Service Assurance (PSA) enables proactive prediction of future system behavior. PSA translates systems-monitoring data into knowledge that helps optimize both systems behavior and service levels by constructing a sophisticated application and resource system model. The model can aid in capacity planning, consolidation planning and application lifecycle development support.

Four new measures have been added to the on-line Service Level Management Assessment to assess Predictive Service Assurance (PSA) practices in Information Technology organizations. They measure the approaches taken to:

  • Information Technology Planning
  • Workload Balancing
  • Capacity Planning
  • Capacity Planning Effectiveness Rating

This is the first analysis of assessments that includes the PSA measures.

Predictive Service Assurance and the Communications Industry

For this analysis, data from all Communications industry sites, Stars and Low Performers, were combined and then compared to the data from other industries.

The Communications industry differed from the other industries on the four PSA measures. The Communications industry tended to show greater sophistication, relative to the others. It is possible that this greater PSA sophistication was driven by the need of Communications companies to adapt to the waves of restructuring experienced by the industry in recent years.

Information Technology (IT) Planning

Automated tools were used by 14.5% of the Communication sites, compared to 9.5% of other industries, to perform IT "business" planning in response to mergers/acquisitions, etc. Most sites (57.1% of Communications and 56.6% of other industries) employed manual IT planning processes.

Workload Balancing

Automated configuration tools were used by 47.4% of the Communications sites, compared to 39.4% of other industries, to perform workload balancing. Virtually all of the difference was attributable to the greater use of exact automated configuration tools by the Communications sites.

Workload balancing was done through trial and error by 27.6% of the Communications sites, compared to 34.8% of other industries. One fourth of all sites (25.0% of Communications and 25.8% of other industries) did not do workload planning.

Capacity Planning

Automated capacity planning tools were used to some degree by 45.5% of the Communications sites, compared to 42.1% of other industries. Completely automated capacity planning tools were used by 10.4% of the Communications sites, compared to 4.7% of other industries.

Totally manual capacity planning was employed by 37.6% of the Communications sites and 35.1% of other industries sites. One fifth of all sites (19.5% of Communications and 20.3% of others) had no formal process for capacity planning.

Capacity Planning Effectiveness

A site's capacity planning effectiveness rating is directly related to the sophistication of its capacity planning approach. Eighty-six percent of all sites (Communications and other businesses combined) with completely automated capacity planning processes rated their capacity planning as mostly or completely effective. Only 43.5% of sites with partially automated capacity planning processes rated their capacity planning as mostly or completely effective, and 17.1% of sites with completely manual capacity planning processes rated their capacity planning as mostly or completely effective. This pattern was observed for both the Communications industry and other industries when analyzed individually.

Predictive Service Assurance and the SLM Groups within Communications

The Stars showed significantly greater sophistication in the area of predictive service assurance (PSA), relative to the Low Performers.

Approach to IT Planning

Stars (22.5%) were four times as likely to use automated tools for IT planning as Low Performers (5.4%). Furthermore, 80.0% of the Stars did some form of IT planning, compared to 62.2% of the Low Performers sites.

Approach to Workload Balancing

A significantly greater proportion of Stars (69.2%) used automated tools for workload balancing than Low Performers (24.3%). Furthermore, 84.6% of the Stars did some form of workload balancing, compared to 64.9% of the Low Performers. Trial and error was the workload balancing approach reported most frequently for the Low Performers (40.5%).

Approach to Capacity Planning

A significantly greater proportion of Stars (64.1% vs. 26.3%) used automated tools for capacity planning. Furthermore, 92.3% of the Stars did some form of capacity planning, compared to 64.9% of the Low Performers sites. Partially automated capacity planning processes were reported most frequently for the Stars (42.1%). The capacity planning approach reported most frequently for the Low Performers (42.1%) was manual processes.

Capacity Planning Effectiveness

A significantly greater proportion of Stars (50.0%%) rated their site's capacity planning as mostly or completely effective, than Low Performers (9.1%). Furthermore, none of the Low Performers rated their site's capacity planning as completely effective, compared to 20.6% of the Stars.

Respondents and Sites

For purpose of these analyses, the data was divided into two categories, "Communications" and "Other". Communications included all telecommunications businesses, including wireless. Other included all other businesses, educational and government organizations. The Communications category accounted for 7.3% of the respondents. Other businesses and organizations accounted for 92.7% of the respondents. The analyses in this section combine Stars and Low Performers into the single category - Communications.

Job Role of Respondents

Communications industry respondents had less representation than other industries from executive managers (14.7% vs. 21.3%) and from service level managers (6.9% vs. 12.3%). Communications industry respondents had more representation than other industries from technical support managers (12.7% vs. 8.1%) and consultants (20.6% vs. 12.3%).

Geographic Location

North America and Latin America represented a slightly smaller proportion of Communications sites (56.4%) then other business sites (58.3%). Europe, the Middle East and Africa (EMEA) also represented a slightly smaller proportion of Communications sites (28.7%) then other business sites (30.5%). Asia Pacific represented a slightly larger proportion of Communications sites (14.9%) then other business sites (11.5%).

Computing Environments

Slight differences were found between the computing environments of Communications sites and those of other businesses. Communications had a larger proportion of sites (54.5%) that relied on distributed systems only, compared to 51.4% for other businesses. Three percent of Communications sites reported mainframes only, compared to 5.0% of the other businesses.

Services and Applications

Respondents selected the set of services or group of applications to profile in the assessment. Communications respondents showed a slightly higher propensity to profile e-business oriented services (36.3%) compared to respondents from other industries (30.4%). A slightly smaller proportion of the Communications respondents (9.8%) profiled Enterprise Resource Planning (ERP) applications vs. other businesses (13.8%). A smaller proportion of Communications respondents (25.5%) profiled custom applications, than other businesses (34.7%).

Number of Users

In general, Communications industry respondents described larger user communities when compared to other businesses.

The proportion of Communications sites with at least 10,000 users was 45.6%, compared to 30.9% for the other industries. The proportion of Communications sites with at least 5,000 users was 71.8%, compared to 57.4% for the other businesses.

Down Time Cost

The cost per hour of downtime was higher for Communications. Forty-two percent of Communications industry respondents, compared to 29.8% of other respondents, described their profiled service/application as having downtime cost in excess of $100,000. In addition, almost twice the proportion of Communications respondents described their profiled service/application as having down time cost greater than $400,000 per hour, 11.5% compared to 5.8% for other businesses.

Extrapolated average downtime cost in the Communications industry was $140,000/hour compared to $106,000/hour in other industries.

User Satisfaction

A smaller proportion of Communications sites had satisfaction ratings of satisfied or better (56.6% vs. 68.8%). The average satisfaction ratings were very similar: 2.7 for Communications and 2.9 for other industries.

Overall Rating

In addition, on a 100-point Overall SLM scale, Communications sites were rated slightly lower, on the average, than other industries (65.9 compared to 67.9).

Methodology

This report is based on data collected from assessments taken in 4Q2000 through 2Q2001. A total of 3,049 assessments were collected. Of these, 1,415 yielded data of sufficient quality to be used for the analyses presented in this paper. Communications industry respondents provided 103 useable assessments. The remaining 1,312 useable assessments were from respondents representing other industries.

The survey and assessment questions focus on a service chosen by the respondent. The validity of inferences to sites, industries, geographies, etc. is dependent on the assumption that, collectively, a representative sample of services was selected by the respondents.
Filigree Consulting, Inc. compiled this report specifically for BMC Software Corporation.

Other papers in this series

The following papers are also available:

Best Practices in Service Level Management - Baseline Research Report (October, 2000)

Service Level Management Profiles - Finance, Banking and Insurance (May, 2001)

See how you rate at www.nextslm.org/benchmark.

Sponsor

Report funded by BMC Software Corporation.

home


sponsored by
BMC Software
Remedy



White Paper: Implementing Service Level Management: Practical Guidelines for the IT Support Organization
by Char LaBounty


White Paper: Implementing ITIL with Remedy (pdf, 198kb)


Datasheet: Remedy's Comprehensive ITIL Solution (pdf, 624kb)

 

 
Copyright (c) 2000-2003, nextslm.org. All Rights Reserved. Legal Statement.