Service Level Management- Global Survey 2001

Introduction

• This white paper will extend the concepts presented in our late 2000 paper: "Service Level Management- North America Survey 2000" to include an international perspective. It is intended as an update to that paper which can be found at www.nextslm.org. While we will attempt to make this paper stand-alone, the reader will benefit from reading the series in order.

• By early January 2001, 509 middle and executive management respondents from large organizations had taken the on-line Service Level Management assessment. The assessment tool can be found at http://www.nextslm.org/benchmark. Based on their input, in combination with the initial survey, this paper includes a comparative analysis of the key differences in Service Level Management practices in three major geographic regions. The three regions compared in this paper are North America/Latin America (NA/LA), Europe/Africa/Middle East (EAME) and Asia/Pacific (AP).

• In this analysis North America/Latin America is represented by close to 300 observations, Europe/Africa/Middle East has close to 100 observations and Asia Pacific has less than 100 observations. Conclusions for subsets of the Asia Pacific data are at times based on statistically low numbers of observations, and should be used with caution.

• The assessment tests the impact of 22 factors on user satisfaction. The factors included key measurements of availability and performance, recovery, Service Level Management features (agreements, reporting, measurement, monitoring, and documentation), resource use, and trends in help desk use and software costs. A Service Level Management Rating (scale: 0 to 100) is calculated based on the relationships between these factors and user satisfaction. In the next release of the assessment tool we will expand this to 26 key factors.
  

2001 Update - Highlights

• There are two significant changes in the composition of the assessments from our original analysis - a shift to e-business applications (away from ERP and custom-in-house applications) and a redistribution of industries represented to more e-industries.

• The top factors contributing to user satisfaction have also shifted. In our initial analysis, the top factors contributing to user satisfaction were: meeting availability requirements, having an improving or stable availability trend, meeting performance requirements, and having short recovery times from unplanned outages. The news is that performance has taken a higher priority than availability with two performance factors (status and trend) appearing in the top four factors - and a shift in order within the top five. Availability was the top factor in the prior analysis - it's now fifth. Performance status (1 st ) and performance trend (4 th ) have been emphasized. This is strongly related to the shift in application type.

• In aggregate, the Service Level Management ratings appear to differ little between regions. All respondents scored an average of 70.4 - led by AP (71.1), followed by NA/LA (70.9) and EAME (67.8).

• Overall user satisfaction is lower for services profiled by EAME. AP and NA/LA follow the overall averages closely. User satisfaction scores (on a seven scale, with seven representing "extremely satisfied") show all services scoring 5.0 - led by AP (5.1), followed by NA/LA (5.0) and EAME (4.6).

• Key differences between the regions include their performance in the areas of Service Level Agreement contents (and the percentage having agreements), Service Level Reporting Methods, Satisfaction Measurement Programs, Availability Status, and Performance Status.

• In our initial study sixty-one (61%) percent of the respondents had service level agreements in place for the mission critical service they were profiling. In the new data 68% of the services profiled had agreements in place - led by AP (82%), EAME (76%) and NA/LA with 65%. On an industry basis, the differences are more dramatic with ASP's having agreements in place for 89% of their profiled services, contrasted with Finance (71%), Distribution (53%), Manufacturing (49%), Utilities (47%), and Education (42%).

• Most of those agreements had goals, objectives, roles and responsibilities defined, and they generally included specifications of support availability. Penalties for non-performance or missed objectives (34%) and incentives for exceptional performance (13%) were in place in significantly fewer agreements. EAME agreements included a higher percentage (53%) of adjustment procedures compared to the overall responses (43%).

• The data continues to reinforce the concept that Service Level Management is implemented in an evolutionary fashion. Three clear groups of respondents exhibit low, medium or high performance in SLM. A key difference between the groups is the performance of the service in meeting basic needs - both the medium and high performing groups accomplish this. High performance appears to be contingent both on meeting basic needs and extending service level management to customer care.

Attaining Excellence In our initial study, we indicated that there were three distinct groups of performers - Stars, Medium Performers and Low Performers. This cluster analysis continues to be supported by results from the assessment tool. While the distinction based on industry has lost some of its clarity, the two basic factors of "underlying infrastructure" and "customer care", remain strong.

(1) Underlying Infrastructure A solid infrastructure is critical to any underlying SLM effort. If a service is meeting availability and performance requirements then users are generally satisfied. Meeting availability and performance expectations appear to be conditions that must exist for high levels of user satisfaction. They are the "basic" requirements (needs) for user satisfaction to be achieved. The change driven by the international content is an increased emphasis on performance.

(2) Customer Care What continues to distinguished the "stars" is that they do many things right. Not only do they provide high availability and performance, but also they provide a "customer care" approach to SLM in the form of robust service level agreements, high quality reporting to their users and formal measurement of user satisfaction. Customer focus is what separates the solid SLM performers from the stars. Stars distinguished themselves through high quality communications with their customer-users. The table has been updated to reflect minor changes in average rating and membership percentages.

graphic: Filligree Consulting

Membership in the three groups is approximately the same between the regions, with the most significant differences in EAME. EAME respondents are more likely to be in the lower performing groups with twice as many in the lowest group and fewer "Stars" compared to the total population.

graphic: Filligree Consulting

In this multiple response question participants are asked to identify the contents of their service level agreements.

The results are not radically different between geographies with the exception of more emphasis on "adjustment procedures" in EAME. AP also indicates a stronger emphasis on "goals and objectives."

graphic: Filligree Consulting

In this multiple response question participants are asked to describe their reporting procedures.

Customized reporting has a lower incidence in EAME.

AP, while equal to the overall in customized reporting also tends to provide "data on components" more frequently than other geographies.

Services profiled by both EAME and AP respondents include reporting more frequently than NA/LA.

graphic: Filligree Consulting

In this multiple response question participants are asked to describe the high availability infrastructure that supports the profiled service.

While not statistically significant at high confidence levels - EAME has a higher incidence of high availability server configurations and remote mirrored disk. Both EAME and AP show lower frequencies for locally mirrored disk.

graphic: Filligree Consulting

Participants are asked to describe the availability and performance underlying their profiled service.

Significant differences exist between services profiled by EAME and AP participants and NA/LA participants.

Generally both EAME and AP service meet user requirements about 10% less frequently than NA/LA services.

graphic: Filligree Consulting

Participants are asked if they have a formal end-user satisfaction measurement program.

Services profiles by EAME participants have a significantly lower number of formal satisfaction measurement programs.

graphic: Filligree Consulting

Overall user satisfaction is significantly lower for EAME with almost twice the incidence of dissatisfied users. Users are not satisfied in over one third of the EAME services assessed.

AP and NA/LA tend to parallel the total response, with the exception of more neutral responses from AP.

graphic: Filligree Consulting

The 600+ assessments of services collected to date show a broad industry distribution with a strong emphasis on ASP participation.

Attracting ASP responses has been a key (and successful) objective of the netslm.org team.

graphic: Filligree Consulting

The distribution of application type for the service profiled to date is similar for NA/LA and AP.

EAME has a much higher percentage of e-business applications profiled, which is consistent with EAME's ISP and ASP content.

Methodology

This analysis is based on data collected during our initial research survey, combined with assessments taken over the last several months at the nextslm.org site. Assessments from the site were screened for completeness and for outlying results. From the 509 assessments completed on-line, a total of 267 were used in the statistical analysis. Comparisons between the data from the sources (research and selected assessments) indicate that little bias was created by combining the two data collection methods. This report was developed by Filligree Consulting and commissioned by nextslm.org sponsors.

Sponsors

This work was sponsored by BMC Software, Sun Microsystems, and PricewaterhouseCoopers.

home