A major trend in the IT industry is Operations and Server Consolidation, which is being driven primarily by the need to reduce the Total Cost of Ownership of IT resources, as well as to provide increasing value for enterprise and business unit applications processing.

Customers with significant server investments are becoming increasingly aware that IT Operations Consolidation can translate into large cost reductions. A well-designed consolidation framework allows IT to use its data centers and IT business processes to evolve into a provider of a Compute Utility, which can offer value to business units and end-user customers. The data centers are the right place for this to occur because they can provide lower costs and better services through economies of scale as well as the critical mass for capacity planning, security, and architecture compliance.

The advantages of Operations and Server Consolidation include benefiting from improved economies of scale, the ability to easily scale-up capacity in a modular fashion, reductions in administration and support costs, the ability to flexibly reconfigure resources, improvements in resource utilization, and maintaining performance and availability levels for existing applications when deploying new applications.

This presentation will describe HP's consolidation architecture, as well as describe the different types and aspects of Consolidation. It will also discuss the benefits of Consolidation and how to realize them.

What is Server Consolidation? Whether stated as "Host Recentralization", "Operational Consolidation", "Server Consolidation", or simply "Consolidation" -- these terms are getting a lot of use in many large IT organizations, and may be interpreted to mean different things. Some use Server Consolidation to refer to the process of migrating a given workload from a large number of low to midrange servers to a small number of high-end servers. More broadly defined, the term also refers to the process of streamlining and re-architecting a data center infrastructure and operational processes in order to achieve a more efficient and cost-effective data center. When we refer to Server Consolidation, we have this broader definition in mind. A more accurate phrase that better captures this definition is Data Center Optimization, of which the consolidation of servers is merely one of many possible tactical approaches.

As shown in the figure titled "What is Consolidation", we use the term to refer to the process of Data Center Optimization. What is shown in this figure is a three-dimensional space. The horizontal dimension represents the server inventory that a data center currently maintains. The vertical dimension represents the application inventory that a data center currently maintains and supports for its end users. The third dimension represents a Total Cost of Ownership (TCO) dimension, where moving forward along this dimension means achieving a reduction in TCO. Different points in this three-dimensional space represent different numbers of servers, application instances, and TCO. At a high level, the process of Data Center Optimization is to move forward in the TCO dimension while still maintaining service levels as measured by performance, availability, and data integrity -- that was provided in the non-Consolidated or unoptimized data center. Moving forward in the TCO dimension may imply a reduction in the number of servers to support a given workload, which is often characterized by some number of users running a particular collection of transactions or queries.

When we discuss Consolidation, we refer to three Consolidation types: Physical, Logical, and Rationalization [1]. Physical Consolidation involves the co-location of computing resources that were previously geographically distributed, to a single or smaller number of data centers. Logical Consolidation occurs when multiple servers are managed in common management framework from a centralized or reduced number of management console(s). Rationalization (also known as Combination Consolidation [1]) occurs when applications and services are combined onto a more cost-effective computing infrastructure, typically resulting in fewer servers. This latter form of Consolidation may involve some careful planning and challenges, particularly when combining multiple application instances on a single operating system instance -- but when this can be achieved it typically provides the highest payoff in terms of TCO reduction. Note that these three types of Consolidation are not necessarily mutually exclusive.

The benefits from Consolidation come from several factors, which may be categorized into both reductions in hardware costs (fewer server and mass storage units to support a given workload), and from reductions in management and support costs ("soft factors").

These "soft factors" (i.e. factors other than hardware costs) include reductions in administrative headcount, lower software licensing fees, lower support costs, and lower environment charges (power, floorspace costs, etc.). The largest cost savings often come from these factors.

As an example, consider the case study reported in a Gartner Group Report [2]. In this study, support costs were reduced by approximately 60%, whereas component cost reductions only amounted to approximately 8%.

There are many aspects of IT management that can be optimized as part of a Consolidation project. From experience, we have found that there are five Consolidation focus areas, one or more of which are typically the target of a Consolidation project. These are listed below:

1. System Management Consolidation -- refers to a common system management framework applied across a data center or across an enterprise, as opposed to managing individual systems independently.
2. Mass Storage Consolidation – refers to the sharing of mass storage devices across a large number of servers, as opposed to a storage device-per-server architecture.
3. Network Consolidation – refers to the sharing of a high-bandwidth interconnect to handle traffic from multiple networks, as opposed to having multiple independent networks.
4. Backup Consolidation – refers to the sharing of large-capacity tape backup libraries or silos, as opposed to having a backup device-per-server architecture.
5. System Consolidation – refers to the sharing of systems by multiple applications, as opposed to having a dedicated system-per-application architecture.

We will elaborate further on each in turn, as we next describe HP’s Consolidation Architecture.

HP’s Consolidation Architecture is based on the Enterprise Parallel Server (EPS) Architecture, a general-purpose computing platform that is particularly well-suited for a broad class of commercial and technical workloads. For diverse workload types such as Mission Critical OLTP, ERP, DSS/Data Warehousing, Technical Compute, or Internet / Web Servers, the EPS architecture provides the flexibility of mixing different node types (ranging from low-end D-class, mid-range K-class, to high-end V-class), with a high-bandwidth node-to-node interconnect as well as a flexible mass storage interconnect technology based on FiberChannel. This combination of enabling technologies is particularly well suited to addressing the needs of Consolidation, in that multiple applications can run on a single system within an EPS with guaranteed availability and resource allocation based on business objectives, while sharing resources across systems such as mass storage devices, backup devices, high-bandwidth networks, as well as a system management infrastructure that simplifies the management of the EPS nodes. The EPS Architecture is illustrated in the following diagram:

In this diagram, there are multiple diverse node types, including D-, K-, and V-class nodes, all interconnected by a high-bandwidth interconnect. This high-bandwidth interconnect is referred to as a Virtual Backplane, and is based on the HyperFabric node-to-node interconnect. The mass storage interconnect that connects the nodes to shared mass storage and backup devices is based on FiberChannel. All the nodes are managed by a common management station using ClusterView Plus. We elaborate on each of these features in turn.

One of the key enabling technologies for a Consolidated computing environment is a high-bandwidth interconnect for inter-node communications and running backups over a network. The following table compares the performance characteristics of HP’s HyperFabric, which forms the basis of the Virtual Backplane in an EPS configuration. Note that the bandwidth of the HyperFabric switch is substantially higher than that of other alternatives, which includes both standards-based as well as proprietary networking technologies. The HyperFabric interconnect enables parallel DSS, parallel OLTP, and multi-tier message-passing applications, such as large ERP applications, to have excellent performance by virtually eliminating networking as a source of potential bottlenecks. It also makes network backups practical, as it can sustain sufficient bandwidth to keep tape backup devices streaming.

The following table compares the performance characteristics of the FiberChannel Arbitrated Loop (FCAL) mass storage interconnect, and compares it to SCSI as well as other proprietary mass storage interconnect technologies. FibreChannel as a mass storage interconnect provides substantially higher I/O bandwidth and cabling distances than any of the other technologies, and also allows for a substantial increase in the number of systems and mass storage devices that can be connected, with the addition of FiberChannel hubs.

FiberChannel hubs provide high availability for FCAL connections involving multiple hosts and storage devices, and provides 10 ports per hub for connecting systems and storage devices. Hubs can also be cascaded for additional connectivity.

The figure above illustrates a large scale mass storage configuration connected to multiple nodes through FCAL hubs, with a large shared backup server interconnected to all the nodes through the Virtual Backplane.

Another key aspect of Consolidation is a simplified multi-system management framework. EPS provides this with ClusterView Plus, an enhancement of ClusterView. ClusterView Plus is an extension to IT/O, which allows an administrator to create Management Clusters, a new form of cluster definition (in addition to ServiceGuard Clusters for High Availability and LockManager Clusters for Oracle Parallel Server). A Management Cluster is simply a user-defined grouping of nodes that are logically related in some way. For example, a Management Cluster may be defined to correspond to all nodes in a data center that are running SAP, or another might correspond to all nodes in use by a given department, or another might correspond to all nodes managed by a given administrative team. A node may belong to more than one Management Cluster or to a Management Cluster and another type of cluster such as a ServiceGuard Cluster.

To summarize, with the EPS architecture, it is possible to create flexible solutions that allow for the efficient utilization of resources, to flexibly share resources across systems and applications, and to manage multiple systems from a single management station.

Now that we have introduced the five focus areas of Consolidation and presented the features of HP’s EPS Architecture, we illustrate how each of the focus areas of Consolidation can be achieved on EPS. We discuss each in turn.

1. System Management Consolidation – refers to a common system management framework applied across a data center or across an enterprise, as opposed to managing individual systems independently.

In an EPS cluster, ClusterView Plus provides the basis for a centralized management station which provides one management console for multiple systems. With ClusterView Plus, an administrator can create Management Clusters, which are a user-defined grouping of nodes. Once management clusters are defined, an administrator can monitor the status of those nodes, and can launch system management tools to apply monitoring or administrative tasks across a set of nodes. For example, it is possible to launch PerfView on a set of nodes in a Management Cluster, as illustrated by the figure below.



Other companion system management products that support the single point of management include OpenView and IT/O for integrated network, system, and service management; SD-OV (Software Distributor for OpenView) for software distribution; PerfView and MeasureWare for performance measurement and management; Ignite/UX for replicating system configurations; and MC/ServiceGuard for High Availability. All of these products play a key role in simplifying the management of multiple systems and applications.

2. Mass Storage Consolidation – refers to the sharing of mass storage devices across a large number of servers, as opposed to a storage device-per-server architecture.

For Mass Storage Consolidation, the key enabling technology in EPS is the FibreChannel Arbitrated Loop (FCAL) mass storage interconnect. With FibreChannel as the mass storage interconnect (Storage Area Network, or SAN), mass storage devices such as large disk arrays can serve multiple systems, not just one. Also, by having the large mass storage shared across multiple systems, the result is a free pool of storage for the entire data center, as opposed to administrators having to manage storage space on a per application or per system basis. Another advantage of FCAL over other interconnect technologies such as SCSI cables is that storage can be easily moved between systems without having to re-cable. Also, additional systems can be added to a shared mass storage configuration with less effort than in the storage-per-system approach.

3. Network Consolidation – refers to the sharing of a high-bandwidth interconnect to handle traffic from multiple networks, as opposed to having multiple independent networks.

The problem faced in many large data centers today is that of a large number of networks running throughout the data center. These multiple LAN installations may have grown up over time to address a variety of needs, such as a "campus LAN", intranet, "site backbone", client LANs, various legacy integration LANs, WANs, backup LANs, failover LANs, etc.). Often these multiple networks have low average utilization rates, and yet their sheer number requires I/O cards for each network interfaces, thereby consuming valueable I/O slots in servers. One way to address this problem is to route low-volume traffic from legacy networks onto a single high-bandwidth network, such as that provided by the Virtual Backplane in EPS. This allows for the legacy network to be connected to only a single node in the EPS, which routes the traffic on that network over the Virtual Backplane to the target node. This approach is enabled by the high-bandwidth of the Virtual Backplane, which is based on HyperFabric.

4. Backup Consolidation – refers to the sharing of large-capacity tape backup libraries or silos, as opposed to having a backup device-per-server architecture.

The goal of Backup Consolidation is to have one backup solution for multiple servers, as opposed to having separate backup solutions for individual servers. This implies not only a reduction in the number of tape backup hardware devices, but also a significant improvement in the efficiency in the operations of backup procedures and backup media handling. This is enabled by the Virtual Backplane, in that high-speed backups can be performed over the HyperFabric interconnect.

5. System Consolidation – refers to the sharing of systems by multiple applications, as opposed to having a dedicated system-per-application architecture.

System Consolidation involves a reduction in the number of systems to support a given workload, where a system is defined by a node running an operating system instance. In order to achieve a reduction in the number of systems to support a given workload, this implies running multiple applications per system. One immediate advantage gained from this approach is a reduction in the number of instances of HP-UX that need to be managed, as well as the number of systems. This leads to a reduction in both administrative and support costs as the number of systems decreases. This approach is enabled by the following software infrastructure products: MC/ServiceGuard for High Availability, Process Resource Manager for controlling the allocation of resources to applications based on business objectives, and Load Sharing Facility for load balancing applications across multiple systems.

The following figure illustrates a common situation in many IT organizations today, where there is a proliferation of servers, typically running one application per server, each server with its own dedicated mass storage and tape backup devices, and each with its own management console. This configuration of servers leads to an inefficient use of floorspace, power, support costs, licensing costs, etc.

In order to reap the benefits of Consolidation, data centers are moving to a more cost-effective architecture as shown in the next figure, where there is much better use of floorspace, featuring rack-mounted systems, shared mass storage and tape backup devices, and shared management stations. This is the architecture illustrated in the figure below.

In order to realize the benefits of Server Consolidation, it is first necessary to understand what are the immediate challenges facing your IT organization, and where are the best opportunities for Data Center Optimization. Some may choose to focus first on just one of the five Consolidation areas, and then gradually expand into the others. One critical success factor for successful Consolidation projects is that there be broad support for it throughout the IT organization, and particularly that there be support from high level management in the IT organization.

In order to choose a pilot Consolidation project, the best opportunities are those that would produce the greatest benefits for the least amount of effort or impact on the ongoing processes of the data center. For example, if all the applications are in production and no new application deployments are scheduled, this may dictate starting with Mass Storage Consolidation or Backup Consolidation. On the other hand, if new applications are about to be deployed, these may be candidates for rolling out on a new Consolidation architecture. Also, if applications are being modified or recertified for Year 2000 testing, this may provide an opportunity for migrating these applications from a one-application-per-server architecture to a multiple-applications-per-server architecture.

This section outlines the steps to follow for implementing a successful Consolidation project.

1. Understand the core technology

The core technology includes EPS, MC/ServiceGuard (for high availability), ClusterView Plus, PRM, FiberChannel Mass Storage, and HyperFabric.

2. Migrating the application to the new platform

Migrating applications to the new platform involves moving (or deploying) applications on an EPS platform, which may require some porting activity if the applications are not yet running on HP-UX 10.2 or later.

3. Sizing / assessment of multiple applications per server

Sizing applications involves measuring their resource requirements over critical time periods. This should be coupled with an assessment of the applications to determine which ones may be compatible for co-existing with which other applications on the same system. For example, it is generally safe to run multiple instances of the same application on one system.

4. Testing / deployment

Finally, it is important to test the applications in the new consolidated environment under real-world conditions, and only then to move them into production.

As a result of a Consolidation project, depending on the scope of the project, some increase in efficiency or reduction in costs is highly likely to result. For example, System Consolidation is likely to result in a 50%-75% reduction in floorspace usage. Mass Storage Consolidation is likely to result in a reduction in management and support costs that was associated with managing mass storage devices dedicated to individual servers. Similarly, Backup Consolidation will result in a dramatic improvement in operational efficiency. The actual cost savings will vary with each installation, but the trends are clear. As was stated earlier, the greatest opportunities for realizing cost savings comes from reducing management and support costs.

The term Server Consolidation is being widely used across many IT organizations to refer to a process of centralizing computing resources within one or more data centers in order to reduce the Total Cost of Ownership of computing systems, as well as to improve service and value as experienced by the end-user customers. The definition of Server Consolidation as described in this paper includes this concept, but also refers to the broader underlying goal of Data Center Optimization.

We have described three different types of Consolidation, and identified five key focus areas where Consolidation principles may be applied. We examined HP’s Consolidation Architecture, based on the Enterprise Parallel Server, and showed how it can be effective in achieving the goals of Consolidation. We sketched out the Consolidation implementation process. Finally, we discussed the potential benefits of a Consolidation project.

[1] Shipley, T. Data Center Server Consolidation, Key Issues K-802-077, Data Center Strategies (DCS) Research Note, Gartner Group, March 29, 1996.

[2] Cappucio, D. Server Consolidation: A Cost Evaluation Model, Key Issues K-LAN-1396, Local Area Communications (LAC), Gartner Group, February 15, 1995.

Send email to Interex or to theWebmaster
©Copyright 1998 Interex. All rights reserved.