Jump to content United States-English
HP.com Home Products and Services Support and Drivers Solutions How to Buy
» Contact HP
More options
HP.com home
Managing Serviceguard Fifteenth Edition > Chapter 4 Planning and Documenting an HA Cluster

Cluster Configuration Planning

» 

Technical documentation

Complete book in PDF
» Feedback
Content starts here

 » Table of Contents

 » Index

A cluster should be designed to provide the quickest possible recovery from failures. The actual time required to recover from a failure depends on several factors:

  • The length of the cluster heartbeat interval and node timeout.

    See the parameter descriptions for HEARTBEAT_INTERVAL and NODE_TIMEOUT under “Cluster Configuration Parameters ” for recommendations.

  • The design of the run and halt instructions in package scripts. They should be written for fast execution.

  • The availability of raw disk access. Applications that use raw disk access should be designed with crash recovery services.

  • The application and database recovery time. They should be designed for the shortest recovery time.

In addition, you must provide consistency across the cluster so that:

  • User names are the same on all nodes.

  • UIDs are the same on all nodes.

  • GIDs are the same on all nodes.

  • Applications in the system area are the same on all nodes.

  • System time is consistent across the cluster.

  • Files that could be used by more than one node, such as /usr files, must be the same on all nodes.

The Serviceguard Extension for Faster Failover is a purchased product that can optimize failover time for certain two-node clusters. The clusters must be configured to meet certain requirements. When installed, the product is enabled by a parameter in the cluster configuration file. Release Notes for the product are posted at http://docs.hp.com -> high availability.

Heartbeat Subnet and Re-formation Time

The speed of cluster re-formation is partially dependent on the type of heartbeat network that is used. If two or more heartbeat subnets are used, the one with the fastest failover time is used.

NOTE: For heartbeat requirements, see the discussion of the HEARTBEAT_IP parameter later in this chapter.

Cluster Configuration Parameters

You need to define a set of cluster parameters. These are stored in the binary cluster configuration file, which is distributed to each node in the cluster. You configure these parameters by editing the cluster configuration template file created by means of the cmquerycl command, as described under “Configuring the Cluster ”.

The following parameters must be configured:

CLUSTER_NAME

The name of the cluster as it will appear in the output of cmviewcl and other commands, and as it appears in the cluster configuration file.

The cluster name must not contain any of the following characters: space, slash (/), backslash (\), and asterisk (*).

NOTE: In addition, the following characters must not be used in the cluster name if you are using the Quorum Server: at-sign (@), equal-sign (=), or-sign (|), semicolon (;).

These characters are deprecated, meaning that you should not use them, even if you are not using the Quorum Server, because they will be illegal in a future Serviceguard release.

All other characters are legal. The cluster name can contain up to 39 characters (bytes).

CAUTION: Make sure that the cluster name is unique within the subnets configured on the cluster nodes; under some circumstances Serviceguard may not be able to detect a duplicate name and unexpected problems may result.

In particular make sure that two clusters with the same name do not use the same quorum server; this could result in one of the clusters failing to obtain the quorum server’s arbitration services when it needs them, and thus failing to re-form.

QS_HOST

The name or IP address of a host system outside the current cluster that is providing quorum server functionality. This parameter is only used when you employ a quorum server for tie-breaking services in the cluster.

QS_POLLING_INTERVAL

The time (in microseconds) between attempts to contact the quorum server to make sure it is running. Default is 300,000,000 microseconds (5 minutes).

QS_TIMEOUT_EXTENSION

The quorum server timeout is the time during which the quorum server is not communicating with the cluster. After this time, the cluster will mark the quorum server DOWN. This time is calculated based on Serviceguard parameters, but you can increase it by adding an additional number of microseconds as an extension.

The QS_TIMEOUT_EXTENSION is an optional parameter.

FIRST_CLUSTER_LOCK_VG, SECOND_CLUSTER_LOCK_VG

The volume group containing the physical disk volume on which a cluster lock is written. This parameter is used only when you employ a lock disk for tie-breaking services in the cluster. If you are creating two cluster locks, enter the volume group name or names for both locks.

Use FIRST_CLUSTER_LOCK_VG for the first lock volume group. If there is a second lock volume group, specify the SECOND_CLUSTER_LOCK_VG on a separate line in the configuration file.

NOTE: Lock volume groups must also be defined in VOLUME_GROUP parameters in the cluster configuration file.
SITE_NAME

The name of a site to which nodes (see NODE_NAME) belong. Can be used only in a site-aware extended-distance cluster, which requires additional software; see the documents listed under “Cross-Subnet Configurations” for more information.

You can define multiple SITE_NAMEs. SITE_NAME entries must precede any NODE_NAME entries See also SITE.

NODE_NAME

The hostname of each system that will be a node in the cluster.

CAUTION: Make sure that the node name is unique within the subnets configured on the cluster nodes; under some circumstances Serviceguard may not be able to detect a duplicate name and unexpected problems may result.

Do not use the full domain name. For example, enter ftsys9, not ftsys9.cup.hp.com. A Serviceguard cluster can contain up to 16 nodes (though not in all third-party configurations; see “Veritas Cluster Volume Manager (CVM)”, and the latest Release Notes for your version of Serviceguard).

IMPORTANT: Node names must be 39 characters (bytes) or less, and are case-sensitive; for each node, the NODE_NAME in the cluster configuration file must exactly match the corresponding node_name in the package configuration file (see Chapter 6 “Configuring Packages and Their Services”) and these in turn must exactly match the hostname portion of the name specified in the node’s networking configuration. (Using the above example, ftsys9 must appear in exactly that form in the cluster configuration and package configuration files, and as ftsys9.cup.hp.com in the DNS database).
SITE

The name of a site (defined by SITE_NAME) to which the node identified by the preceding NODE_NAME entry belongs. Can be used only in a site-aware extended-distance cluster, which requires additional software; see the documents listed under “Cross-Subnet Configurations” for more information.

If SITE is used, it must be used for all nodes in the cluster (that is, all the nodes must be associated with some defined site, though not necessarily the same one).

NETWORK_INTERFACE

The name of each LAN that will be used for heartbeats or for user data. An example is lan0.

For information about changing the configuration online, see “Changing the Cluster Networking Configuration while the Cluster Is Running”.

HEARTBEAT_IP

IP notation indicating a subnet that will carry the cluster heartbeat.

A heartbeat IP address must be an IPv4 address.

Any subnet that is configured as a monitored_subnet in a package configuration file (or SUBNET in a legacy package; see “Package Configuration Planning ”) must be specified as either a STATIONARY_IP or HEARTBEAT_IP. For information about changing the configuration online, see “Changing the Cluster Networking Configuration while the Cluster Is Running”.

Heartbeat configuration requirements:

A minimum Serviceguard configuration on HP-UX 11i v2 or 11i v3 needs two network interface cards for the heartbeat in all cases, using one of the following configurations:

  • Two heartbeat subnets; or

  • One heartbeat subnet with a standby; or

  • One heartbeat subnet using APA with two physical ports in hot standby mode or LAN monitor mode.

Considerations for cross-subnet:

IP addresses for a given heartbeat path are usually on the same subnet on each node, but it is possible to configure the heartbeat on multiple subnets such that the heartbeat is carried on one subnet for one set of nodes and another subnet for others, with the subnets joined by a router.

This is called a cross-subnet configuration, and in this case at least two heartbeat paths must be configured for each cluster node, and each heartbeat subnet on each node must be physically routed separately to the heartbeat subnet on another node (that is, each heartbeat path must be physically separate).See “Cross-Subnet Configurations”.

NOTE: Because Veritas Cluster File System from Symantec (CFS) requires link-level traffic communication (LLT) among the nodes, Serviceguard cannot be configured in cross-subnet configurations with CFS alone. But CFS is supported in specific cross-subnet configurations with Serviceguard and HP add-on products such as Serviceguard Extension for Oracle RAC (SGeRAC); see the documentation listed under “Cross-Subnet Configurations” for more information.

Considerations for CVM:

  • If you will be using Veritas CVM 4.1 or later, multiple heartbeats are permitted, and you must configure either multiple heartbeat subnets or a single heartbeat subnet with a standby. HP recommends multiple heartbeats.

  • If you will be using CVM 3.5, however, you can only use a single heartbeat subnet; in this case, configure the heartbeat either with a standby LAN or as a group of aggregated ports on each node. If you use aggregated ports (APA), HP recommends using Hot Standby mode to eliminate the single point of failure (SPOF) that a single switch represents. See Chapter 3 of the HP Auto Port Aggregation Administrator’s Guide at http://docs.hp.com -> I/O Cards and Networking Software -> Auto Port Aggregation (APA) for more information. A standby LAN always uses two switches and thus does not entail a SPOF.

  • You cannot change the heartbeat configuration while a cluster that uses CVM is running.

NOTE: The use of a private heartbeat network is not advisable if you plan to use Remote Procedure Call (RPC) protocols and services. RPC assumes that each network adapter device or I/O card is connected to a route-able network. An isolated or private heartbeat LAN is not route-able, and could cause an RPC request-reply, directed to that LAN, to risk time-out without being serviced.

NFS, NIS and NIS+, and CDE are examples of RPC based applications that are frequently used on HP-UX. Other third party and home-grown applications may also use RPC services directly through the RPC API libraries. If necessary, consult with the application vendor to confirm its usage of RPC.

STATIONARY_IP

The IP address of each subnet that does not carry the cluster heartbeat, but is monitored for packages.

Any subnet that is configured as a monitored_subnet in a package configuration file (or SUBNET in a legacy package; see “Package Configuration Planning ”) must be specified as either a STATIONARY_IP or HEARTBEAT_IP.

If you want to separate application data from heartbeat messages, define one or more monitored non-heartbeat subnets here. You can identify any number of subnets to be monitored.

IMPORTANT: In a cross-subnet configuration, each package subnet configured on an interface (NIC) must have a standby interface connected to the local bridged network; see “Cross-Subnet Configurations”.

A stationary IP address can be either an IPv4 or an IPv6 address. For more details of IPv6 address format, see “IPv6 Address Types”.

For information about changing the configuration online, see “Changing the Cluster Networking Configuration while the Cluster Is Running”.

CLUSTER_LOCK_LUN

The pathname on this node for the LUN used for the cluster lock. Used only if a lock LUN is used for tie-breaking services.

Enter the path name as it appears on each node in the cluster (the same physical device may have a different name on each node).

You cannot create a dual cluster-lock configuration using LUNs.

FIRST_CLUSTER_LOCK_PV, SECOND_CLUSTER_LOCK_PV

The name of the physical volume within the Lock Volume Group that will have the cluster lock written on it. Used on only if a lock disk is used for tie-breaking services. This parameter is FIRST_CLUSTER_LOCK_PV for the first physical lock volume and SECOND_CLUSTER_LOCK_PV for the second physical lock volume. If there is a second physical lock volume, the parameter SECOND_CLUSTER_LOCK_PV is included in the file on a separate line. These parameters are only used when you employ a lock disk for tie-breaking services in the cluster.

Enter the physical volume name as it appears on each node in the cluster (the same physical volume may have a different name on each node). If you are creating two cluster locks, enter the physical volume names for both locks. The physical volume group identifier can contain up to 39 characters (bytes).

For information about changing the configuration while the cluster is running, see “Updating the Cluster Lock Disk Configuration Online”.

HEARTBEAT_INTERVAL

The normal interval, in microseconds, between the transmission of heartbeat messages from each node to the cluster coordinator.

Default value is 1,000,000 microseconds; setting the parameter to a value less than the default is not recommended.

The default should be used where possible. The maximum recommended value is 15 seconds and the maximum value supported is 30 seconds or half the NODE_TIMEOUT.

Can be changed while the cluster is running.

NODE_TIMEOUT

The time, in microseconds, after which a node may decide that another node has become unavailable and initiate cluster reformation.

Maximum value: 60,000,000 microseconds (60 seconds).

Minimum value: 2 * HEARTBEAT_INTERVAL

Default value: 2,000,000 microseconds (2 seconds).

Recommendations: You need to decide whether it's more important for your installation to have fewer cluster reformations, or faster reformations:

  • To ensure the fastest cluster reformations, use the default value. But keep in mind that this setting can lead to reformations that are caused by short-lived system hangs or network load spikes.

  • For fewer reformations, use a setting in the range of 5,000,000 to 8,000,000 microseconds (5 to 8 seconds). But keep in mind that this will lead to slower reformations than the default value.

  • The maximum recommended value is 30,000,000 microseconds (30 seconds).

Remember that a cluster reformation may result in a system reset on one of the cluster nodes. For further discussion, see“What Happens when a Node Times Out”.

There are more complex cases that require you to make a trade-off between fewer failovers and faster failovers. For example, a network event such as a broadcast storm may cause kernel interrupts to be turned off on some or all nodes while the packets are being processed, preventing the nodes from sending and processing heartbeat messages. This in turn could prevent the kernel’s safety timer from being reset, causing a system reset. (See “Cluster Daemon: cmcld” for more information about the safety timer.)

Can be changed while the cluster is running.

AUTO_START_TIMEOUT

The amount of time a node waits before it stops trying to join a cluster during automatic cluster startup. All nodes wait this amount of time for other nodes to begin startup before the cluster completes the operation. The time should be selected based on the slowest boot time in the cluster. Enter a value equal to the boot time of the slowest booting node minus the boot time of the fastest booting node plus 600 seconds (ten minutes).

Default is 600,000,000 microseconds.

Can be changed while the cluster is running.

NETWORK_POLLING_INTERVAL

The frequency at which the networks configured for Serviceguard are checked. In the cluster configuration file, this parameter is NETWORK_POLLING_INTERVAL.

Default is 2,000,000 microseconds in the configuration file (2 seconds). Thus every 2 seconds, the network manager polls each network interface to make sure it can still send and receive information. Using the default is highly recommended. Changing this value can affect how quickly a network failure is detected. The minimum value is 1,000,000 (1 second). The maximum value recommended is 15 seconds, and the maximum value supported is 30 seconds.

Can be changed while the cluster is running.

MAX_CONFIGURED_PACKAGES

This parameter sets the maximum number of packages that can be configured in the cluster.

The minimum value is 0, and the maximum value is 150. The default value for Serviceguard is 150, and you can change it without halting the cluster.

VOLUME_GROUP

The name of an LVM volume group whose disks are attached to at least two nodes in the cluster. Such disks are considered cluster-aware. The volume group name can have up to 39 characters (bytes).

Access Control Policies (also known as Role Based Access)

For each policy, specify USER_NAME, USER_HOST, and USER_ROLE. Policies set in the configuration file of a cluster and its packages must not be conflicting or redundant. For more information, see “Setting up Access-Control Policies”.

FAILOVER_OPTIMIZATION

You will only see this parameter if you have installed Serviceguard Extension for Faster Failover, a separately purchased product. You enable the product by setting this parameter to TWO_NODE. Default is disabled, set to NONE. For more information about the product and its cluster-configuration requirements, go to http://www.docs.hp.com -> High Availability and choose Serviceguard Extension for Faster Failover.

NETWORK_FAILURE_DETECTION

The configuration file specifies one of two ways to decide when a network interface card has failed:

  • INOUT

  • INONLY_OR_INOUT

The default is INOUT.

See “Monitoring LAN Interfaces and Detecting Failure ” for more information.

Can be changed while the cluster is running.

Cluster Configuration: Next Step

When you are ready to configure the cluster, proceed to “Configuring the Cluster ”. If you find it useful to record your configuration ahead of time, use the worksheet in Appendix F “Blank Planning Worksheets”.

Printable version
Privacy statement Using this site means you accept its terms Feedback to webmaster
© Hewlett-Packard Development Company, L.P.