Planning for Recovery

Flexibility in configuration, one of the major benefits of LVM, can also be a source of problems in recovery. The following are guidelines to help create a configuration that minimizes recovery time:

Keep the number of disks in the root volume group to a minimum; HP recommends using three disks, even if the root volume group is mirrored.
Root volume groups with many disks make reinstallation difficult because of the complexity of recovering LVM configurations of accessory disks within the root volume group.
A small root volume group is quickly recovered. In some cases, you can reinstall a minimal system, restore a backup, and be back online within three hours of diagnosis and replacement of hardware. Another benefit is that exact match to the previous root disk layout is not required.
Three disks in the root volume group are better than two, because of quorum restrictions. With a two-disk root volume group, the loss of one disk can require you to override quorum to activate the volume group; if you must reboot to replace the disk, overriding quorum requires you to interrupt the boot process. If you have three disks in the volume group and they are isolated from each other such that a hardware failure only affects one of them, then failure of only one disk enables the system to maintain quorum.
There are two reasons to expand the root volume group beyond a minimal size.
- A very small root disk.
  In this case, HP recommends migrating or installing to a larger disk.
- Providing for dump-to-swap for large memory systems.
  Swap volumes targeted for dump must be in the root volume group. A better solution is to configure an extra dedicated disk for dump up front.
Do not use network-based backup programs such as Omniback or Networker for basic root volume group backups. The complexity associated with these utilities can significantly delay the resumption of processing.
HP recommends using Ignite-UX and the make_net_recovery command to back up and recover your root volume group.
Create adequate documentation.
Output from ioscan -kf, vgcfgrestore -lv for all groups and vgscan -pv, and lvlnboot -v are very minimal requirements. Recovery from almost any problem is possible if these and output from vgdisplay -v for all groups and lvdisplay -v for all volumes are available. Extent mappings are critical to recovery of LVM volumes with corrupted headers. Additionally, output from pvdisplay -v for all physical volumes, although not as important, provides complete volume group information. Hard copy is not required or even necessarily practical, but accessibility during recovery is important and should be planned for.
Many small groups are better than fewer large groups when configuring auxiliary volume groups.
Reloading dozens of gigabytes of data because one disk is missing out of a group after a root disk rebuild can be difficult. Also the scope of a catastrophic single disk failure within a group is minimized with many small groups.
Size implies complexity. The larger the number of disks in a single group, the more chances an administrator has to create an error that affects the entire group. It is simpler to identify and import smaller groups if necessary. It is also simpler to conceptualize and map smaller groups when required.

Preparing for LVM System Recovery

To ensure that the system data and configuration are recoverable in the event of a system failure, follow these steps:

Load any patches for LVM.
Use Ignite-UX to create a recovery image of your root volume group. Although Ignite-UX is not intended to be used to back up all system data, you can use it with other data recovery applications to create a method of total system recovery.
Perform regular backups of the other important data on your system.
Without a valid backup, you risk losing some or all of your data.
Regularly print out your system configuration.
The configuration details stored on the system might not be accessible during a recovery. A printed copy is an invaluable reference. HP recommends printing the configuration details once a week and every time a change is made. Some of the commands create large amounts of output. An alternative to printing them is to output the information to a file and then storing the file on tape which enables quick recovery of the information when needed. You can include this configuration file with the backup in step 3.
The easiest way to save the configuration is to set up a cron job to run regularly, so that the system backs it up automatically.
Use the following commands to get a useful output:
/usr/sbin/ioscan -fk
/usr/sbin/vgdisplay -v
/usr/sbin/lvlnboot -v
/usr/sbin/lvdisplay -v /dev/vgXX/lvYY (for every logical volume)
/usr/sbin/pvdisplay -v /dev/dsk/c#t#d0 (for every LVM disk)
lp /etc/fstab
As an alternative, you can write an intelligent script that detects any changes in the configuration and only print out those changes. An example script is included at the end of this section.
Back up the LVM configuration after every configuration change.
The vgcfgbackup command copies the LVM headers from the system area of the disk to a disk file, which resides in the /etc/lvmconf directory. After this information is stored in a disk file, it can be backed up to tape during file system backups.
The information in this file enables you to replace the LVM headers on the disk in the event of the disk being replaced or your LVM configuration becoming corrupted.
It is important that these configuration backups are taken whenever you make a change to any part of the LVM configuration. By default all the commands perform a backup so a manual vgcfgbackup after each command is not required.
Do this task on a regular basis, whether you have made changes or not. Enter the following command:
# /usr/sbin/vgcfgbackup /dev/vgXX (for every volume group)
Update the boot structures after every change to the root volume group.
This task is only required if you are using LVM on your boot disk. Whenever you make changes to the root volume group, which is usually named /dev/vg00, the BDRA on the boot disk must be updated. This is normally performed automatically by the LVM commands. To update the BDRA manually, enter the following command:
# /usr/sbin/lvlnboot -R

Example Script for LVM Configuration Recording

The following example script captures the current LVM and I/O configurations. If they differ from the previously captured configuration, the script prints the updated configuration files and notifies the system administrator.

#!/usr/bin/ksh
WORKDIR=/lvmbackup # directory is regularly backed up
LOG=$WORKDIR/log
SYSADM=root
if [ -f "$LOG" ]
then
    rm -f "$LOG"
fi
if [ ! -d "$WORKDIR" ]
then
    echo "missing directory $WORKDIR" exit 1
fi
cd $WORKDIR
/usr/sbin/vgdisplay -v -F > vgdisplay.new
LVMVGS=`grep vg_name vgdisplay.new | cut -d: -f1 | cut -d= -f2`
LVMPVOLS=`grep pv_name vgdisplay.new | cut -d: -f1 | cut -d= -f2 | cut -d, -f1`
LVMLVOLS=`grep lv_name vgdisplay.new | cut -d: -f1 | cut -d= -f2`
/usr/sbin/pvdisplay -v $LVMPVOLS > pvdisplay.new
/usr/sbin/lvdisplay -v $LVMLVOLS > lvdisplay.new
/usr/sbin/lvlnboot -v > lvlnboot.new 2> /dev/null
/usr/sbin/ioscan -fk > ioscan.new
cp /etc/fstab fstab.new
for CURRENT in *new.
do
    ORIG=${CURRENT%.new}
    if diff $CURRENT $ORIG > /dev/null then
        # files are the same....do nothing
        rm $CURRENT
    else
        # files differ...make the new file the current file, move old
        # one to file.old.
        echo `date` "The config for $ORIG has changed." >> $LOG
        echo "Copy of the new $ORIG config has been printed" >> $LOG
        lp $CURRENT
        mv $ORIG ${ORIG}old.
        mv $CURRENT $ORIG
    fi
done
if [ -s "$LOG" ]
then
    mailx -s "LVM configs have changed" $SYSADM < $LOG
fi
exit 0

Planning for Recovery

Technical documentation

» Table of Contents

» Glossary

» Index

Preparing for LVM System Recovery

Example Script for LVM Configuration Recording