Mirroring HP-UX Root Disks (LVM)

Mirroring the root disk in HP-UX is not necessarily as hard as it sounds. There are excellent directions out there. Firstly, go get the PDF from HP titled “When Good Disks Go Bad” (make sure you have the most recent!). This is frequently recommended and you should have it if you don’t already.

Appendix D: Procedures contains detailed instructions on both mirroring PARISC boot disks and Itanium disks. Here is a broad overview to get you prepped:

  • 1. Prepare the disk for LVM. This means using the insf command to add the disk files as appropriate, and using pvcreate to lay down LVM structures.
  • 2. Add the new volume to the root volume group. Use vgextend to add the new disk to the volume group.
  • 3. Make the disk bootable. This is a hairy part. You have to copy the proper data from the original disk, and so forth. Follow the directions closely.
  • 4. Mirror the logical volumes on the first disk to the new one. This has to be done in a specific order or it will fail. It’s not hard, just time consuming.
  • 5. Update system data with new boot information. This involves configuring the machine to boot with the new disk.

Setting up Itanium boot mirrors requires a few additional steps: the disk must be partitioned into three parts: 1) EFI boot partition; 2) HP-UX root disk data; and 3) HP Service Partition. The second partition is equivalent to the entire disk in a PARISC system. The first partition (EFI boot) is like a mini-disk that contains all of the data needed to boot HP-UX.

The last partition – the HP service partition – is not configured unless you copy it from the original source. If you do not do this, you do not actually have a service partition – which could be a nasty surprise when you need it!

HP-UX Boot Disks on Integrity systems (in contrast to PARISC systems)

On PARISC-based HP-9000 systems, configuration of system boot disks was simple: the entire disk was used, split apart using logical volumes with LVM. Thus, an HP-9000 system (PARISC) will have a “standard” full disk for the boot disk – such as /dev/disk/disk56 (using the new disk labeling).

However, when using Integrity systems, space must be made at the beginning for EFI and at the end for an HP System Partition – which shows up in HP-UX as a disk with three partitions.

An Integrity system will have several more disks associated with the boot disk (using disk32 as the example):

  • /dev/disk/disk32 – this is the full disk. The disk, however, is split into three parts as described below.
  • /dev/disk/disk32_p1 – this is the EFI partition. When the system boots, it is this partition which loads the EFI data and runs the EFI shell.
  • /dev/disk/disk32_p2 – this is where the HP-UX operating system data is stored. The logical volumes associated with HP-UX will be created here, and /dev/disk/disk32_p2 will be in volume group vg00.
  • /dev/disk/disk32_p3 – this partition is an HP system partition of some sort. It is automatically created during installation.

Thus, if you are on an Integrity system and are attempting to follow some older directions, remember to use the appropriate disk label.

There are tools that are designed for Integrity systems with EFI that will help maintain or document these partitions. First is idisk:

# idisk -p /dev/rdisk/disk32
idisk version: 1.44

EFI Primary Header:
        Signature                 = EFI PART
        Revision                  = 0x10000
        HeaderSize                = 0x5c
        HeaderCRC32               = 0x30a62aae
        MyLbaLo                   = 0x1
        MyLbaHi                   = 0x0
        AlternateLbaLo            = 0x88bb991
        AlternateLbaHi            = 0x0
        FirstUsableLbaLo          = 0x40
        FirstUsableLbaHi          = 0x0
        LastUsableLbaLo           = 0x88bb93f
        LastUsableLbaHi           = 0x0
        Disk GUID                 = 43b615f6-a561-11dd-8000-d6217b60e588
        PartitionEntryLbaLo       = 0x2
        PartitionEntryLbaHi       = 0x0
        NumberOfPartitionEntries  = 0xc
        SizeOfPartitionEntry      = 0x80
        PartitionEntryArrayCRC32  = 0x97c6286c

  Primary Partition Table (in 512 byte blocks):
    Partition 1 (EFI):
        Partition Type GUID       = c12a7328-f81f-11d2-ba4b-00a0c93ec93b
        Unique Partition GUID     = 43b61920-a561-11dd-8000-d6217b60e588
        Starting Lba Lo            = 0x40
        Starting Lba Hi            = 0x0
        Ending Lba Lo              = 0xf9fff
        Ending Lba Hi              = 0x0
    Partition 2 (HP-UX):
        Partition Type GUID       = 75894c1e-3aeb-11d3-b7c1-7b03a0000000
        Unique Partition GUID     = 43b6195c-a561-11dd-8000-d6217b60e588
        Starting Lba Lo            = 0xfa000
        Starting Lba Hi            = 0x0
        Ending Lba Lo              = 0x87f37ff
        Ending Lba Hi              = 0x0
    Partition 3 (HPSP):
        Partition Type GUID       = e2a1e728-32e3-11d6-a682-7b03a0000000
        Unique Partition GUID     = 43b61970-a561-11dd-8000-d6217b60e588
        Starting Lba Lo            = 0x87f3800
        Starting Lba Hi            = 0x0
        Ending Lba Lo              = 0x88bb7ff
        Ending Lba Hi              = 0x0

Be careful in using idisk, as you can completely destroy your data easily with idisk, and even render your machine unbootable.

Then there are a number of utilities to work with the EFI partition; these are:

  • efi_fsinit – initialize EFI partition;
  • efi_cp – copy EFI files to and fro;
  • efi_mkdir – make a directory on a EFI partition;
  • efi_ls – list files on a EFI partition;
  • efi_rm – remove files on an EFI partition; and
  • efi_rmdir – remove a directory from an EFI partition.

These commands are further documented in efi(4) and in their respective man pages.

How to change volume names in HP-UX (LVM)

Changing LVM logical volume names in HP-UX is extremely simple: enter into the appropriate volume group directory, and rename both the logical volume file (typically starting with lv) as well as the raw logical volume file (which normally begins with rlv). When both files are renamed appropriately, you are done.

Changing a volume group name is not as easy – not hard, just not as easy. First, you should vgexport the volume group (using the -m option to create a map):

# vgexport -m /opt/vg.map -s vgmine

This deletes the volume group completely from LVM records and from the /dev filesystem tree. It does not, however, change data on the disk – the logical volumes and associated filesystem data are preserved.

Having done this, recreate the volume group with the new name and use vgimport:

# mkdir /dev/vgnew
# mknod /dev/vgnew/group c 64 0x070000
# vgimport -m /opt/vg.map -s /dev/vgnew

This should then import the volume group, and give it the same logical volume names as before. Without the map, the logical volumes would still be present and accounted for and with all data preserved – but the names would be default names (lvol1, lvol2, etc.).

Book Review: Disk and File Management Tasks on HP-UX

Using LVM on HP-UX can be troublesome, and any deviation from the usual steps can cause problems. Specifically, if the data on disk and the data in the system vary, then there can be a variety of problems. The usual symptom is that the LVM environment tools won’t let you do something that seems obvious (like removing a volume group when the relevant disk volume is missing).

The book Disk and File Management Tasks on HP-UX (by Tom Madell) is basically a guide to all things LVM. Despite its age (1996), it is still relevant, and covers LVM, OnlineJFS, and VxFS. The book is also not standard publishing quality, but is a bound set of facsimile pages – this doesn’t detract from the contents however.

The book covers all aspects of disk and file management, including such advanced topics as creating your own bootable root volume, recovering from loss of /etc/lvmtab, and many other details. It covers commands such as vgcreate, vgexport, vgimport, vgreduce, vgremove (and their logical volume counterparts).

It also covers converting to VxFS (from older filesystems), mirroring, and creating striped volumes.

This is a worthy addition to any HP-UX administrator’s library.

Large LVM volumes on HP-UX

When creating a large LVM volume group (over roughly 100G in size) under HP-UX, the process may fail mysteriously or hang. Working through the SAM administration GUI creates more mysterious errors

The solution and cause to this problem was explained nicely by Jim Marsden clear back in 2004. The problem comes up because when the LVM software was created, 9G and 18G volumes were the most common. Now with 100G+ volumes, the numeric limitations of LVM are becoming known.

Another limitation is that the volume group overhead data is becoming so large that it can no longer fit into a single physical extent (PE). This reserved area is called the Volume Group Reserved Area (VGRA). The VGRA is only one of several reserved areas, but is the only one we’ll discuss here.

To get around these limitations, you will have to create the volume group with a PE size large enough to contain the full VGRA, as well as a PE size that will accommodate the entire disk. If the volume group is already created with the default PE size of 4M, then it will have to be recreated with a new PE size. PE sizes are typically 4M, 8M, 16M, 32M, 64M, 128M, and 256M. Pick the smallest size that will work.

After the PE size is set, another limit must be set: the maximum number of physical extents per physical volume. This limit can be found by dividing the size of the largest disk by the PE size (mind those units!). For the current gigabyte-sized drives, this formula is:

MaxPEperPV = (gigabytes * 1024) / PEsize

For example, a 200G drive with 16M PEs translates into a MaxPEperPV of 12800.

Given these two pieces of information, the volume group can be created using the standard LVM utility vgcreate at the command line thusly:

vgcreate -s PEsize -e MaxPEperPV NewVG Disk1 Disk2 ...

For example, using our current values, you might do:

vgcreate -s 16 -e 12800 /dev/vg00 /dev/dsk/c1t0d0

This problem might be present in some form in the Linux LVM implementation, but I rather doubt it – at least not until terabytes become common (uhoh….).