When creating a large LVM volume group (over roughly 100G in size) under HP-UX, the process may fail mysteriously or hang. Working through the SAM administration GUI creates more mysterious errors
The solution and cause to this problem was explained nicely by Jim Marsden clear back in 2004. The problem comes up because when the LVM software was created, 9G and 18G volumes were the most common. Now with 100G+ volumes, the numeric limitations of LVM are becoming known.
Another limitation is that the volume group overhead data is becoming so large that it can no longer fit into a single physical extent (PE). This reserved area is called the Volume Group Reserved Area (VGRA). The VGRA is only one of several reserved areas, but is the only one we’ll discuss here.
To get around these limitations, you will have to create the volume group with a PE size large enough to contain the full VGRA, as well as a PE size that will accommodate the entire disk. If the volume group is already created with the default PE size of 4M, then it will have to be recreated with a new PE size. PE sizes are typically 4M, 8M, 16M, 32M, 64M, 128M, and 256M. Pick the smallest size that will work.
After the PE size is set, another limit must be set: the maximum number of physical extents per physical volume. This limit can be found by dividing the size of the largest disk by the PE size (mind those units!). For the current gigabyte-sized drives, this formula is:
MaxPEperPV = (gigabytes * 1024) / PEsize
For example, a 200G drive with 16M PEs translates into a MaxPEperPV of 12800.
Given these two pieces of information, the volume group can be created using the standard LVM utility vgcreate at the command line thusly:
vgcreate -s PEsize -e MaxPEperPV NewVG Disk1 Disk2 ...
For example, using our current values, you might do:
vgcreate -s 16 -e 12800 /dev/vg00 /dev/dsk/c1t0d0
This problem might be present in some form in the Linux LVM implementation, but I rather doubt it – at least not until terabytes become common (uhoh….).
5 thoughts on “Large LVM volumes on HP-UX”
Your instructions appear to be pretty straight forward, but they’re still talking about drive sizes in hundreds of gigabytes. I’ve got a hardware RAID with a single logical drive configured as 2.72 TB. I’ve just tried
vgcreate -s 64 -e 44565 /dev/vg04 /dev/dsk/c5t0d0
on the command line and it’s hanging. (In SAM, it reports a negative number for the disk size.) Am I running up against other size limits?
BTW, I should have mentioned that I’m running on HP-PUX 11.00 w/ the latest to date cumulative LVM patch (PHKL_35742).
That seems straightforward. As given, you have 64 Megabytes for PE size, and 44565 for maximum number of PE. This translates to 64M * 44565 = 2852160M total size or 2852160 / 1024 = 2785 Gigabytes or 2785 / 1024 = 2+ Terabytes.
Perhaps you need a larger PE size, to account for the VGRA. How about one of these?
vgcreate -s 128 -e 22282 /dev/vg04 /dev/dsk/c5t0d0
vgcreate -s 256 -e 11141 /dev/vg04 /dev/dsk/c5t0d0
If you have questions about how SAM is getting its answers, look in /var/sam/log/samlog for the text of the command it used to get its data.
Note, too, that SAM is no longer supported as of 11i v3.
The reason for this is because LVM in HP-UX 11.00 thru 11iv2 supports a max PV size of 2TB. 11iv3 supports 16TB PVs. See the LVM_limits whitepaper from HP on docs.hp.com
Thanks! That was a very useful post.