Adding a New Disk to HP-UX with Veritas Volume Manager

Adding a new disk to a HP-UX system that is using Veritas Volume Manager is not difficult. It does require some extra steps, but if you follow along there are no problems.

Note well that this discussion does not have anything to do with LVM whatsoever, and the use of the Veritas File System (VxFS) is a minor tangent – other filesytems (like HFS) can be used – but if you’re not using OnlineJFS (which is nearly all of VxFS and its capabilities), the question would be why?

First of all, the usual steps for adding a disk to HP-UX are necessary:

  1. Run ioscan to scan for the new disk and to allocate resources in the kernel for it. My actual favorite invocation for this command is ioscan -fnC disk
  2. Next, run the insf command to create device files. I prefer to use ioscan -eC disk in this instance.

Now we add the disk to the Veritas Volume Manager. Run these commands:

  1. vxdctl enable
  2. vxdisk scandisks

At this point, there is now an additional disk available for use by vxvm but which is not a part of any disk groups yet.

To add the disk to a disk group, use vxdiskadm and option 1. You can use "list" here to get the name of the unallocated disk. Enter the name of the disk and answer the prompts; in most cases you will want the default.

Now to extend a file system in the volume group to use the new disk, you have to figure the amount of space available to use. You can use the command vxdg -g diskgroup free to find the free space in the disk group "diskgroup" - but this does not tell you the actual amount of space you can extend by; that is, it does not include striping and other restrictions - it only includes a flat number of unallocated blocks.

To get the amount of disk space you can grow a volume by, use the command vxassist -g diskgroup maxgrow volume. This will give you the actual amounts (if it is possible to grow). Replace the diskgroup and volume with the actual names of your disk group and volume respectively.

To actually grow the filesystem, you can do the following:

vxresize -b -F vxfs -t mytag group 999999

Fill in with the appropriate information instead of mytag (a tag to identify background vxvm operations), instead of group (disk group), and instead of 999999 (use the max amount of growth - or other appropriate number - identified by the maxgrow operation).

VxVM is aware of VxFS, so resizing the volume will also resize the filesystem as well; it uses fsadm in the case of OnlineJFS.

Veritas Volume Manager (VxVM) and HP-UX

HP-UX comes with VxFS (the Veritas File System) and the Logical Volume Manager (LVM). Only the on-disk filesystem layout comes from Veritas; HP’s volume management is all their own. There’s nothing wrong with HP’s LVM – I tend to prefer it, but then that’s what I know.

Veritas (now Symantec) offers another, competing product called Veritas Volume Manger (refered to as VxVM). The tools are different, the layout is different, and the capabilities are different. Knowing LVM won’t help you much, though the most basic concepts are the same: collect a series of disks together and then parcel them out as a single large group, with user-defined subdivisions.

Veritas Volume Manager is now a part of the Veritas Storage Foundation.

An nice set of links to documents can be found at Aziz’s Blog. In particular, the Veritas Volume Manager Administrator’s Guide has been indispensable. Just about everything you can imagine you might need to do is located here.

Cuddletech offers the VxVM Quickstart with some older, but worthwhile documents that describe VxVM and its concepts. Likewise, Unixway offers a wide variety of documents on VxVM over several versions, as well as tutorials and more.

The AdminsChoice also has a good set of tutorials; there is Veritas Volume Manager part 1 and part 2 (focusing on vxassist).

There is a mailing list, but in recent months the activity has been rather sparse.

If you want to take your knowledge all the way, you can become Symantec Certified for the Veritas Storage Foundation (which mainly includes VxFS and VxVM).

Veritas VxVM works very well on HP-UX, and it is possible to create a root disk that utilizes VxVM and VxFS. When using VxVM, LVM is not used (unless a particular disk uses LVM instead of VxVM). The commands are the same across different platforms, and the on-disk layout is the same – so it should be possible to take a set of disks from a Solaris system and put them onto an HP-UX system and still read the data (but watch out for differing byte ordering!).

In the future I hope to discuss more on VxVM; we’ll see.

Powered by ScribeFire.

Sparse files and Virtual Machines

Sparse files are files that take up less space on disk than they actually use. How is this possible? Any blocks with zeros in them are not stored but are silently skipped. When the system retrieves these blocks later, it returns a block of zeros – and if data is put into the block, it is saved onto the physical disk appropriately.

Sparse files require file system support; if the file system you use doesn’t support sparse files, then you have no recourse but to store every file in the normal fashion. Notable filesystems without support for sparse files are any FAT filesystem (MSDOS), HPFS (a FAT derivative used by OS/2), HFS+ (the Macintosh filesystem), and OCFS v1 (the Oracle Cluster File System). Modern file systems such as NTFS (Windows NT Filesystem), VxFS (Veritas File System), GFS, XFS, JFS, ext2, ext3, Reiser3, Reiser4, and many others all support sparse files. No word on whether or not ODS-5 (OpenVMS) supports sparse files or not.

Sparse file support is valuable for virtual machines as a virtual hard drive by its necessity will be of significant size – but by its nature will also have a lot of empty space.

However, over time, data gets written to the virtual machine hard drives then deallocated. The data remains in these blocks, and the blocks remain on disk. These blocks accumulate, and the file expands – filled with data that is no longer used.

The only way to free these unused blocks is to zero them out and copy the file as a sparse file. You might also want to defragment the disk if this is relevant to your virtual machine’s operating system.

First, zero the unused blocks. Typically, this is done with an erasing program suitable to the operating system in the virtual machine. Make sure that the program is a) erasing only unused data! and b) zeroing out the data, not using random patterns or other wipe patterns.

Once the program is done wiping, shut down the virtual machine. (Yes, this process means downtime.) Copy the original file to the new file using the appropriate flags to make the new file sparse. For Linux (using GNU cp) one could do this:

cp --sparse=always oldvm newvm
rm -f oldvm
mv newvm oldvm

These steps will replace the old VM file with a new sparse VM file that uses much less space.

Quotas

I came across this article about Veritas VxFS disk quotas in Solaris, and it seems like a good concise description of them. However, according to the example, 1 block is 1 kilobyte – not 1 byte (!). I haven’t vetted the article nor checked for accuracy, so caveat emptor!

However, even with a full VxFS installation from Symantec, HP-UX appears to use something altogether different. The commands are present whether the VxFS product from Symantec is installed or not (HP-UX comes with a base VxFS installation). The quota commands are mainly included in the DiskQuota package, but also are included in the JFS package (quotacheck_vxfs), the OS-Core package (quotacheck), and even the CIFS-Server package (smbcquotas).

More on these commands later.

Do you know Veritas?

I’ve seen this question posed, and been posed it myself. Having worked with HP-UX these some years, answering this question becomes even more complicated. After all, Veritas (now a part of Symantec) has these various products:

  • Veritas File System
  • Veritas Volume Manager
  • Veritas Cluster Services
  • Veritas NetBackup

To add to the complexity, HP’s HP-UX shipped with a base version of the Veritas File System until 11i when the operating system began shipping with a full version of the Veritas File System. Besides that, the complete Veritas File System implementation HP uses was rebranded by them as OnlineJFSand the HP-UX implementation uses VxFS within LVM, not the Veritas Volume Manager.

So what does it mean to say “I know Veritas”? Nothing. Better to be specific – and if you are familiar with HP’s HP-UX filesystem implementation, say so (and clarify). All you can do next time someone asks you this question is to educate them on the different products that are branded “Veritas” so you can answer them clearly.

Book Review: Disk and File Management Tasks on HP-UX

Using LVM on HP-UX can be troublesome, and any deviation from the usual steps can cause problems. Specifically, if the data on disk and the data in the system vary, then there can be a variety of problems. The usual symptom is that the LVM environment tools won’t let you do something that seems obvious (like removing a volume group when the relevant disk volume is missing).

The book Disk and File Management Tasks on HP-UX (by Tom Madell) is basically a guide to all things LVM. Despite its age (1996), it is still relevant, and covers LVM, OnlineJFS, and VxFS. The book is also not standard publishing quality, but is a bound set of facsimile pages – this doesn’t detract from the contents however.

The book covers all aspects of disk and file management, including such advanced topics as creating your own bootable root volume, recovering from loss of /etc/lvmtab, and many other details. It covers commands such as vgcreate, vgexport, vgimport, vgreduce, vgremove (and their logical volume counterparts).

It also covers converting to VxFS (from older filesystems), mirroring, and creating striped volumes.

This is a worthy addition to any HP-UX administrator’s library.

VxFS (or HP Online JFS) Snapshots

A disk snapshot is a snap in time, a picture of what a disk looked like “back then”. This can be very useful for maintenance.

For example, being able to freeze a Caché instance, take a disk snapshot, then thaw the Caché instance will permit you to take backups or copies of a Caché database with minimal downtime.

For HP-UX Online JFS and Veritas VxFS the commands are the same (since these are actually the same product – or close to it). To actually do a snapshot:

mount -F vxfs -o snapof=/var/cache/db /dev/snap01 /snap

The first file system presented in the command line is key: it is the source of the snapshot. Note that it can be either a device or a current mount point. The second (device) is a filesystem prepared to hold a snapshot, and the last is the usual mount point.

Once this is done, the normal filesystem can continue to be used while the snapshot retains the older data as it was taken. In the example above, /var/cache/db could be used normally while the snapshot resides on /snap. If there was a directory /var/cache/db/db01 then there would also be a /snap/db01 available as well.

One caveat is that as long as the snapshot is mounted and in use, the changes to the original filesystem are being saved – it is possible that the snapshot volume can run out of space. When this happens, you will receive what may appear to be mysterious disk full errors unless you realize what is happening. So don’t keep your snapshots around forever.