DOS Partitions (fdisk) and the 2TB Limit

If you are trying to create disk volumes over two terabytes (2TB) you’ll find that fdisk won’t let you do it. The problem lies not with fdisk, but with the old PCDOS disk label used on disks for the last three decades or so. Back in 1981 when the IBM PC was introduced, a disk of over two terabytes would have seemed inconcievable.

Thus, we struggle with the limitations of PCDOS disk labels today.

Some (newer?) versions of fdisk report the problem with large drives, giving this warning:

WARNING: The size of this disk is 8.0 TB (8001389854720 bytes).
DOS partition table format can not be used on drives for volumes
larger than (2199023255040 bytes) for 512-byte sectors. Use parted(1) and GUID
partition table format (GPT).

To get around the size limitation, there is only one solution: dump the PCDOS disk label for another label. The usual recommendation is the GPT (the GUID Partition Table) created by Intel. The GPT has a much larger limit, making 2TB partitions feasable.

However, the Linux utility fdisk does not work with drives that use GPT; thus, you have to use a different partitioning tool. The usual recommendation to Linux users is GNU parted. GNU parted handles multiple partition formats, including GPT. Documentation is available in multiple formats (PDF is one).

The steps to getting a large partition done with parted are simple:

1. Create a disklabel (partitioning) on disk.
2. Create a partition of the appropriate size.
3. Create a filesystem (if needed).
4. Mount.

First, create the GPT disklabel – in other words, create the partitioning tables to support GPT:

# parted /dev/sdc
GNU Parted 2.2
Using /dev/sdc
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Ext Hard  Disk (scsi)
Disk /dev/sdc: 8001GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
 
Number  Start   End     Size    Type     File system  Flags
 
(parted) mklabel gpt
Warning: The existing disk label on /dev/sdc will be destroyed and all data on this disk will be lost. Do you want to
continue?
Yes/No? yes

Then after this, create a partition:

(parted) mkpart primary xfs 0 8001.4GB
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c

This is what happens when the disk is not aligned properly on the appropriate boundary. Traditionally, the boundary was 512 bytes; now it is changing to 4K. GPT also apparently use a significant portion of the beginning of the disk.

To get around the alignment problem, you can use a start position of 1MB and an end position 1MB from the end:

(parted) mkpart primary xfs 1 -1
(parted) p
Model: Ext Hard  Disk (scsi)
Disk /dev/sdc: 8001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  8001GB  8001GB               primary

Parted supports a wide variety of units (and defaults to megabytes), all of which can be specified during numeric input – such as for start and end points. Using an end point of “100%” is probably just as good as using “-1″ (1MB from end).

Jamie McClelland has a nice article about 2TB drives and alignment. Don’t blindly trust the tools to get the partitioning right; specify the appropriate numbers and units in order to force correct alignment.

GNU parted also supports an option that at least suggests it will use the best alignment when it can:

parted --align optimal

Again, don’t blindly trust it: check the numbers.

Installing an ISO for Use in VMware ESXi 4

Having a CDROM available for use in a VMware ESXi server can be very useful. However, what if there is no way to get the CDROM into the host? Better yet, what if you just want to avoid the hassle of having to go back and forth inserting and pulling CDROMs?

There is a way to get the ISO onto the VMware ESXi server and make it available to a virtual machine. First, you have to put the ISO on the server itself. Secondly, you have to make the ISO available to the host. That’s it.

First step: copy the ISO to the ESXi server. You could do this with scp (if you’ve activated ssh) or by creating the ISO on the fly from the original disk. You could also use scp on the ESXi server to copy an ISO into the server without activating ssh.

The best destination would probably be a directory in the datastore you set up during installation of the ESXi server. If the datastore is datastore1, then the location for ISOs could be /vmfs/volumes/datastore1/ISO Images/. You’ll have to create the directory ISO Images yourself. Note that “datastore1″ (or whatever your datastore name is) is not used by VMware ESXi, but translates to a hex string: so don’t be alarmed when you see it instead of the datastore name you expect.

Lastly, make the ISO available to the virtual machine. Using vSphere Client, you can get to the appropriate place (“Edit Settings”) in a number of ways. Right click on the virtual machine in the list of machines (either on the tree to the left, or on the list in the Virtual Machines tab) and select “Edit Settings”. Alternatively, click on the virtual machine on the left, then click on the Summary tab – and on that tab, click on “Edit Settings”.

In Edit Settings, click on the CD/DVD Drive 1 entry and then click the button for Datastore ISO File (on the right). If you click on browse – then on the drop-down menu at the top, look for “Datastores” (at the bottom). Your datastores will be shown when you click on Datastores, then the directory you created will be in the datastore you used earlier. Select the ISO you want and click Open – then OK.

Once the CDROM is chosen, you can then install from it, or use it in any way you like from the virtual machine. Don’t forget: this is just like putting a new CDROM into the machine – so whatever your OS needs to have happen, you have to do it after “putting the CDROM” into the drive.

Installing VMware vSphere CLI 4.0 in Ubuntu 10.04 LTS

Installing the VMware vSphere Command Line Interface (CLI) has the potential for problems. In my case, it generated an error – a three-year old error. Perl returns the error:

undefined symbol: Perl_Tstack_sp_ptr

Not only has this error been around for three years, it also has shown up in numerous other instances. Ed Haletky wrestled with the error in VMware vSphere CLI back in June of 2008. The error surfaced in Arch Linux in 2008, both in running their package manager and in running cpan itself. This error also came up (again in 2008) in attempting to build and run Zimbra. (The response from Zimbra support was cold and unwavering: we don’t support that environment and won’t discuss it. How unfortunate.) The error also affected the installation of Bugzilla according to this email thread from 2009.

On the Perl Porters mailing list, there is an in-depth response as to what causes this error. From reading these messages, it appears that there are two related causes:

• Using modules compiled for Perl 5.8 with Perl 5.10
• Using modules compiled against a threaded Perl with an unthreaded Perl

One recommended solution is to recompile the modules using the cpan utility:

cpan -r

That may or may not be enough; it depends on if there were other errors. In attempting to run the vSphere CLI, I get this error:

IO::Compress::Gzip version 2.02 required--this is only version 2.005

To fix this, I ran cpan this way:

cpan IO::Compress::Gzip

In my case, that loaded IO::Compress::Gzip version 2.033.

I also loaded the libxml2-dev package; I don’t know if that was necessary or not:

apt-get install libxml2-dev

Whenever using cpan, I always wonder how it affects my packaged installations and whether it installs for all users or just me (and how to control that) – but I’ve never had any problems and installs as root seem to go into /usr/local – which makes sense.

Having done all this, I can now use the vSphere CLI to activate SNMP on the ESXi 4 servers. For the record, this is an integral part of ESXi 4 and supports all SNMP polling and traps – previously, only SNMP traps were supported. Certainly a nice improvement.

Installing Java on Ubuntu Lucid or Maverick

Sun Java 6 is missing from a standard Ubuntu Lucid or Ubuntu Maverick installation, as well as missing from all of the APT repositories that come prepackaged with Ubuntu – including metaverse, universe, or any other.

Apparently, with the release of Ubuntu 10.04 Sun Java moved to the Canonical Partner repository. This means that to get Sun Java, you have to add the Canonical Partner repository to your list of APT repositories:

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"

If using Maverick, then replace “lucid” with “maverick”:

sudo add-apt-repository "deb http://archive.canonical.com/ maverick partner"

Then update your APT cache and install:

sudo apt-get update
sudo apt-get install sun-java6-jre sun-java6-plugin

Once this is done, Sun Java is installed. However, Sun Java is not set as the default Java to use. To make this change, use update-alternatives:

sudo update-alternatives --config java

With the appropriate selections, this should configure Sun Java as the default Java for applications to use. Browsers will have to be restarted to activate the new Java plugin.