Test Plan Charlie: 41000 Linux Servers on One Box

There was a test done many years ago by David Boyes, an engineer working out of Virginia.  The test was simply to run as many Linux servers on one IBM zSeries mainframe – and to keep adding them until something broke.

The test hit the limit at 41,400 Linux servers – and nothing ever “broke.” This project was widely reported at the time, though it seems to be forgotten now. However, the test caught my fancy. That’s a lot of Linux machines.

As was mentioned, this report was widely reported: Linux Journal had an article on 1 June titled The Penguin and the Dinosaur from Adam Thornton.  That same day, Daisy Whitney authored an article, Linux on Big Iron – possibly in Datamation. Scott Courtney (the Technical Editor for Internet.com) wrote S/390: The Linux Dream Machine on 23 February and wrote It’s Official: IBM Announces Linux for the S/390 on 17 May. What really stands out?  All of these articles reporting on the S/390 and on Test Plan Charlie occurred nine years ago, in 2000.

Scott Courtney followed his articles up with an interview with David Boyes in 2001.

There is one more thing about David Boyes: following Test Plan Charlie, he went on to create Sine Nomine Associates and showcased OpenSolaris running on the IBM zSeries in November of 2007 – with attendant press releases from IBM. Certainly, David is not one to sit idle – and is a figure to contend with in the IBM zSeries arena. IBM has, since the original announcement nine years ago, pushed Linux on zSeries with vigor.  One irony: Test Plan Charlie was part of a study for an IBM customer that was deciding whether to use their existing S/390 or whether to use a new Sun set up.

There is even an open source IBM mainframe emulator called Hercules, which allows the rest of us to try it out and see what happens – even though you won’t be able to run under z/VM, as that is an IBM product.

Update: there was a nice set of updates about OpenSolaris on zSeries over on DancingDinosaur: Here comes (and goes) the Sun (12 April 2009) and Slow times for OpenSolaris on System z (21 July 2009).

Powered by ScribeFire.

Renaming a host (UNIX, OpenVMS)

Renaming a host is not, in general, a pleasant experience. The general requirement is that you must find everywhere that your hostname is specified and change it.

In OpenVMS, this can be an extensive process, and even require relicensing if you have licenses that depend on your hostname. It can also require rebooting of an entire VMS cluster if you miss changing a parameter. The full details are in the VMS FAQ such as this one (dated from 2001) from faqs.org or this one from HP (undated). Hoffman Labs has a copy from September 2006; there is information on changing a node name in section 5.7.

Not that in changing the OpenVMS hostname in a cluster, you must change the SCSNODE parameter (which changes the cluster node’s nodename). If you change the SCSNODE parameter, you must change the SCSSYSTEMID as well or the entire cluster will refuse to function until it is reconfigured. The cluster tracks the pairing between these two parameters, and if the pair changes, then the cluster stops working normally.

For UNIX in general, one way to do it is to go to the /etc directory as root and run a search:

$ su -
Password:
# cd /etc
# find . -type f -print | xargs grep -i myhost

After running this, change all of the instances of myhost that is found.

This is the way to change hostnames in Solaris, including Solaris 9 and Solaris 10. Debian and derivatives (including Ubuntu and Linux Mint) and HP-UX make it simpler.

In Debian, there is a file called /etc/hostname. This will contain the current setting of the hostname. Change this to your desired new hostname, then run the shell script /etc/init.d/hostname.sh.

In HP-UX, change to root and run the program set_parms with the hostname option:

# set_parms hostname

For all of these possibilities, the best thing to do is to reboot afterwards: this will test the new setup as well as change any in-memory hostname settings.

Changing a hostname is a drastic measure, and will include much in the way of system modification and updates. Changing the actual hostname is very likely only the beginning; there may be clients that are set up to contact the host, and any services that the server provided (e.g., NTP server, FTP server, web server, NIS server, etc.) will require reconfiguration on the clients to use the new hostname.

In summary, the very best thing to do is to get the name right in the first place.

Powered by ScribeFire.

UNIX and OpenVMS Online Resources

It is possible to get free online access to UNIX or to OpenVMS; these can be useful in building up your experience on a platform when starting from scratch – or when a review is required.

One of the oldest public access systems in the country is the Super Dimension Fortress (or SDF as it is usually called). SDF offers free accounts, but does ask for US$1 to gain standard access. This isn’t because access is expensive, but because too many people have used the facilities for nefarious purposes (the process suggests that the new user is not a person who will strike and leave).

SDF runs NetBSD on DEC Alphas; this was driven mainly by security and stability. Previously, Stephen Jones, the proprietor, ran SDF using Linux on Intel for several years (which he describes as “the dark years”). BSDTalk had an interview with him back in 2006.

You could also try PolarHome – this shell provider provides access to hosts running Linux (Red Hat, Debian, SUSE, Ubuntu, or Mandriva), OpenVMS (Alpha or VAX), OpenBSD, FreeBSD, NetBSD, HPUX, IRIX, QNX, Solaris, Ultrix, AIX, Tru64, and OpenStep. Unfortunately it requires payment for shell accounts – again because of abuse. The payment is 10 units of your local currency or US$2, whichever is more – and this is per host as well. No other site provides this diverse of a selection.

For truly free UNIX shell accounts, one can try Grex, which is a more professionally-run system (Polarhome and SDF are sole proprietorships). Grex offers totally free shell accounts, but also has memberships (for people to help support the site). It is possible that Grex has the most users as well. Like the others, paid membership does have its privileges – but unlike the others, membership is mainly to provide support for Grex, rather as a security feature.

For OpenVMS, there is a very unique online shell provider: Deathrow Cluster. This is a cluster of three machines running OpenVMS 7.3 – one VAX, one Alpha, and one emulated VAX (SIMH) on a dual Xeon machine. This last is a perfect example of what can be done with an emulator, especially with SIMH which can emulate all manner of old Digital and IBM hardware. However, SIMH does not emulate the Digital Alpha, unfortunately. Like Grex, Deathrow provides completely free shell accounts; like SDF and Polarhome, it is (or appears to be) mainly one person’s purpose to keep it running with a lot of volunteer help.

Any of these will be good sources to keep your shell skills sharp – and in some cases, programming as well. They’re also good people to support; why not offer them some donations if you can?

ZFS and Apple’s new MacOS X (Snow Leopard)

Sun’s ZFS is, by all accounts, the most revolutionary file system to come along in years. The Wikipedia entry on ZFS has some details, and Sun has a ZFS Learning Center where you can learn how to use it.

Of course, ZFS is in OpenSolaris, but it is also being introduced into FreeBSD as well.

The Solaris Internals site has a beautiful ZFS Best Practices Guide.

What does all of this have to do with Apple’s MacOS X (Snow Leopard)?

Just this: early in the development of MacOS X 10.6, Apple announced that they would use ZFS in the new MacOS X Snow Leopard. The ability to read ZFS volumes had been put into MacOS X Leopard Server. However, ZFS is missing from MacOS X Snow Leopard and Snow Leopard Server entirely. Robin Harris over at ZDNet has an excellent article that explains it all. He then went on to expand on his ZDNet article with more details.

The one detail in particular I wanted to note is the lawsuit between NetApp and Sun over ZFS and related patents. Groklaw has been following the lawsuit, but the last update from Groklaw is October 2008; Sun has more details on their lawsuit page. Way back in 2007 when the patent lawsuit erupted, CompuerWorld had an article suggesting that Apple might be forced into the lawsuit since it had been courting ZFS – or could be sued next if NetApp won. Neither Apple nor NetApp would comment.

It would also be worth noting that when IBM was in talks to buy Sun in March 2009, there were articles about how the ZFS lawsuit would affect such talks – especially given that IBM and NetApp had a strong partnership already (IBM remarkets NetApp hardware for instance). AMLawDaily had a nice article about it, as did CNET. It wasn’t much more than a month later – in April 2009 – that Sun announced it was being bought by Oracle.

Powered by ScribeFire.

Solaris 2009.06 (and 2008.10) on an HP nc4010 Laptop

This laptop is a delightful little machine, and borders on being a “netbook” though it has better specs than what would normally be called a netbook. Last night, I overwrote my Solaris 2008 install (of no more than a week or two) with the very latest Solaris 2009.

Since I had problems with running the LiveCD with a full X display, I opted for the text console and began the installation from there. Installing from text mode is simple, but mostly undocumented:

1. Find an X server to use. There are nice X servers out there, not to mention the ones included with every UNIX and Linux installation. I use the Xming server for Windows.
2. Boot the OpenSolaris LiveCD into text mode.
3. Configure the OpenSolaris client to use the remote X server display. This could be as simple as logging in as root (password opensolaris) and setting the DISPLAY variable. This, of course, is not secure: to be secure, log in as root, start the ssh server, and log in over the network using the command ssh -X.
4. Start the install process. This involves the code: pfexec /usr/bin/gui-install. After this, the GUI install process should appear on the remote display and the expected install process can begin.

Installing OpenSolaris this way onto the HP nc4010 was smooth, and the environment works well and is clean. There is a lack of official applications, but this may be expected, though disappointing: open source focus on OpenSolaris is not what it is for FreeBSD or for Linux.

The machine has a 1.7GHz Pentium M and 1Gb of main memory; this seems sufficient – so far. This machine is likely to become my secondary: the primary is Linux Mint – with all of the applications that a Debian/Ubuntu derivative can count on.

OpenSolaris Bundled with Toshiba Laptops

I don’t know how this slipped past me. Earlier this year, Toshiba and Sun announced that OpenSolaris would ship on certain Toshiba laptops then followed through with their announcement on schedule.

This is not necessarily as unusual as it might sound at first glance: Toshiba has been remarketing the Sun Sparc chip for some time, and has a history of working tightly with Sun Microsystems. The only other company that was just as likely to create an OpenSolaris laptop would be Tadpole, although Tadpole laptops have in the past used the Sparc chip from Sun instead of Intel chips.

Two different models are available: the Portege R600 and the Tecra M10. They are available from the OpenSolaris folks at Sun with free shipping until the end of June.

CommunityOne West: June 1-3 2009

Sun is holding its CommunityOne West conference at the Moscone Center June 1-3 in San Francisco, California.

Sounds like there will be a wide range of Solaris and Open Source topics, including virtualization, system management, cloud development, mobile development, web development, and much more.

The OpenSolaris community will be there in force, so don’t miss it!

This is one of the first conferences since Oracle announced their acquisition of Sun; it would be interesting to be plugged into the rumour mill on the floor.

Did anyone go to CommunityOne East in New York City?

Powered by ScribeFire.

About ZFS

I’ve known that ZFS was a revolutionary filesystem, but never understood the details. Now there is an article that explains why ZFS is so desirable, and does so well.

Apple started supporting ZFS read-only in Leopard, and has released beta versions of Leopard with writable ZFS.

FreeBSD committed ZFS to the 7.0 tree in April of 2007. There is an excellent article that describes how to install FreeBSD 7.0 with ZFS. The FreeBSD Project also has a wiki page that describes the current state of ZFS under FreeBSD, and has some nice links about ZFS.

So why isn’t ZFS in the Linux kernel tree? Because the license for ZFS, the Sun CDDL, conflicts with the Linux kernel’s GPL license. There was an interesting discussion on the Linux Kernel Mailing List (lkml) summarized at kerneltrap.

One way to avoid the license issues is to run Linux inside a Solaris zone; while the Linux system is not aware of the filesystem used as the backing store for the zone, the Solaris system could use ZFS as the zone’s filesystem.

(Not) Installing OpenSolaris 200805 onto a Compaq nc4010

Solaris is by all accounts a great operating system (I continue to think so) but OpenSolaris 200805 on this laptop does not show any of the excellence that Solaris is supposed to have.

I have tried Solaris x86 in the past, including installing Solaris 2.6 onto an aging 486, and installing Solaris 8 onto several different machines, including laptops. None of these installs have had as many problems as installing OpenSolaris 200805 onto this machine. Installing OpenSolaris 200805 into a VirtualBox virtual machine was slick; not so this system. (I still don’t know why a complete install description is required for virtual environments; it’s just another computer system after all.)

First, I installed OpenSolaris to a physical hard drive using the VirtualBox machine to do so. This worked beautifully. Installed, no problem.

However, booting the installed operating system provided a big problem: apparently the root filesystem definition is buried in the filesystem itself (ZFS) so that booting the disk from anywhere else in the system causes the boot to fail. This is not the problem – the problem is trying to find out how to fix it. With Linux, a kernel parameter and a fix to /etc/fstab is all that is needed.

In searching for the answer to this, there were a number of stumbling blocks – obvious ones – and there seemed to be no one who had answered this problem properly:

• Boot into Failsafe mode and… When I see that, I always wonder what operating system they’re using: OpenSolaris 200805 has no failsafe mode. (Later on, I found out that OpenSolaris 200805 was the first Solaris to not have a failsafe mode…. nice.) This is not helpful, and rules out a majority of the responses right off the bat.
• OpenSolaris 200805 uses ZFS as the root filesystem. This means that a) it is new and not well-tested; and b) most answers to this problem are irrelevant as they are assuming UFS as the root filesystem, not ZFS.

Having had such problems just getting the stupid drive to boot, I gave up: I tried to install directly, using a 3.5″ USB disk caddy with a CD/DVD ROM player in it. The system will boot from this, but the speed was very slow.

The first try resulted in the machine freezing at about 22% done. After rebooting, the system would continually hang right after the initial SunOS boot text. I was able to fix this (after many reboots and freezes) by booting into Linux and overwriting the half-baked install on the internal disk. Thus, the pre-existing data on the internal disk (unused) was enough to cause OpenSolaris to freeze up (I’d used the “entire disk” install option – which presumably wipes the DOS-style partition table clean off the drive).

The second try resulted in a complete install, but that was it. No reboot ever succeeded there after. The system froze first at the “zfs0 is …” text, then at “tz0 is …”, then another one. Trying the option “-B acpi-user-options=0×8″ permitted the machine to boot long enough to shut itself off!

About then is when I decided I’d had enough. Maybe Solaris Express or Belenix will work, but OpenSolaris is extremely poor in this department – which is so disappointing. Did I mention that OpenSolaris does not support JumpStart installs either?

With this sort of track record, I cannot recommend OpenSolaris for laptops – nor for production x86 servers. Sad really – I’d been looking forward to getting OpenSolaris on one of my laptops – very much, as a matter of fact.

Overview: how to install UNIX/Linux to a machine with no bootable disk

Installing operating systems to the HP nc4010 ultralight notebook has been an excercise in how to accomplish the seemingly impossible: installing an operating system to a laptop with no removable disk and no bootable disk.

Generally, there are three different ways to do this:

• Boot from the network using PXE.
• Boot from an external add-on device such as USB CDROM or USB memory device.
• Create a bootable disk in another system and install the disk afterwards.

Booting from the network requires several servers to be set up, including a TFTP server, a NFS server, and a DHCP server. Though they could all be on the same machine, it does represent a significant amount of set up and configuration in order to install, including the need to copy all installation parts to the NFS server to be served up to clients. In addition, there are special configurations needed for DHCP to get this started.

Booting from an external device is much easier, and can be done on the nc4010 and probably can be done on most laptops from the last 10 years or so. This method is probably the easiest to accomplish and without any fuss.

Alternately, it is possible to install the operating system normally in another system and then transfer the disk over to the new system. The biggest problem – the major problem – is that the disk locations all change. What had been /dev/hd1 is now /dev/hd0; all of this will need to be changed in order to have the new system boot properly.

The boot loader may also need to be changed to recognize the new location of the disk.

Linux has a parameter “root=/dev/zzzz” which allows the boot process to specify where the system root disk is. After this, then /etc/fstab will have to be changed (which is standard everywhere).

Solaris has UFS and ZFS, and UFS can be modified to reflect a new source disk location. ZFS is more troublesome and hard to do, as the filesystem is newer and has not been used as a boot drive for hardly any time at all. I still do not have an understanding of how to convert ZFS from using one boot disk to another (in name only) – once that happens, I’ll have OpenSolaris on an nc4010.