Tag: zfs

Native ZFS for Linux Coming (and Other ZFS News)

Two different projects are in full swing developing a native port of the ZFS file system to Linux. Currently, the only available ZFS for Linux is based on FUSE (running file systems in user space, not kernel space). Running with FUSE means a significant penalty in speed.

One ZFS project is being done at Lawrence Livermore National Laboratory and is driven by Brian Behlendorf; the other project is being done by KQ Infotech. OSNews had a good article about the LLNL project.

Phoronix is reporting that KQ Infotech announced they would be releasing source code in September. Unfortunately, the project apparently only supports 64-bit Linux, and does not support data de-duplication and other newer features.

The writer is also unimpressed by ZFS overall; there is mention of BtrFS as being better. What is truly interesting is that both file systems are now are under the Oracle umbrella – BtrFS since the beginning, and ZFS as part of the Sun acquisition.

On top of this, Oracle and NetApp have resolved the lawsuit over ZFS, removing the clouds that had hung over ZFS.

These are interesting times for ZFS. If you don’t understand what the fuss is all about, Paul Rubens has a quick overview, and Sun’s ZFS team has an excellent slide presentation on ZFS. There are more ZFS resources available from the OpenSolaris ZFS Community.

Whither Sun Microsystems?

The recent fourth quarter reports from server manufacturers was dim, and Sun Microsystems was by far the worst (with a 35% loss compared to the same period last year). On top of this, Sun just announced in October (within their 8K filing for the SEC) intentions to lay off 3000 employees in the next 12 months. Infoworld also had a nice piece on this; according to Larry Ellison, CEO of Oracle, the company is losing $100 million each month the European Union regulators put off accepting the merger.

With the Oracle acquisition in progress, there are a lot of questions about the future viability of Sun Microsystems, and of some of its products.

I don’t think people realize just how important the Sun group of products are, and what an impact it would have if most – or even some – of the products were cancelled. Consider this list of Sun products:

Most of the most popular products were mentioned by Oracle in their Sun Acquisition FAQ (PDF), stating that they will increase money spent on each over what Sun spent. These products include: Java, Solaris, SPARC, StarOffice, NetBeans, virtualization products, Glassfish, and MySQL. Other products were not mentioned – such as Lustre, the Modular Data Center, and others.

The list above also does not list the technologies that were spearheaded by Sun – and some still are: ZFS, NFS, NIS (and NIS+), dtrace, containers, and smc.

It would be unfortunate – and materially significant – if Sun were to go under or if any of the majority of their products were to be cancelled. One can only hope this does not happen…

ZFS and Apple’s new MacOS X (Snow Leopard)

Sun’s ZFS is, by all accounts, the most revolutionary file system to come along in years. The Wikipedia entry on ZFS has some details, and Sun has a ZFS Learning Center where you can learn how to use it.

Of course, ZFS is in OpenSolaris, but it is also being introduced into FreeBSD as well.

The Solaris Internals site has a beautiful ZFS Best Practices Guide.

What does all of this have to do with Apple’s MacOS X (Snow Leopard)?

Just this: early in the development of MacOS X 10.6, Apple announced that they would use ZFS in the new MacOS X Snow Leopard. The ability to read ZFS volumes had been put into MacOS X Leopard Server. However, ZFS is missing from MacOS X Snow Leopard and Snow Leopard Server entirely. Robin Harris over at ZDNet has an excellent article that explains it all. He then went on to expand on his ZDNet article with more details.

The one detail in particular I wanted to note is the lawsuit between NetApp and Sun over ZFS and related patents. Groklaw has been following the lawsuit, but the last update from Groklaw is October 2008; Sun has more details on their lawsuit page. Way back in 2007 when the patent lawsuit erupted, CompuerWorld had an article suggesting that Apple might be forced into the lawsuit since it had been courting ZFS – or could be sued next if NetApp won. Neither Apple nor NetApp would comment.

It would also be worth noting that when IBM was in talks to buy Sun in March 2009, there were articles about how the ZFS lawsuit would affect such talks – especially given that IBM and NetApp had a strong partnership already (IBM remarkets NetApp hardware for instance). AMLawDaily had a nice article about it, as did CNET. It wasn’t much more than a month later – in April 2009 – that Sun announced it was being bought by Oracle.

Powered by ScribeFire.

About ZFS

I’ve known that ZFS was a revolutionary filesystem, but never understood the details. Now there is an article that explains why ZFS is so desirable, and does so well.

Apple started supporting ZFS read-only in Leopard, and has released beta versions of Leopard with writable ZFS.

FreeBSD committed ZFS to the 7.0 tree in April of 2007. There is an excellent article that describes how to install FreeBSD 7.0 with ZFS. The FreeBSD Project also has a wiki page that describes the current state of ZFS under FreeBSD, and has some nice links about ZFS.

So why isn’t ZFS in the Linux kernel tree? Because the license for ZFS, the Sun CDDL, conflicts with the Linux kernel’s GPL license. There was an interesting discussion on the Linux Kernel Mailing List (lkml) summarized at kerneltrap.

One way to avoid the license issues is to run Linux inside a Solaris zone; while the Linux system is not aware of the filesystem used as the backing store for the zone, the Solaris system could use ZFS as the zone’s filesystem.

(Not) Installing OpenSolaris 200805 onto a Compaq nc4010

Solaris is by all accounts a great operating system (I continue to think so) but OpenSolaris 200805 on this laptop does not show any of the excellence that Solaris is supposed to have.

I have tried Solaris x86 in the past, including installing Solaris 2.6 onto an aging 486, and installing Solaris 8 onto several different machines, including laptops. None of these installs have had as many problems as installing OpenSolaris 200805 onto this machine. Installing OpenSolaris 200805 into a VirtualBox virtual machine was slick; not so this system. (I still don’t know why a complete install description is required for virtual environments; it’s just another computer system after all.)

First, I installed OpenSolaris to a physical hard drive using the VirtualBox machine to do so. This worked beautifully. Installed, no problem.

However, booting the installed operating system provided a big problem: apparently the root filesystem definition is buried in the filesystem itself (ZFS) so that booting the disk from anywhere else in the system causes the boot to fail. This is not the problem – the problem is trying to find out how to fix it. With Linux, a kernel parameter and a fix to /etc/fstab is all that is needed.

In searching for the answer to this, there were a number of stumbling blocks – obvious ones – and there seemed to be no one who had answered this problem properly:

  • Boot into Failsafe mode and… When I see that, I always wonder what operating system they’re using: OpenSolaris 200805 has no failsafe mode. (Later on, I found out that OpenSolaris 200805 was the first Solaris to not have a failsafe mode…. nice.) This is not helpful, and rules out a majority of the responses right off the bat.
  • OpenSolaris 200805 uses ZFS as the root filesystem. This means that a) it is new and not well-tested; and b) most answers to this problem are irrelevant as they are assuming UFS as the root filesystem, not ZFS.

Having had such problems just getting the stupid drive to boot, I gave up: I tried to install directly, using a 3.5″ USB disk caddy with a CD/DVD ROM player in it. The system will boot from this, but the speed was very slow.

The first try resulted in the machine freezing at about 22% done. After rebooting, the system would continually hang right after the initial SunOS boot text. I was able to fix this (after many reboots and freezes) by booting into Linux and overwriting the half-baked install on the internal disk. Thus, the pre-existing data on the internal disk (unused) was enough to cause OpenSolaris to freeze up (I’d used the “entire disk” install option – which presumably wipes the DOS-style partition table clean off the drive).

The second try resulted in a complete install, but that was it. No reboot ever succeeded there after. The system froze first at the “zfs0 is …” text, then at “tz0 is …”, then another one. Trying the option “-B acpi-user-options=0x8” permitted the machine to boot long enough to shut itself off!

About then is when I decided I’d had enough. Maybe Solaris Express or Belenix will work, but OpenSolaris is extremely poor in this department – which is so disappointing. Did I mention that OpenSolaris does not support JumpStart installs either?

With this sort of track record, I cannot recommend OpenSolaris for laptops – nor for production x86 servers. Sad really – I’d been looking forward to getting OpenSolaris on one of my laptops – very much, as a matter of fact.

Overview: how to install UNIX/Linux to a machine with no bootable disk

Installing operating systems to the HP nc4010 ultralight notebook has been an excercise in how to accomplish the seemingly impossible: installing an operating system to a laptop with no removable disk and no bootable disk.

Generally, there are three different ways to do this:

  • Boot from the network using PXE.
  • Boot from an external add-on device such as USB CDROM or USB memory device.
  • Create a bootable disk in another system and install the disk afterwards.

Booting from the network requires several servers to be set up, including a TFTP server, a NFS server, and a DHCP server. Though they could all be on the same machine, it does represent a significant amount of set up and configuration in order to install, including the need to copy all installation parts to the NFS server to be served up to clients. In addition, there are special configurations needed for DHCP to get this started.

Booting from an external device is much easier, and can be done on the nc4010 and probably can be done on most laptops from the last 10 years or so. This method is probably the easiest to accomplish and without any fuss.

Alternately, it is possible to install the operating system normally in another system and then transfer the disk over to the new system. The biggest problem – the major problem – is that the disk locations all change. What had been /dev/hd1 is now /dev/hd0; all of this will need to be changed in order to have the new system boot properly.

The boot loader may also need to be changed to recognize the new location of the disk.

Linux has a parameter “root=/dev/zzzz” which allows the boot process to specify where the system root disk is. After this, then /etc/fstab will have to be changed (which is standard everywhere).

Solaris has UFS and ZFS, and UFS can be modified to reflect a new source disk location. ZFS is more troublesome and hard to do, as the filesystem is newer and has not been used as a boot drive for hardly any time at all. I still do not have an understanding of how to convert ZFS from using one boot disk to another (in name only) – once that happens, I’ll have OpenSolaris on an nc4010.