Tips and Tidbits About LDAP

Setting up and understanding LDAP is not easy. In my opinion, nothing is obfuscated more and unnecessarily so than LDAP. There are a number of tips that can help you to understand LDAP.

LDAP is not authentication. This was the number one problem I had when I started (a while back). The first time user might search for documents on setting up LDAP when in fact they are looking for documents on how to set up UNIX and Linux authentication using LDAP. An LDAP server at its most basic doesn’t understand UNIX uids, doesn’t understand GECOS fields, doesn’t understand UNIX gids, and doesn’t understand Linux shadow files. Support for all of this must be added.

Support for UNIX authentication must be added. You would think that the most common usage for LDAP would come bundled and ready to go with the server; however, often this is not the case. Even if it is the case, you may find that for certain capabilities you are expected to add new LDIF files to support the fields in LDAP.

LDAP is not just another database server. Virtually everything in LDAP has a different name; it is unlike anything you’ve done before. Take heart: X.500 (where LDAP comes from) was worse. You’ll have to slog through a pile of new terms, but after a while it will become easier to understand.

OpenLDAP is not synonymous with LDAP. There are other servers out there. OpenLDAP does come with virtually every Linux platform; there are however, many others – many of which may be easier to use. There is the 389 Directory Server from Red Hat, the ApacheDS (part of the Apache Directory Project) from Apache, and OpenDJ from ForgeRock. OpenDJ itself is a spin-off from OpenDS, originally from Sun.

OpenLDAP is known for making non-backwards-compatible changes. The most recent example is the complete replacement of the configuration system.

OpenLDAP no longer uses slapd.conf. This will cause you no end of problems: there are a lot of people trying to explain how to set up OpenLDAP, and with a single strike (as of version 2.3) OpenLDAP made all of that documentation obsolete and useless. This is incomprehensible, but it is a fact.

Using and administering LDAP requires command line expertise. This is basically true, but like many things, it is not the complete truth. There are many programs designed to make it easy to browse LDAP stores, along with editing capabilities. Some of the more interesting products include Jxplorer, Luma, and the Apache Directory Studio. Of these, the Apache Directory Studio is the most capable, robust, and actively developed – and by far the largest.

Some LDAP entries can be present more than once or have more than one value. If you are comparing LDAP to a database, then this will come as a surprise. One valuable example is UNIX groups: the original UNIX systems only had one group per user; later, secondary groups were added – thus presenting a single user with multiple groups. This is handled in LDAP in a variety of ways, but they all amount to having multiple entries with different values.

Limiting user logins by host is not available in LDAP. This capability is most likely to be done by using the client host. There are a number of ways to do it, but all require LDAP client configuration, and all are limited in their application. Without client configuration, all LDAP users will have authenticated access to the host.

Be prepared to do a lot of web searches for documentation and solutions. The best places to go for searches are: Google (of course) and Ubuntu Documentation.

There are also very good articles and documents on using LDAP for authentication. There is an article about OpenLDAP authentication on FreeBSD (FreeBSD articles tend to be very well-written). Similarly, Ubuntu documentation is well-written as well; each of the Ubuntu versions has a section in the documentation on using and configuring OpenLDAP for authentication. Ubuntu 11.04 documentation has a good article on OpenLDAP for example.

Ubuntu documentation also includes a lot of well-written (and current) articles. For example, there are articles on OpenLDAP Server (a general article), LDAP Client Authentication, Samba and LDAP (from the 10.04 Server documentation), and Securing OpenLDAP Conenctions. If you plan to use 389 Server instead, there are even a couple of articles on using it with Ubuntu: Fedora Directory Server (the original name of 389 Server) and Fedora Directory Server Client Howto.

A nice overview of LDAP comes from Brian Jones at O’Reilly: specifically, his 2006 articles on Demystifying LDAP and Demystifying LDAP Data. Linux Journal also has myriad different articles on LDAP (not to mention an OpenLDAP theme for the December 2002 issue). Linux Journal also has an article from 2007 on Fedora Directory Server (now 389 Server).

Lastly, an excellent resource is the “book” LDAP for Rocket Scientists from Zytrax.com. You simply must go and read portions of this book. One very apt quote from the introduction to the book which sums up the state of LDAP documentation generally:

The bad news is that [in our humble opinion] never has so much been written so incomprehensibly about a single topic with the possible exceptions of BIND and … and …

(It should be noted that the other book at Zytrax is about DNS. Is it any surprise?)

UPDATE:

Yet another trick to LDAP:

The cn= attribute is not solely a leaf attribute. This can be seen in OpenLDAP’s cn=config tree with OpenLDAP configuration. For example, a typical admin user can be designated like so:

cn=admin,dc=bigcorp,dc=com

However, when you use OpenLDAP’s configuration, the designation for the admin user is this:

cn=admin,cn=config

When you look into the configuration tree, there are more cn= entries – like this:

cn={4}misc,cn=schema,cn=config

/etc/shadow format

Recently, I found myself trying to understand what was going on in the shadow file. For one thing, the “password” field in Ubuntu Lucid Lynx is much longer than it is in Red Hat Enterprise Linux 5. Then there were the exclamation points in front of some password entries.

Originally, there was no /etc/shadow file – the passwords and salts were contained with other user information in /etc/passwd. The problem was that /etc/passwd was readable by anyone, and that attacks became more common and more powerful. It was necessary to move the password into a separate file that would only be readable by root.

Here is an example of a shadow file line:

ddouthitt:$6$QR3drPrQ$JLolPKyiVuXvea1F2IpbPx9F9PEV0s/IGcNCpm6ZrBA6AFDvwHPQG7EsTQHuUqxfCvlsuKRb.O7w5RLPyj8nS/:15119:0:99999:7:::

Each field is separated by a colon (:). There are a number of fields:

1. User name. This must match up with a user name in /etc/passwd; it is the key.
2. Password. This field is a hash of the password (not the actual password itself) combined with an id (specifying which form of encryption was used) and with a salt.
3. Last changed date. This is the day that the password was last set (in days since January 1, 1970).
4. Minimum days. This is the minimum number of days that a password must be active before a password can be changed again. A user’s password cannot be changed by them until this number of days has elapsed.
5. Maximum days. This is the maximum number of days for a password to be valid. Once the maximum number of days has elapsed, the user is forced to change their password.
6. Warn days. Before a password expires, warn the user this many days ahead of time.
7. Inactive days. Once a password has expired, it will be disabled after this many days.
8. Expire date. On this date (in days since January 1, 1970), the account is disabled and the login can no longer be used.

Rather than trying to compute the date from the number of days since 1970, use the chage command:

$ chage -l ddouthitt
Last password change					: May 25, 2011
Password expires					: never
Password inactive					: never
Account expires						: never
Minimum number of days between password change		: 0
Maximum number of days between password change		: 99999
Number of days of warning before password expires	: 7

This also explains what each number means, and incorporates any defaults.

The password is more complicated. The actual password is not stored anywhere; the stored value in the /etc/shadow file is a hash generated from the password.

Originally, the password hash was just the hash itself; later, a two-character salt was added at the beginning.

The password hash today comes in three parts, separated by dollar signs ($):

• Id. This identifies the encryption hash method used. A value of 1 denotes MD5; 2 or 2a is Blowfish; 3 is NT Hash; 5 is SHA-256; and 6 is SHA-512.
• Salt. This is used by the encryption algorithms, and could be up to 16 characters.
• Hash. The actual “password” (or hash) is last. MD5 uses 22 characters, SHA-256 uses 43, and SHA-512 uses 86.

Picking apart the example password entry above shows this:

• Id = 6 (SHA-512)
• Salt = QR3drPrQ
• Hash = JLolPKyiVuXvea1F2IpbPx9F9PEV0s/IGcNCpm6ZrBA6AFDvwHPQG7EsTQHuUqxfCvlsuKRb.O7w5RLPyj8nS/

Lastly, the password entry could be missing entirely (which means no password) or it could have an invalid entry (locked account).

Traditionally, locking was done with the asterisk (*) by convention or by the word LOCKED. However, this has been extended – by convention – to a variety of new suggestions. System logins that never had a password use a double exclamation mark (!!) and accounts that have been locked have the valid password entry invalidated by a exclamation mark prefix. Using the prefix in this way allows the password to be reinstated as it was.

FreeBSD uses a slightly different setup, with a file called /etc/master.passwd, an extended /etc/passwd file with password aging included, and support for an extended DES format (which starts with an underline). Extended DES is more secure than the original DES, but is no longer used in favor of the new modular format described above. FreeBSD standardizes on MD5 – like Red Hat, and unlike Ubuntu.

By the way – that’s not my real password – so don’t look at it.

Upgrading PostgreSQL

This is an old procedure, which is commonly understood by PostgreSQL administrators. However, if you’ve not done it before, over at the FreeBSD Diary there is a nice article describing how to upgrade the client and the server when moving from one version to the next (such as from 8.3.x to 8.4.x).

The basic thing to remember is that a complete dump of the databases is necessary, with the appropriate reload process. The article describes this process well.

Anybody who is interested in FreeBSD should follow the FreeBSD Diary; this “blog” has been around longer than blogs have been.

UNIX and OpenVMS Online Resources

It is possible to get free online access to UNIX or to OpenVMS; these can be useful in building up your experience on a platform when starting from scratch – or when a review is required.

One of the oldest public access systems in the country is the Super Dimension Fortress (or SDF as it is usually called). SDF offers free accounts, but does ask for US$1 to gain standard access. This isn’t because access is expensive, but because too many people have used the facilities for nefarious purposes (the process suggests that the new user is not a person who will strike and leave).

SDF runs NetBSD on DEC Alphas; this was driven mainly by security and stability. Previously, Stephen Jones, the proprietor, ran SDF using Linux on Intel for several years (which he describes as “the dark years”). BSDTalk had an interview with him back in 2006.

You could also try PolarHome – this shell provider provides access to hosts running Linux (Red Hat, Debian, SUSE, Ubuntu, or Mandriva), OpenVMS (Alpha or VAX), OpenBSD, FreeBSD, NetBSD, HPUX, IRIX, QNX, Solaris, Ultrix, AIX, Tru64, and OpenStep. Unfortunately it requires payment for shell accounts – again because of abuse. The payment is 10 units of your local currency or US$2, whichever is more – and this is per host as well. No other site provides this diverse of a selection.

For truly free UNIX shell accounts, one can try Grex, which is a more professionally-run system (Polarhome and SDF are sole proprietorships). Grex offers totally free shell accounts, but also has memberships (for people to help support the site). It is possible that Grex has the most users as well. Like the others, paid membership does have its privileges – but unlike the others, membership is mainly to provide support for Grex, rather as a security feature.

For OpenVMS, there is a very unique online shell provider: Deathrow Cluster. This is a cluster of three machines running OpenVMS 7.3 – one VAX, one Alpha, and one emulated VAX (SIMH) on a dual Xeon machine. This last is a perfect example of what can be done with an emulator, especially with SIMH which can emulate all manner of old Digital and IBM hardware. However, SIMH does not emulate the Digital Alpha, unfortunately. Like Grex, Deathrow provides completely free shell accounts; like SDF and Polarhome, it is (or appears to be) mainly one person’s purpose to keep it running with a lot of volunteer help.

Any of these will be good sources to keep your shell skills sharp – and in some cases, programming as well. They’re also good people to support; why not offer them some donations if you can?

ZFS and Apple’s new MacOS X (Snow Leopard)

Sun’s ZFS is, by all accounts, the most revolutionary file system to come along in years. The Wikipedia entry on ZFS has some details, and Sun has a ZFS Learning Center where you can learn how to use it.

Of course, ZFS is in OpenSolaris, but it is also being introduced into FreeBSD as well.

The Solaris Internals site has a beautiful ZFS Best Practices Guide.

What does all of this have to do with Apple’s MacOS X (Snow Leopard)?

Just this: early in the development of MacOS X 10.6, Apple announced that they would use ZFS in the new MacOS X Snow Leopard. The ability to read ZFS volumes had been put into MacOS X Leopard Server. However, ZFS is missing from MacOS X Snow Leopard and Snow Leopard Server entirely. Robin Harris over at ZDNet has an excellent article that explains it all. He then went on to expand on his ZDNet article with more details.

The one detail in particular I wanted to note is the lawsuit between NetApp and Sun over ZFS and related patents. Groklaw has been following the lawsuit, but the last update from Groklaw is October 2008; Sun has more details on their lawsuit page. Way back in 2007 when the patent lawsuit erupted, CompuerWorld had an article suggesting that Apple might be forced into the lawsuit since it had been courting ZFS – or could be sued next if NetApp won. Neither Apple nor NetApp would comment.

It would also be worth noting that when IBM was in talks to buy Sun in March 2009, there were articles about how the ZFS lawsuit would affect such talks – especially given that IBM and NetApp had a strong partnership already (IBM remarkets NetApp hardware for instance). AMLawDaily had a nice article about it, as did CNET. It wasn’t much more than a month later – in April 2009 – that Sun announced it was being bought by Oracle.

Powered by ScribeFire.

Book review: Learning FreeNAS

I’ve been looking at the book Learning FreeNAS by Gary Sims, and trying out FreeNAS in the process. FreeNAS is now at 0.69.1, and is very stable and robust. FreeNAS is based on FreeBSD and thus is rock solid.

Writing a book about FreeNAS (or any Network Attached Storage system) is difficult for several reasons. The most obvious one is that entire books (big books!) have been written about each storage technology: Windows file-sharing (SMB/CIFS), NFS, iSCSI, FTP, backups, and more.

It is difficult to write a good book about NAS as it is not possible to cover all areas in depth – and alternately, it is not good to reduce the book to “click this button; click that button; next enter this data and click that button…” A NAS can make setting up and using a complicated server quite easy – and finding the right balance between describing all of how Samba works and just specifying which buttons to push can be a hard choice to make.

Learning FreeNAS tends slightly towards the simple end: if you discover any serious problems that require command-line knowledge, the book doesn’t really cover more than it must. In my case, I found that installing FreeNAS resulted in the lack of a default route. I had to add the default network route by hand, though the book never discusses this. This is not necessarily a deficiency, but one to be aware of.

One thing that I always look for in books is an in-depth index. These are simple to find: how many pages does the index contain? How many entries does each letter contain? How many entries can be found under U or X? This book contains 6 pages of index, compared to a similarly sized book that has 17 pages – and a smaller font size. As a reference work then, it will be harder to find items that are of interest.

Overall, this is a good book, worth getting. It could have been more in-depth, but as it stands it is still good. There is no comparable book for the only serious competitor in the open source NAS arena, OpenFiler (which is based on Linux).

The book is available from Packt Publishing in print or in a downloadable PDF.

Network Attached Storage (NAS)

Once you hear what a NAS appliance does, you might be tempted to think (as I did) what all the fuss might be about. But there are reasons for a NAS appliance, though a NAS isn’t for everybody.

Network Attached Storage is nothing more than a server with a pile of disks and a dozen different ways to access them. For most intents and purposes, the difference between a File Server of yesteryear and the Network Attached Storage of today is conceptually rather minimal.

NAS typically provides access to files via such methods as Windows shares, NFS, iSCSI, Appleshare and others.

So what does a NAS appliance provide that a NFS server does not? There are several benefits:

• Special purpose. Since the system is solely for the purpose of serving up files for users, there is no need for any other facilities except those that deal with its specified purpose. Thus, a lot of potentially vulnerable or unreliable code can be removed, and the speed and reliability of the system can be increased. Some systems do not come with a general purpose operating system of any kind, but rather a specially designed operating system for serving files alone.
• Extensive support. In many cases, since the system is specifically designed for serving up file storage, the innumerable variations of network storage protocols come supported out of the box.
• Ease of use. With the system designed to serve one purpose – and to provide the customer with the best possible experience – the system is generally made much easier to configure and easier to use than having to configure the varying servers and protocols independently.

There are two different NAS products that are the heavy-weights in the free and open source arena: FreeNAS (freenas.org) and OpenFiler (openfiler.com).

The most obvious difference between these two is their base (and their associated licenses). The base for FreeNAS is FreeBSD, and like FreeBSD, is licensed using the BSD license. However, OpenFiler uses Linux as its base, and is likewise covered by the General Public License version 2.

This week, I’ll focus on FreeNAS with the assistance of a book entitled Learning FreeNAS by Gary Sims and published by Packt Publishing.

Powered by ScribeFire.

A vulnerability walk-through

The FreeBSD kernel recently had a issue in the kenv(2) kernel call, and this article describes very well what it is – and why it is bad. The vulnerability itself is not terribly bad, but the problem exposed is a common one and shows how all user data must be vetted before it is used: a programmer must treat all user data as suspect.

In fact, there have been studies done by Professor Barton Miller at the University of Wisconsin showing that both commercial and open source programs (in a variety of operating systems) are vulnerable (to differing extents) to a constant barrage of random data.

If your code is to be secure, you absolutely must treat user data as hostile and unknown: any trust placed in the user will be abused by someone, either accidentally or purposefully. If by accident, the user will think your software broken and unreliable; if by purpose, your system (or someone else’s!) could be compromised.

Two excellent books on this topic (from two different angles) are these: Hacking: the Art of Exploitation (by Jon Erickson) and Secure Coding Principles (by Mark Graff and Kenneth van Wyk). The first will show you how broken code can be taken advantage of; the second will show you how not to write broken code.

About ZFS

I’ve known that ZFS was a revolutionary filesystem, but never understood the details. Now there is an article that explains why ZFS is so desirable, and does so well.

Apple started supporting ZFS read-only in Leopard, and has released beta versions of Leopard with writable ZFS.

FreeBSD committed ZFS to the 7.0 tree in April of 2007. There is an excellent article that describes how to install FreeBSD 7.0 with ZFS. The FreeBSD Project also has a wiki page that describes the current state of ZFS under FreeBSD, and has some nice links about ZFS.

So why isn’t ZFS in the Linux kernel tree? Because the license for ZFS, the Sun CDDL, conflicts with the Linux kernel’s GPL license. There was an interesting discussion on the Linux Kernel Mailing List (lkml) summarized at kerneltrap.

One way to avoid the license issues is to run Linux inside a Solaris zone; while the Linux system is not aware of the filesystem used as the backing store for the zone, the Solaris system could use ZFS as the zone’s filesystem.

Why FreeBSD is (and isn’t) My Favorite Operating System

Over at Webmasters by Design there was a very interesting article by Scott Spear about Why FreeBSD is My Favorite *nix OS. Like him, I find FreeBSD to be wonderful and like it a lot. However, I find that I don’t want to use it for everything.

Why I Like BSD: Small Footprint

FreeBSD works in many, many, many more places than Linux – and even more places than Solaris. There just isn’t a lot of overloaded kernel involved. As kernels grow faster and faster, it is refreshing to be able to use something not so bloated.

Why I Like BSD: History

Unlike Linux, BSD goes back a very long ways (longer than Solaris even) and is UNIX. It is possible that with the exception of Unixware and NetBSD, no other UNIX system has as much of a history. Some of the original developers are still involved in FreeBSD (Marshall McKusick comes to mind).

Why I Like BSD: Cohesiveness

No matter how hard they try, a Linux distribution can’t match the overall cohesiveness of one of the BSD systems (such as FreeBSD). Some Linux distributions are very well done, but they still have “missing parts” – usually documentation.

Why I Like BSD: Documentation

There is nothing that matches FreeBSD documentation in the Linux world. Once, I ran a test: I wrote a script to test for the existance of man pages for every binary in the usual locations on the system (/bin /sbin /usr/bin /usr/sbin). Red Hat Linux come up with a number of programs that were undocumented; FreeBSD did not have a single missing man page.

All of the kernel tunables can also be found in man pages, and more.

The FreeBSD Handbook is phenomenal, and a valuable resource. Linux environments don’t have anything like it.

Why I Don’t Like BSD: Linux (In)Compatibility

Linux compatibility fails as often as it succeeds, and it is more of a simulated environment than it is just a compatibility layer. It doesn’t work, it’s bloated and it’s wrong to rely on it in any case.

Why I Don’t Like BSD: Flash et al

Getting to use Flash in FreeBSD is a nightmare. Even following the directions is no guarantee that it will work. Distributions such as OpenSUSE and Ubuntu come ready to plug in Flash support, and Adobe has specified that they will support Linux. That leaves out FreeBSD.

This may be better in PCBSD; I aim to try it soon.

Why I Don’t Like BSD: Installation

Sysinstall is not the easy install process that installation of distributions like Ubuntu and OpenSUSE is. There are way too many technical details to comprehend.

This probably has improved with FreeBSD 7; I’ve not yet tried FreeBSD 7.