Moving a VMware ESXi 4.0 Guest From One Host to Another

Tags

ESXI, esxi 4, vmware

To move an ESXi 4.0 guest is not all that hard – but you must be aware of several things along the way. Taken one step at a time, it won’t be difficult. In this discussion, we assume that you are moving from one ESXi 4.0 host to another – both with the same architecture. (Anything other than that gets much more complicated.)

First, make sure there are no snapshots. Snapshots are not compatible with this process and must be eliminated altogether.

Then, shut down the guest system. We don’t want the guest changing as it is copied across.

The next step is to copy the guest files from the original host to the destination host. This is the longest step considering you probably have gigabytes of data to transfer. This is also done from the ESXi command line.

I would normally use rsync but it doesn’t exist on an ESXi 4.0 system; use scp to copy the files. Your files for the guest should be located in /vmfs/volumes/datastoreX/guest/ where datastoreX is the data store containing the guest and guest is the name of the guest host. If you renamed the host in one of the GUIs such as VSphere Client, then this will reflect the original name.

Make a directory in the remote host (using the ESXi command line interface) in one of the data stores, and then use commands like these from the original host:

cd /vmfs/volumes/datastoreX/guest
scp * remotehost:/vmfs/volumes/datastoreY/guest/

This will copy the files to the remote host.

However, copying is not enough. Log into the remote host and go to the place you copied the files to. Check over the file ending in .vmx for any references to disks that must be changed. Convert remote host paths to local disk paths – you will probably need to know the long hexadecimal path for any paths, so list that before you start editing. If you execute a cd command to the directory containing the guest host, the long path will be in the prompt.

Next you must register the host so the system knows about it. Use this command at the command line to do this:

vim-cmd solo/registervm /vmfs/volumes/datastoreY/guest/guest.vmx

Now, to get the host started: start the host from VSphere Client. The client will give you a question to answer about where the guest came from. Click on the guest’s Summary tab and select I copied it. (which should be the default) and click Ok.

The guest will start up – and discover that the MAC address of its network interface has changed. For Linux, this means a new ethernet interface, and the configuration of the old interface is ignored: that means that there will be no network connectivity. Enter the console and change the old configuration from eth0 to eth1 (or whatever is appropriate; find out with ifconfig -a). This change varies by which Linux distribution you use; for Ubuntu, the configuration is in /etc/network/interfaces.

While you don’t have to reboot, it doesn’t hurt to do so after this change – and it tests the system in a clean reboot. The system should start up cleanly.

Now don’t forget to remove the original. Using the VSphere Client, right-click on the host and select Delete from Disk. This will remove the guest host entirely from the system and delete all of its files. If you want to retain the files, instead select Remove from Inventory which essentially unregisters the host, so that the system is not managing it – but the disk files remain.

Using Browsers with OpenLDAP cn=config Tree

Tags

apache directory studio, ApacheDS, cn=config, OpenLDAP

The new OpenLDAP configuration is an LDAP tree like any other, and can be browsed as such. Using a graphical browser like Apache Directory Studio can simplify configuration (somewhat).

When setting up a new LDAP connection, specify the user as the admin user for the configuration tree – like so:

cn=admin,cn=config

Also specify the tree root as:

cn=config

With the appropriate settings, you should then get a list of the configuration tree, where you can view and edit entries. It would behoove you to be very careful about changing things in this tree: if you change the wrong thing, your LDAP server could stop functioning properly.

OpenLDAP with SSL in Ubuntu Lucid Lynx

Tags

gnutls, libgcrypt11, libnettle, slapd

In researching configuration tasks for OpenLDAP, I found this article about using sudo with OpenLDAP. As I am going to implement SSL with OpenLDAP, problems with sudo and SSL could be fatal, so I decided to investigate further.

It turns out that the problem is embedded in GnuTLS, a GPL-licensed OpenSSL replacement. GnuTLS was used by Debian because of a licensing conflict between OpenSSL and OpenLDAP. A backend library used by GnuTLS (libgcrypt11) causes problems based on the way it is initialized and the way it handles the dropping of privilege (that is, it gets rid of its “root” access). This shows up as suid applications failing when run against LDAP users; Ubuntu bugs 926350 (GnuTLS) and 423252 (sudo) are this exact problem.

GnuTLS is replacing libgcrypt11 with the nettle backend as of 2.11.x, but Ubuntu Lucid Lynx continues to use GnuTLS with the original (flawed) backend. The “fix” espoused by some is to use nscd – but this is acknowledged to be a workaround.

There is a GnuTLS version compiled not against libgcrypt11 but libnettle which should the problem. I did not test this PPA; if you wish to stick with OpenLDAP and GnuTLS this might be the way to go.

However, there is also a version of OpenLDAP compiled against OpenSSL. Add the PPA to your APT configuration and then perform a system update (apt-get update && apt-get upgrade). This will upgrade your OpenLDAP and cause it to stop at least temporarily; thus, make sure you allow for some server downtime.

This version of OpenLDAP works except for a few initial problems that must be overcome. When you first install, it may refuse to run.

First, it seems to add an openldap user and group – and while this is good, it does not completely change appropriate files to give access to the openldap user. There are two locations that need to be fixed:

• /var/lib/ldap (the location of the LDAP data store)
• /etc/ssl (the location of the SSL certificates)

Fixing the first is simple:

chown -R openldap:openldap /var/lib/ldap

The second is not as straight-forward; in my case, I added the user openldap to the group ssl-cert which has access into the /etc/ssl directory and subdirectories. Use the vigr command to make this happen: add openldap to the end of the ssl-cert group line (your group id might be different):

ssl-cert:x:108:openldap

Note that if the SSL certificates aren’t set up right, then running the new OpenLDAP will not work – even if LDAPS is not enabled at startup. (There is a fantastic message showing how to make sure your certificates match from OpenVPN.net.) You also need to make sure that the certificates are not expired; it is reported that OpenLDAP will also fail to start with expired certificates.

As the final step, change the /etc/default/slapd file to start LDAPS:

SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///"

Eventually, the best thing to do is to remove LDAP support entirely (and use LDAPS completely):

SLAPD_SERVICES="ldapi:/// ldaps:///"

Tips and Tidbits About LDAP

Tags

Authentication

Setting up and understanding LDAP is not easy. In my opinion, nothing is obfuscated more and unnecessarily so than LDAP. There are a number of tips that can help you to understand LDAP.

LDAP is not authentication. This was the number one problem I had when I started (a while back). The first time user might search for documents on setting up LDAP when in fact they are looking for documents on how to set up UNIX and Linux authentication using LDAP. An LDAP server at its most basic doesn’t understand UNIX uids, doesn’t understand GECOS fields, doesn’t understand UNIX gids, and doesn’t understand Linux shadow files. Support for all of this must be added.

Support for UNIX authentication must be added. You would think that the most common usage for LDAP would come bundled and ready to go with the server; however, often this is not the case. Even if it is the case, you may find that for certain capabilities you are expected to add new LDIF files to support the fields in LDAP.

LDAP is not just another database server. Virtually everything in LDAP has a different name; it is unlike anything you’ve done before. Take heart: X.500 (where LDAP comes from) was worse. You’ll have to slog through a pile of new terms, but after a while it will become easier to understand.

OpenLDAP is not synonymous with LDAP. There are other servers out there. OpenLDAP does come with virtually every Linux platform; there are however, many others – many of which may be easier to use. There is the 389 Directory Server from Red Hat, the ApacheDS (part of the Apache Directory Project) from Apache, and OpenDJ from ForgeRock. OpenDJ itself is a spin-off from OpenDS, originally from Sun.

OpenLDAP is known for making non-backwards-compatible changes. The most recent example is the complete replacement of the configuration system.

OpenLDAP no longer uses slapd.conf. This will cause you no end of problems: there are a lot of people trying to explain how to set up OpenLDAP, and with a single strike (as of version 2.3) OpenLDAP made all of that documentation obsolete and useless. This is incomprehensible, but it is a fact.

Using and administering LDAP requires command line expertise. This is basically true, but like many things, it is not the complete truth. There are many programs designed to make it easy to browse LDAP stores, along with editing capabilities. Some of the more interesting products include Jxplorer, Luma, and the Apache Directory Studio. Of these, the Apache Directory Studio is the most capable, robust, and actively developed – and by far the largest.

Some LDAP entries can be present more than once or have more than one value. If you are comparing LDAP to a database, then this will come as a surprise. One valuable example is UNIX groups: the original UNIX systems only had one group per user; later, secondary groups were added – thus presenting a single user with multiple groups. This is handled in LDAP in a variety of ways, but they all amount to having multiple entries with different values.

Limiting user logins by host is not available in LDAP. This capability is most likely to be done by using the client host. There are a number of ways to do it, but all require LDAP client configuration, and all are limited in their application. Without client configuration, all LDAP users will have authenticated access to the host.

Be prepared to do a lot of web searches for documentation and solutions. The best places to go for searches are: Google (of course) and Ubuntu Documentation.

There are also very good articles and documents on using LDAP for authentication. There is an article about OpenLDAP authentication on FreeBSD (FreeBSD articles tend to be very well-written). Similarly, Ubuntu documentation is well-written as well; each of the Ubuntu versions has a section in the documentation on using and configuring OpenLDAP for authentication. Ubuntu 11.04 documentation has a good article on OpenLDAP for example.

Ubuntu documentation also includes a lot of well-written (and current) articles. For example, there are articles on OpenLDAP Server (a general article), LDAP Client Authentication, Samba and LDAP (from the 10.04 Server documentation), and Securing OpenLDAP Conenctions. If you plan to use 389 Server instead, there are even a couple of articles on using it with Ubuntu: Fedora Directory Server (the original name of 389 Server) and Fedora Directory Server Client Howto.

A nice overview of LDAP comes from Brian Jones at O’Reilly: specifically, his 2006 articles on Demystifying LDAP and Demystifying LDAP Data. Linux Journal also has myriad different articles on LDAP (not to mention an OpenLDAP theme for the December 2002 issue). Linux Journal also has an article from 2007 on Fedora Directory Server (now 389 Server).

Lastly, an excellent resource is the “book” LDAP for Rocket Scientists from Zytrax.com. You simply must go and read portions of this book. One very apt quote from the introduction to the book which sums up the state of LDAP documentation generally:

The bad news is that [in our humble opinion] never has so much been written so incomprehensibly about a single topic with the possible exceptions of BIND and … and …

(It should be noted that the other book at Zytrax is about DNS. Is it any surprise?)

UPDATE:

Yet another trick to LDAP:

The cn= attribute is not solely a leaf attribute. This can be seen in OpenLDAP’s cn=config tree with OpenLDAP configuration. For example, a typical admin user can be designated like so:

cn=admin,dc=bigcorp,dc=com

However, when you use OpenLDAP’s configuration, the designation for the admin user is this:

cn=admin,cn=config

When you look into the configuration tree, there are more cn= entries – like this:

cn={4}misc,cn=schema,cn=config

Upgrading from MySQL 5.1 to 5.5 on Ubuntu Lucid Lynx

Tags

upgrades

I decided to try Percona’s MySQL server, and went for the most recent version (5.5). Unfortunately, the version of MySQL included with Ubuntu Lucid Lynx is 5.1 – so installing Percona’s 5.5 server means an upgrade.

This upgrade requires some additional steps beyond the usual.

First, the new MySQL 5.5 makes InnoDB the default storage engine – so anything that relies on that default will have to be changed. To revert this “change” you may need to add the following to your my.cnf:

[mysqld]
default-storage-engine=MyISAM


Secondly, the mysql database will need updating. Run this command:

# su - mysql -c "mysqld_safe --init-file=/usr/share/mysql/mysql_system_tables.sql --verbose"

This runs the command as the mysql user; if you are using Ubuntu, the mysql user has a shell of /bin/false which causes this to fail. You can either give the mysql user a new shell (such as /bin/bash) by editing with the command vipw or you can run the mysql_safe command as root, like this:

# mysqld_safe --init-file=/usr/share/mysql/mysql_system_tables.sql --verbose

Once this command runs, make sure that the MySQL instance stops by running service mysql stop or mysqladmin shutdown.

This should fix things. One thing to watch out for is the innodb_buffer_pool_size – this is increased from a default of 128Mb to 132Mb; if you’ve tight memory you’ll want to plan for this.

Percona has a complete guide for migrating from 5.1 to 5.5; MySQL also has a thorough description of upgrading to 5.5. MariaDB 5.5 is not out yet, but there is instructions on upgrading to 5.3 (equivalent to MySQL 5.1).

Using Meta-packages to Standardize Servers

Tags

meta-packages, packages

Both Ubuntu and Red Hat offer meta-packages which have no files of their own, but depend on others – thus requiring a set of packages to be present. You can use these packages to require a set of software to be present on a server, especially those that are not normally installed by the vendor’s install process.

A meta-package can save you time – you don’t have to install each package one at a time. A meta-package can also be included as part of a Puppet environment, so that all servers will be kept up-to-date with the current set of packages. A meta-package can also be a part of an automatic install process, also bringing in all necessary software and simplifying the installation steps.

In Ubuntu, creating your own meta-packages is made easy by using the equivs package. With RPM, you’ll have to create a meta-package using rpm-build and the appropriate SPEC file.

The only way that using a meta-package will truly save time is if you are using a program like APT or YUM to do installations because they will automatically compute the dependencies required and install them automatically when the meta-package is installed.

With a meta-package, you can require a set of packages that should be on every server – as well as force some packages to be removed. You can create a meta-packages that includes all packages that should be on a server, but aren’t usually installed (like gawk, logrotate, sysstate, ntp, logwatch, make, and m4 for instance). When the package server-main is then installed, all of its dependencies will also be installed. Any packages that are listed as conflicting packages will also be removed: packages like unattended-upgrades and command-not-found for instance.

Meta-packages could be created for packages that are from the Ubuntu Main repository, and for those packages that are in the Ubuntu Universe repository. This makes it simple to only include software from the Main repository and preclude those packages that are from the Universe repository.

These meta-packages could then be added to a local repository and added to a system during installation; this simplifies the package installation part of the install process and allows you to update any currently installed systems simply.

As an example, here’s my list for requirements (put into server-main) from the Ubuntu Main Repository:

• lvm2
• byobu
• ruby
• vim
• snmpd
• snmp
• mlocate
• postfix
• ltrace
• strace
• wget
• ntp
• m4
• make
• ifenslave
• dnsutils
• procps
• sysstat
• logrotate
• logwatch
• sharutils
• pdksh
• dc
• bsd-mailx
• nut
• finger
• xfsdump
• xfsprogs

And from the Universe repository, these are my suggested requirements (used as dependencies for server-universe):

• iperf
• jwhois
• apt-file
• chkconfig
• atop
• dstat
• maatkit

Log Rotation for MySQL using logrotate

Tags

logrotate, mysql, mysql server, mysqld

The logrotate utility is a powerful and underrated utility used to rotate logs. It is one of Red Hat’s lesser known utilities; even so, it is available for a number of platforms, including Ubuntu.

However, its set up for MySQL is missing on Red Hat and incomplete on Ubuntu.

For Ubuntu, there is no rotation of the slow query logs. To rotate these logs, just add them to the standard Ubuntu logrotate file for MySQL – that is, /etc/logrotate.d/mysql-server. Add the logs to rotate to the beginning of the file, adding to the list of files already present there.

For Red Hat, a complete MySQL log rotation file is needed as there is none at all. The MySQL logrotation script was removed as part of a security update to Fedora Core 4 back on 17 May 2006, and later removed from Red Hat Enterprise Linux 4 Update 4. The reasoning was detailed in Bug #180639 (not available?) and Bug #182025. Since then, this missing logrotate file has been the subject of several bugs (such as Bug #547007) and also of message threads like this one from Red Hat’s rhelv5-list in July of 2007.

The response to all these queries is that the MySQL logrotate script is broken and it’s up to MySQL to fix it. However, this does not seem to take into account the new FLUSH LOGS command, and admins everywhere are creating their own scripts.

Over at Question Defense, Alex has a fabulous description of the entire process – from enabling logging through implementing log rotation. However, this process uses ~/.my.cnf to automatically log in; better is to use a file like /etc/mysql/maint.cnf the way that Debian (and Ubuntu) does it. In that case, Debian creates a special user and a password to go with it, and puts these into a file /detc/mysql/debian.cnf; here is a sample debian.cnf:

# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host     = localhost
user     = debian-sys-maint
password = i5Px6N4SZ9UhfSWa
socket   = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host     = localhost
user     = debian-sys-maint
password = i5Px6N4SZ9UhfSWa
socket   = /var/run/mysqld/mysqld.sock
basedir  = /usr

For most purposes, only the [client] section is needed, along with the first three entries (host, user, and password). You could also specify the section as [mysqladmin] instead, which would limit the username and password to being used for mysqladmin only – which is the tool used during log rotation.

The critical command is this one:

/usr/bin/mysqladmin --defaults-file=/etc/mysql/logrotate.cnf

…where logrotate.cnf contains username and password details as described above. All the rest of the logrotate file is settings and script-bulletproofing:

# Modified Ubuntu logrotate script for MySQL server
#
# Untested under Red Hat, but should work: filenames will have to be changed
/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log {
        daily
        rotate 7
        missingok
        create 640 mysql adm
        compress
        sharedscripts
        postrotate
                test -x /usr/bin/mysqladmin || exit 0
                MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/logrotate.cnf"
                if [ -z "`$MYADMIN ping 2>/dev/null`" ]; then
                  if killall -q -s0 -umysql mysqld; then
                    exit 1
                  fi 
                else
                  $MYADMIN flush-logs
                fi
        endscript
}

The Metapackage Problem and apt-get autoremove

Tags

apt, apt-get, aptitude, autoremove, equivs, metapackages, packages

Seems that this is a common problem among people using APT. During an apt-get run, it might tell you that there are a number of programs that can be removed using apt-get autoremove. Users that are too trusting may take these recommendations at face value and thus wind up removing too much – like all of GNOME and X for instance.

First, a little description about metapackages and package dependencies. A metapackage is one that only exists to provide dependencies: a metapackage requires a set of other packages, usually to provide a collective set of packages (such as a GNOME Desktop, a KDE Desktop, or other things). When you install a package like gnome-desktop-environment, it installs everything you need to have a complete set (in this case, a complete GNOME Desktop). This is expected behaviour.

The other thing to understand is the concept of an automatically installed package versus a manually installed package. When you manually install a package – that is, installed it explicitly by name – the package is considered a package that you, the user, wanted on the system. An automatically installed package is only there because a manually installed package required it. If you wanted one of the automatically installed packages, you would have installed it explicitly – right?

The problem comes when these two processes collide: when you install a metapackage, you are really saying that you want all of the dependencies installed, even though all of the dependencies are marked as automatically installed. A metapackage is an easy way of saying that you want all of these packages (listed as dependencies) installed without explicitly saying so. However, the system does not know this.

This problem then manifests itself when you remove one of the dependencies.

Let’s continue with this example: having installed the gnome-desktop-environment package, let’s say you wanted to remove GNOME Evolution – the evolution package. APT will warn you that the gnome-desktop-environment package will also be removed. It is at this point that you should pause and seriously consider the ramifications of what is about to happen – but we’ll continue onwards.

Once you have removed Evolution – and the “GNOME Desktop Environment” metapackage – there are a lot of automatically installed packages that are not required by any packages on the system. What does this mean exactly? Normally, an automatically installed package is not one that you wanted to have installed but it was required by something you did want. However, in this case, these automatically installed packages (such as vino, evince, and totem for example) are in actuality software that you want.

If you try to remove packages through the use of the apt-get autoremove command, you will see a list of packages that are marked as automatically installed and that are not needed by any package currently loaded. In our example, this is a long list of packages that you actually want to keep!

If you have already removed these packages…

There are a few ways to fix this problem once it has occurred. One is to mark all packages in the system as manually installed:

aptitude keep-all

This marks everything in the system as a manually installed package. This defeats the “autoremove” process entirely and may cause your system to contain unnecessary packages over time.

Another option is to use tools to mark packages as manually installed. There are a variety of ways to do this. One way is to actually try to install the package (again): APT recognizes this and just flags the already installed package as manually installed. Another way is to use the apt-mark utility or to use the aptitude unmarkauto command to change the marking on the package (such that the system labels it as manually installed).

However, the best way is to reinstall the metapackage: this will reinstall all of the bits that are needed for operation. If necessary, using the --reinstall option:

apt-get install --reinstall gnome-desktop-environment

This command can be done from the command line and can be done without X or other graphical environment. It can also be done from the rescue shell – and possibly without networking if you have never run apt-get clean.

If you are determined enough (and knowledgeable enough) you can get around this problem by building your own metapackages with the software you desire. Building a metapackage is simple enough and is made trivial through the use of the equivs package. However, if you want a package like gnome-desktop-environment but don’t want specific packages, a better idea might be to get the source to the package and rebuild it with just the software desired listed as dependencies.

You could also “fake it” by using the equivs tools to generate a metapackage that fulfils the role of the package or packages you wish to remove. This is not recommended, however.

An excellent article about using equivs was written in the Ubuntu Forums way back in March of 2008 by “epimeteo” from Portugal.

Even with the capabilities of the equivs package and other metapackages, the best thing to do is to keep the normal metapackage: this allows you to keep the system updated with current packages, prevents future surprises, and saves a lot of work.

Using the Message of the Day (MOTD) in Ubuntu Linux

Tags

motd, update-motd

The message of the day file (/etc/motd) used to be so simple… but with great power often comes increased complexity. The Ubuntu motd file is very powerful and can be manipulated easily once you know how.

The actual file is composed in /var/run/motd so that the /etc file system can still be considered read only and the file /etc/motd is a symbolic link to /var/run/motd. To make the motd file a static file as before, just change the link to another file such as /etc/motd.static.

In older versions of Ubuntu, the file /etc/motd.tail was introduced for this purpose; however, it has been deprecated for some time in favor of the newer /etc/update-motd.d framework. The /etc/motd.tail file can still be used and is adapted into the new framework.

The update-motd framework was introduced with the update-motd package in Ubuntu Intrepid; it now resides in the libpam-modules package as part of the pam_motd module.

In earlier versions of update-motd, the /etc/update-motd.d directory contained multiple directories that would be executed hourly, daily, weekly, or monthly. The scripts would be executed by a daemon update-motd that was run from cron.

In Ubuntu Lucid, this structure was phased out in favor of a straight sequential operation. The motd file is now generated by the pam_motd module upon login.

Each of the files in the /etc/update-motd.d is executed in numerical order; those in a Ubuntu Lucid Server install are:

/etc/update-motd.d# ls -l
total 28
-rwxr-xr-x 1 root root  57 2010-04-23 04:45 00-header
-rwxr-xr-x 1 root root 248 2010-04-23 04:45 10-help-text
-rwxr-xr-x 1 root root  65 2010-04-13 15:45 20-cpu-checker
lrwxrwxrwx 1 root root  46 2010-12-02 12:26 50-landscape-sysinfo -> /usr/share/landscape/landscape-sysinfo.wrapper
-rwxr-xr-x 1 root root  71 2010-04-13 15:45 90-updates-available
-rwxr-xr-x 1 root root  61 2010-06-30 09:01 91-release-upgrade
-rwxr-xr-x 1 root root  69 2010-04-13 15:45 98-reboot-required
-rwxr-xr-x 1 root root 261 2010-04-23 04:45 99-footer

These files generate a motd like this:

Linux myserver 2.6.32-32-server #62-Ubuntu SMP Wed Apr 20 22:07:43 UTC 2011 x86_64 GNU/Linux
Ubuntu 10.04.2 LTS

Welcome to the Ubuntu Server!
 * Documentation:  http://www.ubuntu.com/server/doc

  System information as of Tue Nov 22 11:01:48 CST 2011

  System load:  1.84                Processes:           167
  Usage of /:   80.6% of 115.84GB   Users logged in:     0
  Memory usage: 50%                 IP address for lo:   127.0.0.1
  Swap usage:   10%                 IP address for eth0: 10.6.7.1

  Graph this data and manage this system at https://landscape.canonical.com/

21 packages can be updated.
0 updates are security updates.

*** System restart required ***
Last login: Fri Nov 18 09:46:33 2011 from 192.168.15.1

SSH introduces some slight complexity: SSH has its own handling of the motd file and can produce it upon demand. In a Ubuntu Lucid install, this capability is turned off because the PAM framework shows the motd through the pam_motd module on login.

Despite the documentation for motd.tail stating otherwise, there is no /etc/init.d/bootmisc.sh that runs and creates /var/run/motd; this is done upon login as previously mentioned.

If logging in with SSH, the SSH display of motd may be turned off either by adding a file ~/.hushlogin or by reconfiguring the server with a PrintMotd No option. Note that this is separate from the pam_motd process; thus if a Ubuntu system has PrintMotd Yes as well as pam_motd you will see double messages. Likewise, if a ~/.hushlogin file is present – or the option PrintMotd No is set – the pam_motd module will still run and a message of the day will still be seen.

In the PAM configuration of a standard Ubuntu Lucid install, both /etc/pam.d/login and /etc/pam.d/ssh contain the pammotd module. The /etc/motd file is selectable here; a different file entirely could be used by modifying the pammotd line appropriately:

session optional pam_motd.so motd=/etc/motd.other

To modify the motd within the update-motd framework, just create (or delete) scripts in the /etc/update-motd.d directory. Each script must be in the format NN-xxxxxxx where the first two numbers NN specify the order and the last xxxxxxx specifies a reasonable name for the script. The script should be an actual file and not a link, according to the documentation (although the 50-landscape-sysinfo script is a symbolic link!).

Output should be to stdout and start with a blank line and end with a newline. Remember that the script will execute on every login, so it should be quick.

Rescuing an Interrupted Ubuntu Upgrade

Tags

recovery, upgrades, upgrading

Upgrading Ubuntu from one version to the next has always been amazingly reliable; yet, sometimes, things happen. How do you recover when your upgrade is interrupted and you can’t boot the system in the usual way?

First step is to boot the machine to a shell that you can use to recover. This can be done in the standard system by using the Recovery Mode. Press the left shift key as the system boots up and choose a recovery mode kernel from the menu that appears – it will be labeled with “(recovery mode)” on the end. The recovery mode kernel will present you with the option to use a recovery shell.

You could use the rescue mode by booting with an Ubuntu install CDROM and entering “rescue” at the boot prompt, but then you must remember to mount all of your partitions and to perform a chroot to /target where the root partition is mounted.

After the system has booted to the menu, select the item that says “Drop to root shell prompt with networking”. This provides you with the ability to update the software packages from the Internet.

When the prompt appears, start by making sure that all currently installed programs are configured:

dpkg --configure -a

This may take a long time, depending on how many programs were installed before the system stopped. Then go ahead and update the current list of packages:

apt-get update

Following this, upgrade the software on the system:

apt-get upgrade

This last step, again, may take a long time to complete. You should pay attention to the list of packages held back (listed at the top of the output from this command); these packages will have to be requested specifically. Usually, packages are held back because they require new software or other drastic changes: the Linux kernel is always held back. Use a command like this one to install these packages:

apt-get install some-package some-other-package

You may have to repeat this more than once until all of the packages have been installed and none are held back.

Then, you should repeat the update and upgrade in order to completely verify that the system is as updated as it can be:

apt-get update
apt-get upgrade

These last commands should execute quickly, as everything is probably complete – however, it is not impossible that the upgrades would have affected something that requires another upgrade. Not doing this probably won’t matter, but why not do it anyway?

Once done, a reboot is required to make sure that the old software is no longer being used and all that is used is the new upgraded software. Don’t just continue the boot: reboot.

One more thing can be done to clean up afterwards – possibly after the reboot. At a command shell, enter this command to remove unneeded software:

apt-get autoremove

Some general tips on making distribution upgrades as easy as possible: first, upgrade the system to the latest set of patches and software available. If there are bugs in the ugprading software, you want to fix it first.

Once upgraded to the most recent patch set (or the most recent minor level), then upgrade to the next major level (and no more!). Upgrades are normally tested only to the next major release. Any other upgrade is certainly not tested and not guaranteed to work.

If you want to upgrade beyond just the next major release, repeat the above steps for each major release: first upgrade to the most recent software, then upgrade to the next major release.

Make sure that you have reliable power throughout: if a system loses power in the middle of an upgrade process (or otherwise stops cold) then you will have to perform a recovery and hope that nothing was lost.