Tag: make

A New Init for Fedora 14

Apparently, a new project (to replace init, inetd, and cron) named systemd is nearing release and will be used to replace upstart in Fedora 14 (to be released in November – with Alpha Release due today!).

There is a healthy crop of init replacements out there, and the field is still shaking out. Replacing init – or specifically, System V init and init scripts – seems to be one of those never-ending projects: everyone has an idea on how to do it, no one can agree on how.

Let’s recap the current crop (excluding BSD rc scripts and System V init):

I am still waiting for the shakeout – it bugs me that there are dozens of different ways to start a system, and that none of them have taken over as the leader. For years, BSD rc scripts and System V init have been the standard – and both have stood the test of time.

My personal bias is towards SMF (OReilly had a nice article on it) and towards simpleinit – but neither has expanded like upstart has.

So where’s the replacement? Which is The One? It appears that no one is willing to work within a promising project, but rather starts over and creates yet another replacement for init, fragmenting the market further.

Lastly, if the current init scheme is so bad, why hasn’t anything taken over? Commercial UNIX environments continue to use the System V scheme, with the sole exception of Solaris which made the break to System Management Facility (or SMF). Why doesn’t HP-UX or AIX use SMF or Upstart if the current environment is horrible?

Sigh. It’s not that the current choices of replacement are bad – it’s just that there are so many – and more keep coming up every day. Perhaps we can learn something about the causes of this fragmentation from a quote from a paper written about the NetBSD rc.d startup scripts and their design:

The change [in init] has been one of the most contentious in the history of the [NetBSD] project.

Using make and rsync for Data Replication

When maintaining a cluster environment (such as HP Serviceguard) there are often directories and configurations which need to be maintained on two different local disks (on different machines). Using make and rsync (with ssh) are excellent for this.

The rsync command allows you to replicate the local data onto the remote side copying only that which is necessary. This is not necessarily the fastest, but it is the most efficient: rsync was designed for efficiency over slow links, not speed over high speed links. Configure rsync to use ssh encryption automatically in the Makefile, then use rsync as the way to copy the files over:

RSYNC_RSH=/usr/bin/ssh -i /path/to/mykey

rsync -av $(LOCAL_FILES) remoteserver:$(PWD)

To automate this properly, an ssh key will have to be created using keygen and transfered to the other host. The private key (/path/to/mykey in this example) is used by ssh in the background during rsync processing; with the key in place, no interactive login is necessary.

For best purposes, create an “all” tag (at the top of the file) that explains the usable tags, and create a “copy” tag that does the relevant rsync.

I recommend copying only relevant files, not the entire directory: this way, some files can be retained only on one node – this is good for log files and for temporary files.

For example:

LOCAL_FILES=*.pkg *.ctl *.m4
RSYNC_RSH=/usr/bin/ssh -i /path/to/mykey

all:
    echo "To copy files, use the copy tag..."

copy:
    rsync -av $(LOCAL_FILES) remserver:$(PWD)

Make sure to verify the code before you use it in normal operation. Use the rsync option -n to perform a dry run which affects nothing. Also make sure that you don’t update files on different hosts; things might get interesting (and unfortunate…)

After performing the update, the Makefile can trigger a reconfiguration or a reload of the daemons to put the configuration in place.

m4: 5 Places You Should Use It

The utility m4 is underutilized and underappreciated, and when paired with make can be indispensible. The mailer sendmail has made m4 legendary, and behind GNU autotools is a lot of serious m4 code. Why not use it for other purposes as well?

Here are some areas in which m4 can be used to ease your work:

  • cfengine. You can template the configuration of a “standard file” or a “standard directory” as m4 macros and save yourself a lot of typing (and errors). You could also define a standard “shell script” configuration and use m4 to create it every time.
  • Nagios. Nagios configurations benefit a lot from heavy use of m4. Macros can be used to template large configurations, and to template standard configurations.
  • DHCP server. A DHCP server can be set up to assign specific addresses to specific hosts; the configuration, however, can be tedious if there are more than a couple of hosts. Macros can be used to simplify static host configurations.
  • rpmrc. The /etc/rpmrc file is used to configure RPM during package build time. While not used as heavily, m4 can be used to create an rpmrc file specially tailored for the type of CPU running – this configuration will then be used when RPMs are built.
  • HTML generation. When creating hand-crafted HTML, m4 macros can simplify the creation of the standard tags, especially taking care of the beginning and ending tags transparently with only the content visible.

m4 makes templating easy. Anywhere that you need to set up a configuration with multiple sizable elements, m4 can help. A single m4 macro can expand to numerous configuration lines, and a couple dozen m4 lines can result in an extensive configuration set.

Using m4 macros in this manner prevents errors (because of less typing) and all appropriate configuration elements are the same (no mistaken copies).

m4 also adds “include” file capabilities to any configuration file where it is used. This permits common configurations to be reused everywhere, even though the configuration file may not support include files directly.

Even though make is not present on most system installs, m4 is. Adding make will complete this pair and set you on your way to automatic configuration. Try it today!

Nagios and m4

The macro processor m4 is perhaps one of the most underappreciated programs in a typical UNIX environment. Sendmail may be the only reason it still exists – or would that distinction include GNU autotools?

Configuring Nagios can be dramatically simplified and made easier using m4 with Nagios templates. Even when using Nagios service and host templates, there is a lot of repetition in the typical services file – perhaps an extreme amount of duplicated entries.

Using m4 can reduce the amount of work that is required to enter items.

Here is an example:

DEFHPUX(`red',`10.1.1.1')
DEFHPUX(`green',`10.1.1.2')
DEFHPUX(`blue',`10.1.1.3')
DEFHPUX(`white',`10.1.1.4')
DEFHPUX(`black',`10.1.1.5')
DEFHPUX(`orange',`10.1.1.6')

In my configuration, each line above expands into 64 lines (including three lines of header in the comments). So the result of those six lines is 384 lines of output.

Every DEFHPUX creates a host, complete with standard service checks such as PING, SSH, and TELNET. This is done all with just a few macro definitions at the beginning of the file.

Read about m4 and understand it, and your Nagios configurations will be much easier. You can use the program make to automate the creation of the actual config files and the check and reload necessary for Nagios to incorporate the changes.