UNIX Administratosphere

In Solaris 9, two new utilities were added: pkill and pgrep. These tools are perhaps old news to Solaris admins. However, these utilities were then quietly added to Linux in short order, and now show up in HP-UX 11i v3 and perhaps others. What do these tools do?

First of all, pkill is just a wrapper for pgrep. Well, what does pgrep do? It searches the current processes for a match based on arguments that you give to it.

The most common use would be to search for a command. The pgrep utility will then return each process ID, one per line. The pkill utility will send the default kill signal to each pid. The pgrep utility can search based on a large array of factors, including userid, groupid, virtual size, effective user ids, command name, full command string, and more.

Here’s an example:

# pgrep cache
13
12
14
15
33
49
22006
21950
21973
21976
#

When combined with scripting, these commands can be quite useful. Consider this ksh snippet:

for i in $(pgrep cache) ; do
  // do some commands here
done

HP-UX has some nice features in their initialization scripts, but you have to be aware of them in order to take advantage of them.

One good starting point is the rc(1m) man page. The rc program is the actual program that does all of what we can initialization: that is, all of the startup processes, the rc.d directories, the symbolic links, the run levels - rc does all this. It is rc that init(1) calls to make it happen.

Also, the init scripts are not where you would first look - or second. The scripts live in /sbin/init.d. This directory contains all of the scripts (and no links). Then look at one of the scripts in order to determine what can be done with your own scripts - and to see if there are any new features.

There are several features of the HP-UX init script mechanisms that you can take advantage of:

• The /etc/rc.config.d directory - to set configurable parameters for each individual subsystem (usually as represented by a script)
• The startup and shutdown messages: these are formatted nicely and make for quick viewing of the startup or shutdown process
• Results: not just SUCCEED or FAIL but also N/A (Not Applicable) and others.

To use the /etc/rc.config.d directory, just source the appropriate configuration “script” into your environment. Then the appropriate variables will be set.

The startup and shutdown messages are exquisitely simple: instead of start and stop, the routine is called with start_msg and stop_msg. Respond to these subcommands by printing the appropriate text for a descriptive message. With these set, your init script will display its appropriate message when it is started or stopped during startup or shutdown.

Then the results of the script - which show up in the HP-UX startup or shutdown in the far right hand side of the line. The possible results are:

• (0) Subsystem was brought up (or down) successfully.
• (1) Subsystem encountered errors.
• (2) Subsystem was skipped, such as from configuration in the /etc/rc.config.d file or other reasons, and did nothing.
• (3) A reboot of the system is requested; rc(1m) will perform the actual reboot. /etc/rc.bootmsg will be displayed on the console prior to reboot, then deleted.
• (4) Subsystem was successful and started a process in the background.

Any return value larger than 4 is the same as 1: subsystem encountered errors.

Note, too - there is nothing that mandates that these must be shell scripts (except perhaps for rc.config.d file syntax). If you must, it could be a perl script - although a shell script is best. Just make sure to have the proper header so that the system doesn’t try to interpret your perl script as a shell script.

And a quickie tip for you! If you find yourself using a shell and are expecting to edit the history using vi - only to find that there is no history - you can find that your display no longer responds (!). In reality, the system is responding just fine, except you can’t see it (small things, right?). This is because the editing sequence for vi is ^[k (ESCape-K). This mucks up the display as it is not being handled by the shell but rather by the terminal display.

How to get out? I use the following command sequence: first, do a set command - then another. How does this work? Well, I can’t explain everything, but the first output from set will contain the ^[k sequence in it (look in the variable $_). My theory is that that is not the only place to find it. It also does not seem to matter what the second command is, just one with output (and not just a carriage return).

Anyone who feels the need for a challenge can research this - I just know it works, and can be typed very fast.

Over at Onkar Joshi’s Blog, he wrote an article about how to write a shell script to process a text file using parallel processing. He provided an example script (provided here with my commentary in the comments):

# Split the file up into parts
# of 15,000 lines each
split -l 15000 originalFile.txt
#
# Process each part in separate processes -
# which will run on separate processors
# or CPU cores
for f in x*
do
runDataProcessor $f > $f.out &
done
#
# Now wait for all of the child processes
# to complete
wait
#
# All processing completed:
# build the text file back together
for k in *.out
do
cat $k >> combinedResult.txt
done

The commentary should be fairly complete. The main trick is to split the file into independent sections to be processed, then to process the sections independently. Not all files can be processed this way, but many can.

When the file is split up in this way, then multiple processes can be started to process the parts - and thus, separate processes will be put onto separate threads and the entire process will run faster. Other wise, with a single process, the entire process would only utilize one core, with no benefit of multiple cores.

The command split may already be on your system; HP-UX has it, and Linux has everything.

This combination could potentially be used for more things: the splitting of the task into parts (separate processes) then waiting for the end result and combining things back together as necessary. Remember that each process may get a separate processor, and that only if the operating system is supporting multiple threads will this work. Linux without SMP support will not work for example.

I did something like this in a Ruby script I wrote to rsync some massive files from a remote host - but in that case, there were no multiple cores (darn). I spawned multiple rsync commands within Ruby and tracked them with the Ruby script. It was mainly the network that was the bottleneck there, but it did speed things up some - with multiple cores and a faster CPU, who knows?

Also, in these days, most every scripting language has threads (which could potentially be run on multiple CPUs). I’ll see if I can’t put something together about threading in Ruby or Perl one of these days.

A while ago, I talked about making big files. This was, by nature, UNIX-specific - that’s what I deal with all day, and the focus of this blog.

However, not all systems are UNIX (or Linux) - and not all the systems I deal with all day long are UNIX or Linux. However, perl is everywhere - and can be used quite easily to generate large files whatever you might be on.

For example, to make a 5M file, try this:

perl -e "open(FD, 'myfile'); print FD 'x' x (1024 * 1024 * 5);"

If you are inside of vim (a vi-clone which also runs everywhere), try this:

:!perl -e "print '-' x (1024 * 1024 * 5);"

This gives you a single line (5 megabytes in size). To make multiple lines:

:!perl -e 'for ($i = 1; $i < 500; $i++) { print "x" x 39, "\n"; };'

This makes 500 lines of 40 characters each (including single-character line terminator). If the system line terminator is two characters, then use 38 instead of 39. In total, this gives 19000 characters (about 18 kilobytes).

Perl is quite useful for creating portable scripts - but is by no means the only one. The ideas given here carry over to other languages that may be available. For instance, tcl and python and ruby are also available in other environments, and can do the same things as perl does here.

Of course, perl’s repetition operator ‘x’ makes it particularly easy here.

Update: corrected perl one-liner.

The Korn shell (as well as bash and the POSIX shell) has a history mechanism that can be very useful. This history also can be used with line-editing in order to edit the command before entry - and either vi editing or emacs editing can be used.

There are two environment variables that control the history:

• HISTFILE (stores commands; default $HOME/.sh_history)
• HISTSIZE (number of commands to keep)

Ksh-93 instroduces two new variables to go with these:

• HISTEDIT (replaces FCEDIT)
• HISTCMD (number representing current command)

Neither of those are of much if you use command-line editing.

The location of the history file (contained in HISTFILE) is of some importance. When ksh is used in an environment with NFS-mounted home directories, then the history file will be stored on the NFS volume. This has been known to cause problems in some environments (HP-UX, for one). In these cases, the HISTFILE can be changed to a local directory such as /tmp/.$$_hist_file.

This also brings up another thing of importance: this history file is read by all of the shell logins of that user. So if multiple people are logged into the same account (root for instance), then the same history file is used. This can be confusing, so it may be useful to change the HISTFILE setting for that session to avoid interference.

This history file is also preserved across logins - so some root sessions will disable the history mechanism entirely, preventing others from reading the history file. However, history is quite useful, so a compromise would be to limit the number of commands kept (by changing the HISTSIZE variable).

Line-editing is what makes command history so eminently useful. The mode used (vi or emacs) is chosen based on the setting of the EDITOR and VISUAL environment variables (if set to vi, emacs, or gmacs). Alternately, the option may be chosen directly by using the command:

set -o vi

or the command to choose emacs:

set -o emacs

Once this is done, then standard editing commands can be used. The current line is treated like a single-line window into the shell history file; so going up goes up a line (or command) in the history, and going down goes down a line (command) in the history. In emacs this translates to ^P (previous line) and ^N (next line); in vi it translates to the commands k (up) and j (down). The only special commands are file-completion commands. In both vi and emacs modes, they are similar.

• ESC-\ (filename completion - or as much as possible; emacs uses META-ESC and vi has an alternate ESC-ESC)
• ESC-= (outputs list of possible completions)
• ESC-* (completes file with all matches and enters editing mode)
• ESC-_ (enter nth word from end of last command - e.g., first from end, second from end, third from end - default is first from end)

In emacs mode, META translates into ESC normally.

Here is an example of a list of possible completions (from ESC-= entered at end of first line):

$ ls -l s
1) sec-2.4.1.tar.gz
2) sec-2.4.1/
$ ls -l s

The command is presented a second time for editing at the end of the list. If ESC-* is pressed the command line becomes:

# ls -l sec-2.4.1 sec-2.4.1.tar.gz

Alternately, if ESC-\ is pressed, the command becomes:

# ls -l sec-2.4.1

It was the lack of understanding of the shell history and command-line editing that held me back from adopting the Korn shell over the C shell for a number of years (I used csh interactively, but wrote scripts in ksh).  I made the switch and never went back.