Compare and contrast: your favorite *NIX

I seem to find myself attempting to find good, quality answers to questions that usually invite flamewars. I usually manage to do alright.

I’ve discussed the merits of Linux and BSD in recent days. Why does it matter? Who cares if Linux is better than BSD – or if Solaris is better than Linux?

It is important, especially in the corporate data center, because we must justify the use and support of whatever system we want to use. It is not enough to say it’s better – we must justify our choices to people who don’t care which one looks better or which has the better development model.

What do executives care about? Stability and reliability. Twenty-four hour phone support. On-call support. Security against hacks. User base. These are some of the things that executives (like CIOs and CEOs) want to know about. Even for open source desktops, these same qualities are of importance to them.

If you know the ins and outs of all of the systems that are available, then you can better judge which may be good and can explain your choices (or prefered choices!) to your CIO or other management. Better still – can you back it up with examples and details? If you’re going to pitch FreeBSD (or a FreeBSD-based desktop) you’ll have to.

This is also why many times the systems that are installed are not the most reliable (they are the most well-supported) or the most technologically capable (they are the most widely known). This is truly unfortunate – it is the oft-reported “barrier to entry” (in this case, it is the network effect) – but it’s the way it is until you find management willing to take a chance.

Working in a development environment

In a standard business environment, a production system is one that must be up and stable, and cannot be changed without a lot of forethought and a lot of getting people to coordinate and okay the process. A development system is one that the administrators use to prepare for bringing systems into production.

However, if your users are developers, then things may be different – especially if you are also using the software in a stable environment.

Development, by its nature, produces unstable code which is prone to crashes and other undesirable behavior. This stands the usual system administration goals on their head: your systems, though they are in “production” (that is, they are used by normal users on a daily basis) – these “production” systems behave like test systems in that they are not reliable. With reliability issues, it may seem as if they are not production systems – but they are.

What’s more, there may be actual “production” systems – systems with the same software which is not being developed, but being used. These systems then, are also systems that should not change (in production, we would say), but do not have reliability problems.

Even though the development environment may feel like a test lab at times, with systems going experiencing hangs and so forth, these systems still need to be treated like a normal production system. Never forget that your users, even though they seem to do “bad” things to the system, still rely on the system being there on a daily basis.

It also means that you will have to respond to problems faster, and be proactive in preventing problems – and that you will have more problems to resolve.

In short, the normal software development environment is more challenging to the admins that support this environment – but also more exciting.

Refresh yourself!

System administration can be a very stressful position. When systems go down, and CEOs and other executives start leaning over your shoulder, or a system fails and you suddenly (and temporarily) become one of the most sought-after people in the company – the stress is immense and undeniable.

Even if you release your stress by doing what I do – administering your systems instead of someone else’s – it is even better to give up the computer, give up the Internet, and find something radically different to do – something which requires a different set of skills and thought processes.

In my case, when I do this, one thing I might do is to pick up a camera. Currently I am using a Pentax PZ10 (film) but I hope to augment it with a Pentax digital SLR sometime in the future. (Yes, I know Canon is great – and so is Nikon – but they won’t fit my Pentax lenses…)

Another thing I do is attend my favorite conferences. The biggest conference I like to attend is happening this week (aviators already know which one!) – Airventure. To us in the neighborhood of this world-wide gathering of aviators, we just call it “Oshkosh” (Airventure is located in Oshkosh, Wisconsin) or “EAA” (the Experimental Aircraft Association, or EAA, are the sponsors and conference operators).

Events and hobbies like these force you to use other skills, and other thought patterns – relieving stress and perhaps even lengthening your life. Stress is a killer, and learning new things is thought to lead to a longer life as well.

Books Every Admin Should Read – but Hasn’t (part 2)

Another book to read is The Humane Interface by Jef Raskin. This book is an easy read, very comprehendable and understandable.

When a system administrator writes a script, it behooves us to make it simple to use and simple to understand. If we write utilities for users, it is even more important to get the user interface right. Making the interface simple, understandable, and yet powerful can be daunting.

This book does an excellent job of introducing us to the challenges (and, indeed, the joys) of making the interface easy for normal human beings to use. This, in turn, will reduce the number of human errors that happen – human error is the number one cause of system failures. It will also introduce you to the fact that a lot of what passes for “difficult to use” is actually design failures in the interface.

Books that every sysadmin should read (but probably hasn’t)

There are a number of books that a system administrator should be reading. In this new weekly series, I plan to cover specific books and review them and their importance to the system administrator.

Book One: Re-imagine! by Tom Peters.

This book covers many things about business and conducting business in the new century. The book itself is a marvel of disruptive and in-your-face design (which is just the way Tom wanted it).

Probably the most important concepts covered (in depth) are the fact that you are a “professional service firm” of one, and that the customer (not the shareholder) is king – that is, provide the best service possible even at a cost to the company or to the employees.

Apache Cocoon and Customer Service

This is a very interesting article about using Apache Cocoon working, and how it compares to Struts and why. It is, first and foremost, a story of an Apache Cocoon customer and why they are unsatisfied. Don’t read just the article; read the comments too.

The article details several points where the Apache Cocoon community fails their customers:

  • Missing and incomplete documentation
  • Requirements on other, undesired systems (such as Maven)
  • Inflexibility: inability to use IntelliJ instead of Eclipse, for example (or at least, no documentation on how)

I don’t know about some of these, but I can sympathize with the lack of documentation and the inability to use Cocoon in a production environment. Some of the Apache projects, especially Java ones, seem to want to use all the latest and fanciest tools, but forget that others use something different – which makes it impossible to integrate Cocoon into established environments.

This lack of recognition of the corporate customer will only hurt Cocoon in the end. The switch to Apache Maven (instead of using Apache Ant as previously) also seems to have hurt Cocoon more than helping.

Many open source projects lack this customer focus: it is not a buzzword (or should not be), but rather a way of thinking. Many projects are done purely because somebody wanted to – but a serious long-term project outgrows its leader/founder and needs to consider the ramifications to its user base in what it does. A serious project also needs to consider what do the customers want? What do they need?

Programmers often do not understand why not everyone can read their documentation and just “get it.” A good example is the page “Your first Cocoon application using Maven 2“. I’ll annotate certain sections here, with likely responses from a corporate perspective (imagine a system administrator turned manager talking):

Note: First, make sure that you have Maven 2.0.6 or above installed. You can check this by calling mvn –version from command line. If this doesn’t work for you, read the Maven in 5 Minutes tutorial.

I don’t want to use Maven. I shouldn’t need to install a new product just to use Cocoon – and we’ve only just gotten Cocoon approved. Now I need to get another big package tested and approved? Why can’t you use Apache Ant like everybody else?

Note: Cocoon is not tied to Eclipse IDE by any means. This step only describes what can be done to avoid tedious work of setting up project in Eclipse manually.
If you don’t use Eclipse, you can either skip this step or find a similar procedure to load the block in the IDE of your choice.

That’s good to know. So how do I do it with IntelliJ? How do I do it with NetBeans? How do I do it without any IDE at all? I don’t want to have to install yet another big package that has to get approved.

After creating the block you probably want to run it. For this purpose there is a Maven plugin, that generates a minimal web application that loads your block.

Great. I need Maven just to run the application? Come on. Do I really need to get another big package approved?

Note: The mentioned minimal web application is automatically created, when mvn jetty:run is invoked. This happens because the rcl goal of the Cocoon plugin is bound to the Maven build lifecycle which is invoked too, when the jetty:run goal is executed. See the block’s pom.xml for details.

What’s an rcl goal? What’s a Cocoon plugin? What’s a Maven build lifecycle? What’s a jetty:run goal? Now I’m really confused – and you tell me to “read the source” rather than explaining yourself. And it sounds like I need Jetty, too. Great. Tomcat is on our system, and now I’m supposed to use Jetty? Or am I?

Apache Cocoon is a fantastic product – and I’ll keep using it where it fits in.  However, I do not expect to use 2.2 any time soon (which appears to be the first to require Apache Maven).  The project could use a very strong dose of customer relations.  The only reason other projects win is because they work with what people already have and are easy to use – and Apache Cocoon 2.2 neither works with current tools nor is easy to use.

Stress relief for System Administrators

Who, me, stress?  Nah….. sysadmins never have any stress, right?

Well, just in case you do – my feelings are that laughter is the best medicine – and I mean laugh out loud funny.  So what do you do?

There are a lot of ways to tickle the funny bone, and they’re personal as well.  Here are several that I use:

  • Subscribe to RSS feeds like Sharky’s Column and The Daily WTF.
  • Favorite comics (for me, that means: Liberty Meadows, Calvin and Hobbes, and Get Fuzzy)
  • Personal copies of laugh out loud funny comics
  • Funny movies

If you can indulge in some of these away from the office, it is much better.  Fresh air is good, too – as is doing something totally unrelated to work (even if it is other work).

Burn-out is a real danger to system administrators; so get away from the tension and away from the office!

Presentations (and Life and Creativity)

It’s time I put these links up and let you see some very (in my opinion) thoughtful and delightful lectures – and on topics that are very relevant to our lives today.

The first has been around for quite a while (a week or so) – and it is quite possible you know of it already. This is a lecture in a series that was once called the Last Lecture Series (but is now called Journeys), and was given at Carnegie-Mellon University. While the series was renamed, the concept remains the same – and in this specific lecture by Dr. Randy Pausch, it was no hypothetical lecture – it was very likely his last lecture. The lecture was not just poignant, but excellent in delivery, presentation, and in content.

Do yourself a favor – listen to Randy Pausch’s lecture. Listen to the entire thing. The entire thing.

You won’t regret it – and you may find things to use in your work – and your life – and your presentations. Listen to what he says – and watch how he uses the slides. Watch how he handles “The Elephant in the Room.”

The second lecturer, Tony Buzan, is a lecturer and researcher on the mind and on creativity and learning. He speaks earnestly and deeply about the importance of creativity in the young student, attempting to encourage the audience of teachers to develop creativity in their students. Listen, too, to what he says – and how he says it. He doesn’t have the flair of Steven Jobs or the high impact delivery of Randy Pausch, but he does it well.

No ums, ahhs, or ya’know from these speakers.

I have taken to watching presentations of late, looking for and watching for quality speakers. These two educators are, while very different, with different styles and different messages, are nonetheless quality speakers.

Want to learn more? Join with the local Toastmasters.

Customer Service (Who’s Your Customer?)

Who is your customer? As a member of the IT staff (which most system administrators are), you are a part of the support staff in the company. This is not necessarily true in all cases (such as those that work for contractors or outsourcing firms), but most system administrators are helping their companies maintain the infrastructure and computing resources of the company.

In some companies, the IT department (like other support departments) is forced to “bill” its customers (the rest of the company). This actually distorts the relationship between the departments in the company, forcing each into a sort of adversarial relationship.

However, it is worthwhile to remember that you do have customers. Your service is to provide reliable computing services for your customers, and to respond quickly to problems that arise.

With customers comes the need for customer service. However, don’t be misled – don’t provide just customer service – provide what Tom Peters calls “the Wow Factor” – provide superlative customer service which is over the top.

I’ve begun rereading Tom Peter’s book Re-Imagine! and have also begun to read the book magnetic service by Chip and Bilijack Bell. I’ll have more to say on these, but having read portions of Re-imagine! already, I can say that it is an excellent book.

Are your customers happy?

What is a SysAdmin?

Over at, a recent blog post described what goes into being a system administrator and what one should learn and remember as a system administrator. This is a phenomenal post, and covers a lot. Don’t read this post without reading that one.

A (unfairly) abbreviated list of his best points would be:

  • Go with your passion. Someone with a passion for the work will stand out from those who do not.
  • Be willing (even eager) to do the grunge work no one else wants to do. Tom Peters also makes this point (and often).
  • Remember that you also deal with people – and develop your people skills.

I’ve always had a passion for computing in general, and have developed many skills that dovetail into system administration nicely. Remember that World of Warcraft and Second Life are not going to improve your professional standing – whereas puzzling out that new version of Apache outside of work just might.

What are some personal projects one could take on (adjusted according to your passions) in order to become a more technically proficient administrator?

  • Install a DHCP server on your home network
  • Set up automated installs for whatever operating system suits your fancy
  • Configure thin clients for boot via PXE
  • Install a new operating system – then fix problems as they arise
  • Get some non-Intel hardware – and install an operating system on it – especially Linux on non-Intel
  • Create your own distribution – such as using Linux From Scratch or Core Linux
  • Configure a netdump or Linux Kernel Crash Dump facility – then try it.

However, don’t forget – this is only the technical side (and is UNIX/Linux biased to boot). Have I done any of these suggestions? Actually… I’ve done all of them. So what are you waiting for?