Mainframe Linux: Pros and Cons

Why would one want to move Linux to the mainframe (such as IBM’s z10)? There are many reasons – and many reasons not to. Computerworld Australia had a good article describing (in part) some of the reasons the insurance company Allianz did just that. IBM has been pushing Linux on the z series for some time, and Red Hat and SUSE offer Linux variants for that purpose.

One common reason to move to a mainframe is that Linux servers have proliferated in the data center, taking up valuable space and becoming quite numerous. When all you need for a server is the hardware and a low-cost or no-cost Linux, then servers start popping up all over the place.

A single mainframe such as the z10 can handle thousands of servers (a test done in 2000 put 41,400 Linux servers on one IBM mainframe). The replaced servers can then be eliminated from the data center, freeing up valuable space and reducing the workload of current system administrators.

A common instance is where the company already has a mainframe in-house, running COBOL applications. Thus, the purchase cost of a mainframe (in the millions of dollars) has already been absorbed. Such a scenario also makes the case for a new mainframe much more appealing, as it puts the enhanced power to work immediately.

Replacing thousands of Intel-based Linux servers with a single mainframe will reduce cooling costs, power costs, physical space requirements, and hardware costs.

So why would anyone not want to use a mainframe?

If there is not already a mainframe in the data center, purchasing a mainframe just for the purpose of consolidation can be too much – mainframes typically cost in the millions of dollars, and require specially trained staff to maintain. Adding a mainframe to the data center would also require training current staff or adding new staff. A new mainframe also requires a new support contract. All of this adds up to not just millions of dollars of additional cost up front, but additional costs every year.

Another consideration is the number of Linux servers in the data center that would be moved. If there are dozens – or a hundred or two – it may not be entirely cost-effective to focus a lot of energy on moving these servers to the mainframe.

A supercomputer such as HP’s Superdome (with its attendant iCap and Integrity Virtual Machine capabilities) would probably be a better choice to consolidate dozens of Linux servers. The costs are lower, and the power requirements are lower – and you can purchase as much or as little as you need and grow with iCap. Most companies also already have UNIX staff on hand, and adapting to HP-UX is not generally a problem if needed.

Another benefit is that a server such as the Superdome offers virtualization of not just Linux systems, but Microsoft Windows and HP-UX as well – and soon, OpenVMS as well.

Using a large Intel-based server can virtualize a large number of servers with software from companies like VMWare and Sun.

These options won’t necessarily allow you to virtualize thousands of servers – but then, do you need to?

Subversion joins Apache

ApacheCon 2009 ended recently – and like other good conferences, there were a number of announcements of interest.

One of the interesting announcements was on 4 November 2009, when the Subversion project (currently hosted at Tigris.org) announced that they would become absorbed under the Apache Foundation umbrella as part of the Apache Incubator. (Subversion has an excellent online book available).

There doesn’t seem to be any licensing change. It should not affect other projects based on Subversion; most notably for this author is SVNKit, the Java-based client library – which, in theory at least, will run under OpenVMS with Java.

HP Instant Capacity (iCap)

One of the things that may affect any clusters you have – or other systems – is that management does not want to spend enough to handle any possible load.  With a cluster, this means that you may not be able to handle a fail-over because there is not enough spare processing power to handle the extra load when it happens.

HP’s Instant Capacity (“capacity on demand”) is an answer to this dilemma.  The base idea is that you have extra hardware already in the data center that is not available for use until it is necessary.  The switch that will enable this expanded capacity can be automatic or manual; when some portion of the extra capacity is enabled, you pay for it and it can be used from then on.

Yet, Instant Capacity (iCAP) is more flexible than this.  The capacity may be enabled only temporarily instead of permanently – this is known as TiCAP (temporary iCAP).  Thus, you can save even more by buying extra hardware but enabling only a small portion of it.  During the recent HP Tech Days that I attended in San Jose, California, a situation was described where an HP Superdome could be purchased with a large amount of the hardware already in place – but only a small amount of the hardware enabled.  When the extra power is needed, for example, a cell in the Superdome could be enabled until such time as the power is no longer necessary.

There is also Global Instant Capacity (GiCAP) which even allows the movement of power from one system to another.  For example, if a CPU on one system is underutilized and another system needs the resource more – then the CPU resource can be “logically” moved from one system to the other through GiCAP.  Alternately, if one system dies and another system needs its power, the dead system’s resources can be used by the active system by moving them through GiCAP.

iCAP and TiCAP are available for HP-UX (on PARISC and Itanium) and for OpenVMS (only on Itanium). GiCAP is only available for HP-UX. 

I find iCAP and TiCAP to be very interesting.  From a cost perspective, you pay only a minimal amount to keep the resource; when it is enabled, you then pay for it for the duration – or buy the hardware outright for permanent use as needed.

Powered by ScribeFire.

HP Tech Day: HP Superdome

I was recently invited to take part in an HP Tech Day in San Jose, California, celebrating the 10th anniversary of the HP Superdome (the most advanced server in the Integrity line).  This was an event designed to introduce several of us in the blog world to the HP Superdome.  The attendees included David Adams from OSNews, Ben Rockwood from Cuddletech, Andy McCaskey from Simple Daily Report (SDR) News, and Saurabh Dubey of ActiveWin.  This was a quite eclectic and broad mix of perspectives: David Adams covers operating systems; Ben Rockwood covers Sun goings on (as he mentions in his article, he wore a Sun shirt: bold as brass!); Saurabh Dubey covers Microsoft goings on; and I, as loyal readers may know, cover system administration (with a focus on HP-UX, OpenVMS, and Linux – all of which will run on Superdome). Andy McCaskey over at SDR News also had a nice writeup on his experiences.

It is possible I was the most familiar with the architecture and with the capabilities, though I’ve not seen or worked with a Superdome in the past: the capabilities of the Superdome are largely based on the fact that it is cell-based.  The rp7420 cluster which I have maintained over the last several years uses the same technology, though cells from the rp7420 are incompatible with the Superdome.  The software is the same: prstatus, etc.  The System Management Homepage (SMH) was also shown, although it was almost shown as a benefit of the Superdome (it’s actually in every HP-UX since 11i v2, and is an option for OpenVMS 8.x).

There was a lot of talk about “scaling up” (that is, use a larger, more powerful system) instead of “scaling out” (using a massive cluster of many machines).  The Superdome is a perfect example of “scaling up” and is possibly one of the best examples.  I was impressed by what I saw as the capabilities of the Superdome.  There was a lot of comparison with the IBM zSeries, which is the epitome of the current crop of mainframes.  The presenters made a very strong case for using Superdome over zSeries.

They did seem to focus on running Linux in an LPAR, however; this creates a limit of 60 Linux installations.  Using z/VM as a hypervisor, one can run many more Linux systems.  I have heard of a test run in Europe (years ago) where a zSeries was loaded with one Linux installation after another – when the testers reached into the tens of thousands (30,000?) the network failed or was overloaded; the zSeries system was still going strong.  Trouble is, I’m not able to back this up with a source at the moment: I’m sure it was available as part of a print (Linux) journal – it may have been called “Project Charlie.”  Can anyone help?

The usability features of the Superdome were in prime display: for example, the power supplies were designed so that they could not be inserted upside-down.  Another example: the cells for the Superdome are in two parts: the “system” (including CPU, memory, and chip glue) and the power supply.  This makes it much easier to remove in the typical datacenter row and makes each part lighter, making it easier for users. There are innumerable items like this that the designers took into account during the design phase.  The engineering on these systems are amazing; usability has been thought of from the start.  In my opinion, both HP and Compaq have been this way for  a long time.

Speaking of the tour, this system that they showed us was a prototype of the original HP Superdome that shipped for the first time in 2000.  This system was still going and was using modern hardware: these systems are not designed for a 3-4 year lifecycle, but a much longer, extended lifecycle.

There were a lot of features of the system that I’ll cover in the next few days; it was enjoyable and educational.  I very much appreciate the work that went into it and hope to see more.

By the way, if you read Ben Rockwood’s article at Cuddletech, look at the first photograph: your author is center left, with the sweater.

Update: thanks to Michael Burschik for the updated information on Test Plan Charlie, which saw 41,000 Linux machines running on the IBM zSeries back in 2000. I’ll have a report on it soon. Strangely enough, I still haven’t found the article I was thinking of – but of course, a project like that isn’t reported in just one periodical…

Powered by ScribeFire.

The Dichotomy of a System Administration Career

When you choose to work in system administration, generally you have to focus on one operating system or another. The dichotomy comes in choosing a system to focus on for your career.

How do you go about choosing which system you want to administrate as a career? Do you go with a common system like Microsoft Windows or a relative rarity such as OpenVMS?

If you go with Microsoft Windows Server, for example, there will always be jobs available (relatively so, anyway). Every corporation seems to have at least one Microsoft Windows Server, and they all need to be taken care of by someone who knows what to do. However, there will be lots of other people that do the same thing. So even as there are jobs out there, there are lots of applicants and lots of competition. With this abundance of people who know how to administrate Windows servers (or think they do) comes a lower pay, as an employer can be selective in who they choose. This is the basic economic principle of supply and demand at work.

On the other side is administering UNIX servers – or even more so, OpenVMS servers. The number of people who can administrate these servers is less than those who work with Windows, which means their expertise is more expensive. For a variety of reasons, UNIX is present less in the average enterprise, and the number of UNIX servers is very likely dwarfed by the number of Windows servers. This is an advantage as the pay scale will be higher, but the disadvantage is that the jobs will be fewer.

When the market is tight, those with more specialized skills will find themselves having to move where the work is, and will have to search further afield for possible openings. It is a trade-off – and it’s your choice. Just be sure you have the facts first before you choose.

Expanding OpenVMS Memory

When you expand OpenVMS memory, there are a number of other parameters you may wish to revisit. If you increase your memory dramatically, you will certainly have to change these SYSGEN parameters. You can also look each parameter up using HELP:

HELP SYS VMS_MAX_CACHE

(The parameter SYS is short for SYS_PARAMETERS.)

Some parameters to consider changing are the following:

• GBLPAGES. If you don’t increase this, you’ll be getting warning messages when you try to take advantage of all that memory. In short, this parameter sets the amount of memory that the kernel can keep track of; if you use too much this parameter is a limiting factor.
• GBLPAGFIL. The page file needs to be able to take all of the pages that it might be called upon to reserve; increase this parameter.
• VCC_CACHE_MAX. If you’ve not tuned your cache (XFC) then you’ll find half of your memory to be taken by the cache. This is almost certainly not what you want; modify this parameter to reduce the amount of memory your cache is allowed to take. Even so, do remember that your cache will decrease and increase dynamically in any case – but if you scale it back, then you’re not wasting memory so much.
• MAXPROCESSCNT. This sets the maximum number of process slots – in essence, the maximum process count (which is what the parameter is called, after all). If you have a lot more memory, you’ll want to use it to run more, right? That’s not any good if you use too many processes and can’t run any more.
• BALSETCNT. If you set MAXPROCESSCNT, you should set BALSETCNT to the same amount minus two – and never higher.

These changes can be made in the SYS$SYSTEM:MODPARAMS.DAT file and then use the AUTOGEN command to configure the sysetm. The MODPARAMS.DAT file uses a simple format; for our purposes, you can use something like this:

ADD_GBLPAGES=1000
ADD_GBLPAGFIL=1000
VCC_CACHE_MAX=2048
ADD_MAXPROCESSCNT=1024
ADD_BALSETCNT=1024

In place of ADD_* you can also use MAX_* or MIN_*. You can see more examples in HELP AUTOGEN MODPARAMS.DAT. AUTOGEN is described in the HELP; be careful using it! You don’t want to muck up the system so bad you have to reboot or to reinstall.

Powered by ScribeFire.

UNIX and OpenVMS Online Resources

It is possible to get free online access to UNIX or to OpenVMS; these can be useful in building up your experience on a platform when starting from scratch – or when a review is required.

One of the oldest public access systems in the country is the Super Dimension Fortress (or SDF as it is usually called). SDF offers free accounts, but does ask for US$1 to gain standard access. This isn’t because access is expensive, but because too many people have used the facilities for nefarious purposes (the process suggests that the new user is not a person who will strike and leave).

SDF runs NetBSD on DEC Alphas; this was driven mainly by security and stability. Previously, Stephen Jones, the proprietor, ran SDF using Linux on Intel for several years (which he describes as “the dark years”). BSDTalk had an interview with him back in 2006.

You could also try PolarHome – this shell provider provides access to hosts running Linux (Red Hat, Debian, SUSE, Ubuntu, or Mandriva), OpenVMS (Alpha or VAX), OpenBSD, FreeBSD, NetBSD, HPUX, IRIX, QNX, Solaris, Ultrix, AIX, Tru64, and OpenStep. Unfortunately it requires payment for shell accounts – again because of abuse. The payment is 10 units of your local currency or US$2, whichever is more – and this is per host as well. No other site provides this diverse of a selection.

For truly free UNIX shell accounts, one can try Grex, which is a more professionally-run system (Polarhome and SDF are sole proprietorships). Grex offers totally free shell accounts, but also has memberships (for people to help support the site). It is possible that Grex has the most users as well. Like the others, paid membership does have its privileges – but unlike the others, membership is mainly to provide support for Grex, rather as a security feature.

For OpenVMS, there is a very unique online shell provider: Deathrow Cluster. This is a cluster of three machines running OpenVMS 7.3 – one VAX, one Alpha, and one emulated VAX (SIMH) on a dual Xeon machine. This last is a perfect example of what can be done with an emulator, especially with SIMH which can emulate all manner of old Digital and IBM hardware. However, SIMH does not emulate the Digital Alpha, unfortunately. Like Grex, Deathrow provides completely free shell accounts; like SDF and Polarhome, it is (or appears to be) mainly one person’s purpose to keep it running with a lot of volunteer help.

Any of these will be good sources to keep your shell skills sharp – and in some cases, programming as well. They’re also good people to support; why not offer them some donations if you can?

OpenVMS and Network Information

If you don’t know where to look, OpenVMS networking information can seem to be confined inside a mysterious black box. It doesn’t have to be.

The ANALYZE command can provide a lot of good information. Be sure to have a large enough scroll-back buffer on your terminal when you do this:


$ ANALYZE /SYSTEM
SDA> SHOW LAN /FULL


You can also find out a lot of good information in a hurry with the LANCP command:


$ RUN SYS$SYSTEM:LANCP
LANCP> SHOW CONFIGURATION


You can also look up information using the TCPIP command:


$ TCPIP
TCPIP> ifconfig -a


However, while this information is all good, it isn't complete without marking the back of the computer in some way so that you know which port is which. If you have to, you can hook up a laptop with a network cable and watch the traffic: the DECNet clustering traffic is such that you'll see it on every active interface - which provides you with the MAC address for that port.

Why You Should Learn Perl

Previously I spoke on why one should learn vi (summary: learn vi because it’s on every UNIX and Linux system you’ll install…). Well, why should one learn Perl?

Because it’s only every UNIX and Linux system you’ll install…. and on OpenVMS… and available for Windows, too.

Unlike vi, I’m not as big a fan of Perl as I once was: having been interested in (and a fan of) object-oriented programming (OOP) for years – it only took Ruby little time to dislodge me from my interest in Perl (that would have been just prior to Perl 5).

Yet, this does not matter: Ruby is nice, but not ubiquitous. In particular, making Ruby run on HP-UX has proven to be extremely difficult in recent years – and it is not loaded by default in any case. I don’t know of any UNIX that installs Ruby as part of the base package (or that makes it available at all).

Learning Perl is not as hard as it may seem: since it is ubiquitous, there are many excellent books from which to learn Perl – and excellent references as well:

• Learning Perl by Schwartz, Phoenix, and Foy
• Programming Perl by Wall, Christiansen, and Orwant
• Effective Perl Programming by Joseph N. Hall with Randal L. Schwartz
• Perl in a Nutshell

I have all of these, and find all of them to be useful. In my progression of learning Perl (or relearning it…) I find that Effective Perl Programming is fantastic. Specifically, presents a series of items (or tips) then shows you how to use and understand the tip in detail. I recommend this book fully.

Don’t neglect Perl, as it is everywhere, unlike any other language (including Korn shell!). If all you write is Korn shell, then your program will be unusable in any environment that does not provide ksh (think Linux and FreeBSD and OpenVMS for three). It’s true: ksh is not installed on Linux by default: bash is – and FreeBSD uses the C shell. However, all three environments provide Perl.

Using NET-SNMP and HP-UX SNMP together

Why would one want to do such a thing? A major reason is that the HP-UX SNMP daemon only supports the EMANATE protocol for subagents; this means that subagents that support the AgentX protocol (which NET-SNMP – provided as part of HP’s Internet Express – supports) are not supported and cannot be accessed via HP’s SNMP daemon.

However, the HP-UX specific information is only available via the HP-UX native SNMP daemon. What is the answer?

Change one or the other to run on a non-native port, that’s the answer. With the two daemons listening on different ports – in essence, acting like to discrete damons – the capabilities of both can be exploited. Since the native HP-UX snmp daemon does not provide the capability of specifying the port, the net-snmp daemon can be moved – and it is relatively trivial to do so as well.

There is probably already a line that says:

agentaddress 161

Change this line to a new port – I used 166:

agentaddress 166

Restart the daemon. Once the NET-SNMP daemon has been moved, enable HP’s SNMP daemon (if you’ve not already done so) and start it up again:

cd /sbin/init.d
SnmpMaster start

This should enable your two SNMP daemons on different ports. Now you can access whichever one holds the data you want. For example, using the command snmpwalk, getting Caché data can be as simple as:

snmpwalk -m ALL -v 2c -c public my:166 .1.3.6.1.4.1.16563

Whereas getting HP-specific data can be retrieved this way:

snmpwalk -m ALL -v 2c -c public my .1.3.6.1.4.1.11

Note the contrast between the two commands: one accesses the host my with the standard port (my); one uses the host my with the port 166 (my:166).

As a side note, note that Caché provides AgentX subagents, and note, too, that OpenVMS supports SNMP and AgentX as of v8.x. Thus, there’s no fighting with the SNMP daemon on OpenVMS.