Category: Perl

HowTo Fix: “[…] only available with the XS version of Scalar::Util”

Recently, we upgraded our systems, and one of the updates was an update to Perl. After this update, a portion of Bugzilla failed to work as expected; the problem turned out to be in SOAP::Lite:

# perl -MSOAP::Lite -e "print;"
weaken is only available with the XS version of Scalar::Util at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Lite.pm line 2502
BEGIN failed--compilation aborted at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Lite.pm line 2502.
Compilation failed in require.
BEGIN failed--compilation aborted.

This is on a machine running Red Hat Enterprise Linux (RHEL) 5.7. The bug (Bug #375621) was initially reported back in November 2007 with RHEL 5.2. The final resolution came with bug #594768, where the problem was explained completely and resolved for good in RHEL 6.

The problem is that the updates to Perl are overwriting files added via CPAN. When this happens, versions that have an XS version are reverted back to their original forms without XS support. This appears to happen with Scalar::Util the most.

The fix is to reinstall the CPAN modules affected by the Perl upgrade. A force will be necessary. For example, this command to install Scalar::Util:

cpan> force install Scalar::Util
Running install for module Scalar::Util
Running make for G/GB/GBARR/Scalar-List-Utils-1.23.tar.gz
Fetching with LWP:
  ftp://cpan.uchicago.edu/pub/CPAN/authors/id/G/GB/GBARR/Scalar-List-Utils-1.23.tar.gz
CPAN: Digest::MD5 loaded ok
Fetching with LWP:
  ftp://cpan.uchicago.edu/pub/CPAN/authors/id/G/GB/GBARR/CHECKSUMS
Checksum for /root/.cpan/sources/authors/id/G/GB/GBARR/Scalar-List-Utils-1.23.tar.gz ok
Scanning cache /root/.cpan/build for sizes
Scalar-List-Utils-1.23/
Scalar-List-Utils-1.23/Changes
Scalar-List-Utils-1.23/lib/
Scalar-List-Utils-1.23/ListUtil.xs
Scalar-List-Utils-1.23/Makefile.PL

Once the relevant modules are reinstalled, the files will be reverted back to the way they were before the upgrade took place – and all should be working again.

Using Perl Modules from CPAN with Distribution Packaged Modules

When I first started using CPAN, this was a concern of mine that I was never able to fully address: how do Perl modules from CPAN interact with modules installed via distribution packages?

It turns out that the answer is simple. First, however, let’s install a set of tools called pmtools; these tools will tell you a lot and make it simple. Using these will help you to understand how your system is set up as well. In Ubuntu, the package is called pmtools; in Red Hat, it is perl-pmtools. To install, use this command on an Ubuntu system as root:

$ sudo apt-get install pmtools

On a Red Hat Enterprise Linux system, use this command instead:

$ sudo yum install perl-pmtools

Once that is installed, use the tool pmdirs to see what directories are searched for modules in order. Here is one example from an Ubuntu 10.04 LTS server system:

$ pmdirs
/etc/perl
/usr/local/lib/perl/5.10.1
/usr/local/share/perl/5.10.1
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.10
/usr/share/perl/5.10
/usr/local/lib/site_perl
.

With this, you can see that several site-specific directories in /usr/local are searched before system directories in /usr.

Same thing holds true for Red Hat; here is an example from a Red Hat Enterprise Linux 5.7 system:

$ pmdirs
/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.8
/usr/lib/perl5/site_perl
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.8
/usr/lib/perl5/vendor_perl
/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi
/usr/lib/perl5/5.8.8
.

In Red Hat, system-installed Perl modules go into /usr/lib/perl5/vendorperl/$version/ while modules installed with CPAN go into /usr/lib/perl5/siteperl/$version/. On the other hand, in Ubuntu, CPAN modules go into /usr/local/share/perl5/$version/ while packaged modules are placed in /usr/lib/perl/$version/.

This means that modules installed with CPAN will be searched (and found) first – whether they are newer or not. Usually they will be newer, but if they are the same version you’re better off staying with the packaged version of the Perl module as it will often have bugfixes and patches not present on the original.

If you want to see where a particular module lives – use pmpath:

$ pmpath CPAN
/usr/local/share/perl/5.10.1/CPAN.pm

This shows that CPAN was installed from (or updated from) CPAN as well.

If you want to see what version is installed, use pmvers:

$ pmvers CPAN
1.9600

Note: In Ubuntu you may find that the directory /usr/local/lib/site_perl is missing and that some pmtools utilities will fail because of this. Use this command to create the directory (and stop the error messages):

# mkdir -p /usr/local/lib/site_perl

Installing VMware vSphere CLI 4.0 in Ubuntu 10.04 LTS

Installing the VMware vSphere Command Line Interface (CLI) has the potential for problems. In my case, it generated an error – a three-year old error. Perl returns the error:

undefined symbol: Perl_Tstack_sp_ptr

Not only has this error been around for three years, it also has shown up in numerous other instances. Ed Haletky wrestled with the error in VMware vSphere CLI back in June of 2008. The error surfaced in Arch Linux in 2008, both in running their package manager and in running cpan itself. This error also came up (again in 2008) in attempting to build and run Zimbra. (The response from Zimbra support was cold and unwavering: we don’t support that environment and won’t discuss it. How unfortunate.) The error also affected the installation of Bugzilla according to this email thread from 2009.

On the Perl Porters mailing list, there is an in-depth response as to what causes this error. From reading these messages, it appears that there are two related causes:

  • Using modules compiled for Perl 5.8 with Perl 5.10
  • Using modules compiled against a threaded Perl with an unthreaded Perl

One recommended solution is to recompile the modules using the cpan utility:

cpan -r

That may or may not be enough; it depends on if there were other errors. In attempting to run the vSphere CLI, I get this error:

IO::Compress::Gzip version 2.02 required--this is only version 2.005

To fix this, I ran cpan this way:

cpan IO::Compress::Gzip

In my case, that loaded IO::Compress::Gzip version 2.033.

I also loaded the libxml2-dev package; I don’t know if that was necessary or not:

apt-get install libxml2-dev

Whenever using cpan, I always wonder how it affects my packaged installations and whether it installs for all users or just me (and how to control that) – but I’ve never had any problems and installs as root seem to go into /usr/local – which makes sense.

Having done all this, I can now use the vSphere CLI to activate SNMP on the ESXi 4 servers. For the record, this is an integral part of ESXi 4 and supports all SNMP polling and traps – previously, only SNMP traps were supported. Certainly a nice improvement.

Why Java is the Future for OpenVMS

Support for Java in OpenVMS has increased over the years, and now Java for OpenVMS on Integrity is part of the basic system, and includes Java JDK 6.0 and JRE 6.0.

Installing the Secure Web Server (SWS) – based on Apache – you also get Tomcat and Perl for free.

As long as OpenVMS remains viable, I personally expect both Perl and Java to flourish on this platform. Especially, when not using Perl for typical administration tasks, I expect that Java will be available for more powerful duties.

I would even expect to be able to put things like Stripes, Spring, or even Scala and Lift onto OpenVMS. With the portability of Java, one could potentially just copy over class files, Java archives, or even web application archives and expect things to (mostly) work.

The support for both Perl and Java on OpenVMS makes for an exciting time – and Tomcat to boot.

Share your experiences with Java on OpenVMS…

Programming Languages on OpenVMS

Looking at OpenVMS administration, and investigating the possible languages to use on OpenVMS, there are many options – some supported by HP and some community open source options.

One of the first options is Perl; apparently Perl was originally designed as a way to write scripts that worked under UNIX and VMS both. Bernd Ulmann wrote an article for OpenVMS Technical Journal 13 about Perl on OpenVMS and gave a presentation on it in the spring of 2009 at an HP Connect OpenVMS Meeting (English translation) in Germany.

The HP version of Perl appears to be tied to the Secure Web Server (SWS) but it can stand alone.

Another language that is growing on OpenVMS is Java. Jean-Yves Bourles and Thierry Uso wrote on Java and OpenVMS in OpenVMS Technical Journal 10. Netbeans is available from HP to facilitate Java development on OpenVMS.

With Java available, that opens up the possibility of perhaps using a language that runs on the Java JVM as well. That brings to mind JRuby, Jython, Groovy, Scala, and Clojure. Information on most of these is rather scarce unfortunately; only Scala and JRuby have ports (both by the aforementioned Thierry Uso). Personally, these two are the most interesting to me; Scala has unmatched integration with Java itself as well.

Python is also available. Python seems to be the new administration tool of choice; at least, Red Hat seems to think that way.

As part of the Secure Web Server (SWS), you also get HP’s version of PHP. However, this does not seem to be a separate product as Perl is, and there is no description of using PHP as a scripting language (which you can do by running PHP against a file from the command line).

Lua is graciously made available for OpenVMS by our friends over at Hoffman Labs. Lua is a fantastic scripting language that doesn’t get the cover that it deserves.

Lastly, Tcl/Tk is available as well.

So which do I recommend? Well, Perl, PHP, and Java are all HP supported products, so one could start there. With Java, I see Scala and JRuby being fantastic languages as well, although they are not supported by HP. Lua is also a favorite of mine; an OpenVMS version is wonderful; however, Lua is not as widely available for other platforms as is Perl and Java.

I should mention that PL/I is still active on OpenVMS; it is commercially sold and supported by Kednos. PL/I was an interesting language, but it doesn’t have modern capabilities.

At the German openvms.org site there is also a big list of OpenVMS ports (including languages).

Are you ready for programming on OpenVMS? I am!

Perl 5 Development Resumes: 5.12 Released

Perl 6 development began in 2000, and ten years later it remains unready for production; thus several developers have come along and kick-started Perl 5 development once again – and now Perl 5.12 has been released.

Jesse Vincent made the announcement on the Perl development mailing list; he also announced the new release schedule for Perl 5, which is a production release in the spring and a development release monthly. The official release page for Perl 5.12 is over at CPAN:
http://search.cpan.org/~jesse/perl-5.12.0/

Over at ActiveState – the best-known supporter of commercial scripting languages such as Perl, Ruby, and Tcl – the ActiveState blog announced the release of Perl 5.12, followed by the release of ActiveState Perl 5.12.

For HP-UX, Merlijn Brand announced he was building Perl 5.12 for HP-UX, and the HP-UX Porting Centre already has Perl 5.12 packaged for download.

HP uses ActiveState Perl for HP-UX, but uses standard Perl on OpenVMS. I don’t see any word about 5.12 on OpenVMS, but no doubt it will come. Likewise, Perl 5.12 on HP-UX will have to run through the vetting process before it is officially introduced into HP-UX.

I see that Ubuntu has not rolled out Perl 5.12 into Karmic Koala. Their software roll-outs also depend on Debian, so we’ll see how long this takes.

Part of the reason that Perl 5 was revived is because the development of Perl 6 – a complete rewrite and redesign from scratch – is taking so long. Arguably, the complete redesign of Perl is contributing to the stagnation of Perl development (until this year). A complete redesign is a difficult thing, and some people believe that the redesign of the Netscape browser lead to Netscape’s downfall.

It really does appear that a complete redesign of a successful software project is rarely successful; more successful is the evolving process that most software goes through – including, in some cases, refactoring and subsystem replacements (for example, replacing the virtual memory subsystem in the Linux kernel or the replacement of the Ruby execution interpreter for 1.9).

We’ll just see what the future holds for Perl 6 – but I’m not holding my breath.

5 Programs That Should be in the Base Install

There are a number of programs that never seem to be installed with the base system, but should be. In this day and age of click-to-install, these programs will often require an additional install – I maintain that this should not be.

Most of these will be relevant to Linux, but the programs will often be missing on other commercial UNIXes also.

  • Ruby. This is the first that comes to mind. I have been installing ruby onto systems since 1.46 – and ruby is still a fantastic scripting language, and one of the best implementations of object-orientated programming since Smalltalk.
  • m4. I recently wrote about m4, and thought it was already installed on my Ubuntu Karmic system – not so. I used it to create a template for the APT sources.list file.
  • ssh. This should be installed everywhere automatically, and not as an add-on. For many UNIX systems, ssh is an add-on product that must be selected or compiled from source.
  • rsync. Rsync is a fabulous way to copy files across the network while minimizing traffic – even though it is not designed to be a fast way.
  • ksh. This will surprise most commercial UNIX administrators. However, Linux does not come with ksh installed – and the emulation by GNU bash is weak. Now you can install either AT&T ksh-93 (the newest version!) or the old standby, pdksh (which is close to ksh-88).

OpenVMS is a different animal – and some of the things that should be installed by default would be perl, ruby, java, SMH, and ssh. I’m not sure if perl or ssh is installed by default, but they should be. OpenVMS should also support compliant NFS v3 and v4 support out of the box – without making it difficult to connect to other NFS servers.

What programs do you think should be in the base install?

Scripting on the Java Virtual Machine

Now that OpenVMS has Java, and that HP-UX has Java, I started wondering about the possibility of running a scripting language on the Java Virtual Machine (JVM) as a way of supporting all these diverse environments with the same code and tools.

Choosing a language can be difficult, especially when there are so many good ones to choose from. I’ll assume for purposes of discussion that one is already familiar with at least one or more computer languages already (you should be!).

So what are the criteria a system administrator should use to choose a language on the JVM?

  • Does the language have a strong and vibrant community around it? The language might be nice, but if no new development is being done on it, it will eventually fail and stop working on the newer JVM. Bugs will not be fixed if development has halted. It also helps to have a large variety of people to call on when trouble arises, or when your code has to be maintained in the future.
  • Does the language support your favored method of programming? If you have no desire to learn functional programming, then don’t choose a language that is a functional language. Find a language that thinks the way you do (unless you are specifically trying to stretch your mind…).
  • Is your preferred langauge already available for the JVM? There are implementations of Ruby (JRuby), Python (Jython), LISP (Armed Bear Common Lisp), Tcl (Jacl) and many others. A language that you already know will reduce your learning time to near zero on the Java Virtual Machine.
  • What are the requirements? For example, JRuby requires a dozen libraries; Clojure and Armed Bear Common Lisp have no outside requirements. Which is simpler to install onto a new machine?

So what languages am I looking at? I am looking at these:

  • Clojure – a LISP-like functional programming language which seems to be taking off handsomely.
  • JRuby – Ruby is my all-time favorite scripting language, and having it available whereever the Java VM is is a very tintillating prospect. It’s also directly supported by Sun, the makers of Java.
  • Groovy – this is a new language that takes after Ruby and Smalltalk, and it is growing in popularity at a dramatic pace.
  • Scala – this is a language with a strong developer base and an object-oriented and functional design. Don’t know much more about it yet.
  • Armed Bear Common Lisp – ABCL is a full Common LISP implementation for the Java VM, and is used as part of the J editor. Unlike ABCL, development on J seems to have stopped; development on ABCL has gone through a resurgence after not quite dying for several years. ABCL is the closest thing to LISP on the JVM, and is usually the first mentioned – even though its development community is not nearly as strong as Scala or JRuby.

These are only the ones I’ve chosen to focus on; there are many, many more.

Perl Tidbits: Annoyances and Surprises

Having given up Perl at one time to use Ruby almost exclusively, I have returned to Perl in recent years since Ruby is not available for HP-UX 11i v3 nor for OpenVMS 8.3. Perhaps some Ruby folks are listening; it is abominable that HP-UX 11i has gone without Ruby for so long. It would be nice to be able to say that it is shameful that Ruby doesn’t exist on OpenVMS 8.3 – but we know how popular OpenVMS is…

Here are some of the things I’ve discovered in bits and pieces about Perl since my “return”:

  • HP-UX does not use the Perl core, but rather ActiveState Perl. This may not sound like much, but it can be important when using modules. For instance, the Net::FTP module has a bug in it that causes the last character of ASCII transfers to be “snipped”, and several Perl core modules are missing entirely.
  • It is possible for Perl to quietly abort without warnings. This makes no sense, but I’ve experienced it. Only stepping through the code with the debugger will show what line it is – and removing the line in question makes the program start working again.
  • Using CPAN can corrupt your HP-UX Perl installation. I suspect this is because CPAN assumes that you are using the Perl core, and if not, then things can break. In my case, the module Data::Dumper was hosed so bad I needed to reinstall Perl.
  • Perl requires an additional module to emulate Ruby’s p command. This was a real disappointment; attempting to print data in a readable format (no matter what its type) is a simple command in Ruby; in Perl, you must include an additional module and use a function with a confusing name (is it Dump? or is it Dumper? or is it dump?) – I often have to correct my code to use the right one.
  • Testing for a module’s existance is easy. Use the command perl -e 'use My::Module;' to see if the module is there.
  • Don’t think of $var as the name of a variable, but as the value of a variable. Just this one change in thinking has made a tremendous amount of difference in programming. When you think this way, you see an entry like @myarray[4] for what it is: a list with a single element in it. Likewise, $mine{entry} is a scalar value returned from a hash. The same “variable” can be found with differing leading punctuation based on its value, not on its type.
  • If your script needs to be portable across platforms, then write it in Perl. The Korn shell is nice and even more ubiquitous than Perl, and Ruby is (in my opinion, anyway) much nicer than Perl, but Perl is where ksh and Ruby are not. Being there is a significant part of the battle: if that snazzy scripting language isn’t installed, you can’t use it – and Perl is there.
  • The documentation (in perldoc) is tremendous! Use perldoc whenever you can: it includes more than a man page would. It includes tutorials, frequently asked questions (FAQs), module documentation, function documentation, and more.

Though I’d still write in Ruby at the drop of a hat, I have indeed rediscovered the joy of Perl. Part of this is due to a book that has illuminated the dusty corners of my knowledge: Effective Perl Programming by Joseph Hall with Randal Schwartz – an awesome book!

Why You Should Learn Perl

Previously I spoke on why one should learn vi (summary: learn vi because it’s on every UNIX and Linux system you’ll install…). Well, why should one learn Perl?

Because it’s only every UNIX and Linux system you’ll install…. and on OpenVMS… and available for Windows, too.

Unlike vi, I’m not as big a fan of Perl as I once was: having been interested in (and a fan of) object-oriented programming (OOP) for years – it only took Ruby little time to dislodge me from my interest in Perl (that would have been just prior to Perl 5).

Yet, this does not matter: Ruby is nice, but not ubiquitous. In particular, making Ruby run on HP-UX has proven to be extremely difficult in recent years – and it is not loaded by default in any case. I don’t know of any UNIX that installs Ruby as part of the base package (or that makes it available at all).

Learning Perl is not as hard as it may seem: since it is ubiquitous, there are many excellent books from which to learn Perl – and excellent references as well:

I have all of these, and find all of them to be useful. In my progression of learning Perl (or relearning it…) I find that Effective Perl Programming is fantastic. Specifically, presents a series of items (or tips) then shows you how to use and understand the tip in detail. I recommend this book fully.

Don’t neglect Perl, as it is everywhere, unlike any other language (including Korn shell!). If all you write is Korn shell, then your program will be unusable in any environment that does not provide ksh (think Linux and FreeBSD and OpenVMS for three). It’s true: ksh is not installed on Linux by default: bash is – and FreeBSD uses the C shell. However, all three environments provide Perl.