The Advent of NoSQL

The concept of “NoSQL” (that is, non-relational databases) is more of a phenomenon than you might think. The NoSQL Live conference will take place on March 11, 2010, put on by the people behind MongoDB, a non-relational database.

In June 2009, a number of folks gathered in San Francisco to discuss the various NoSQL technologies (such as Cassandra, Voldemort, CouchDB, MongoDB, and HBase). Johan Oskarsson has an article about the meeting, with videos and presentations from the presenters.

ComputerWorld took note of the event, discussing NoSQL and how Amazon.com and Google are using non-relational databases for their data stores. Likewise, too, Facebook converted to non-relational databases.

Digg posted a nice article that talks about their conversion from MySQL to Cassandra, showing how they came to the point of considering non-relational databases.

Possibly the oldest non-relational database is non other than MUMPS (or M). This includes GT.M (open source) and Intersystems Cache. Long before relational databases came on the scene, MUMPS was running and saving data – and it continues to this day, working hard in finance and healthcare settings.

Over at nosql-database.org, they claim to be the Ultimate Guide to the Non-Relational Universe. This may be true; certainly they have an extensive list of links to noSQL articles, and a list of NoSQL events.

The NoSQL world has been covered by Dave Rosenberg, who noted the upcoming NoSQL Live event in his discussion of real-world use of non-relational databases. Dave had reported earlier about the pervasiveness of non-relational databases in the cloud.

Now to go read some more about NoSQL…

Using NET-SNMP and HP-UX SNMP together

Why would one want to do such a thing? A major reason is that the HP-UX SNMP daemon only supports the EMANATE protocol for subagents; this means that subagents that support the AgentX protocol (which NET-SNMP – provided as part of HP’s Internet Express – supports) are not supported and cannot be accessed via HP’s SNMP daemon.

However, the HP-UX specific information is only available via the HP-UX native SNMP daemon. What is the answer?

Change one or the other to run on a non-native port, that’s the answer. With the two daemons listening on different ports – in essence, acting like to discrete damons – the capabilities of both can be exploited. Since the native HP-UX snmp daemon does not provide the capability of specifying the port, the net-snmp daemon can be moved – and it is relatively trivial to do so as well.

There is probably already a line that says:

agentaddress 161

Change this line to a new port – I used 166:

agentaddress 166

Restart the daemon. Once the NET-SNMP daemon has been moved, enable HP’s SNMP daemon (if you’ve not already done so) and start it up again:

cd /sbin/init.d
SnmpMaster start

This should enable your two SNMP daemons on different ports. Now you can access whichever one holds the data you want. For example, using the command snmpwalk, getting Caché data can be as simple as:

snmpwalk -m ALL -v 2c -c public my:166 .1.3.6.1.4.1.16563

Whereas getting HP-specific data can be retrieved this way:

snmpwalk -m ALL -v 2c -c public my .1.3.6.1.4.1.11

Note the contrast between the two commands: one accesses the host my with the standard port (my); one uses the host my with the port 166 (my:166).

As a side note, note that Caché provides AgentX subagents, and note, too, that OpenVMS supports SNMP and AgentX as of v8.x. Thus, there’s no fighting with the SNMP daemon on OpenVMS.

Using SNMP with Intersystems Caché

Intersystems Caché can be monitored using SNMP, but it must be started. The details of using SNMP in Caché are detailed in the Caché Monitoring Guide in Appendix B.

Firstly, to make life easier, the SNMP MIB for Caché is included in the installation of a Caché instance. Go to the top level directory (which contains the CPF file) and then change to the directory SNMP. This directory contains the SNMP MIB (named ISC-CACHE.mib).

Put this file with the other MIBs that your client uses. This will provide names and details for your SNMP client. If using net-snmp under Red Hat Enterprise Linux, put the MIB file in /usr/share/snmp/mibs/.

To start using SNMP in Caché (assuming your SNMP server supports AgentX and is already running), use this command (in the %SYS namespace):

%SYS> d start^SNMP(705,20)

The first parameter is the standard port for AgentX (705), and the second is a timeout value (default of 20). When you look at the jobs running in %SYS (using THIS^%SS) you will see a job named SNMP.

To stop SNMP, just enter (again, in the %SYS namespace):

%SYS> d stop^SNMP()

(Don’t forget the parenthesis; it won’t work otherwise.) Logs are written to the mgr/SNMP.log file.

Once SNMP is started, you can check Caché data:

snmpwalk -m ALL -v 2c -c public server .1.3.6.1.4.1.16563.1.1

This command is a net-snmp command, and assumes a server running SNMP v2 with a “public” community and Caché SNMP running. If SNMP is fully set up, you will get a variety of details about your Caché instance. The MIB file is well-documented as to what each element is and means.

VxFS (or HP Online JFS) Snapshots

A disk snapshot is a snap in time, a picture of what a disk looked like “back then”. This can be very useful for maintenance.

For example, being able to freeze a Caché instance, take a disk snapshot, then thaw the Caché instance will permit you to take backups or copies of a Caché database with minimal downtime.

For HP-UX Online JFS and Veritas VxFS the commands are the same (since these are actually the same product – or close to it). To actually do a snapshot:

mount -F vxfs -o snapof=/var/cache/db /dev/snap01 /snap

The first file system presented in the command line is key: it is the source of the snapshot. Note that it can be either a device or a current mount point. The second (device) is a filesystem prepared to hold a snapshot, and the last is the usual mount point.

Once this is done, the normal filesystem can continue to be used while the snapshot retains the older data as it was taken. In the example above, /var/cache/db could be used normally while the snapshot resides on /snap. If there was a directory /var/cache/db/db01 then there would also be a /snap/db01 available as well.

One caveat is that as long as the snapshot is mounted and in use, the changes to the original filesystem are being saved – it is possible that the snapshot volume can run out of space. When this happens, you will receive what may appear to be mysterious disk full errors unless you realize what is happening. So don’t keep your snapshots around forever.

PWN to OWN Contest at CanSecWest 2008

The PWN to OWN Contest is a hacking contest at the CanSecWest security conference that pits a standard install of Linux, Windows, and MacOS X against all comers. Each laptop has a default installation on it, and has not been hardened at all. The successful hacker will not only win a cash prize, but the system in question as well.

The MacBook Pro was the first to fall, and the laptop running Microsoft Vista Ultimate second. However, there will be those that misinterpret the results by not realizing how the contest was conducted.

Each contestant gets 30 minutes to attempt to crack the machine, and can choose which machine to attack. The attacks are limited by the rules, and each day that went by the rules allowed a wider range of attack vectors. It was a third party application (Adobe Flash) that permitted the compromise of the Microsoft Vista machine.

No part of the contest can be considered a scientific study into which system is more secure than the other: contestants attacked a single machine of choice, and contestants were allowed their attempts one at a time – and the operating system was not hardened.

This is entirely different than, for example, the Capture the Flag contest at DEFCON. That contest consists of setting up a server and trying to capture the other teams “flag” through compromising the server in some way. In that contest, any and all comers are permitted to enter and to attack at will during the contest with whatever vulnerabilities and methods they have available.

Speaking of DEFCON, DEFCON 9 saw the entrance of an Alpha-based VMS machine – installed with the standard setup – which remained unscathed throughout the contest, though try they did. The VMS Team (the Green Team) had a writeup and also wrote a white paper afterwards.

If you are interested in DEFCON, DEFCON 16 will be August 8-10 in Las Vegas, Nevada.

Also, speaking of DEFCON – let’s not forget the similarly named but totally unrelated InterSystems DEVCON2008, which is just wrapping up. DEVCON, among other things, covers Caché development and related. It is interesting to note that InterSystems DEVCON began 15 years ago, whereas DEFCON began 16 years ago. I wonder how much Caché security is covered at DEVCON2008.

The Caché Command Line

When you are working in a Caché session, you don’t need to stick to a single line per command. People think in one step at a time.

For example:

csession myinst
MY> zn "%SYS"
MY> d ^%SS

Instead of this, why not do this:

csession -U '%SYS' 'ALL^%SS' |more

The ALL line entry into the %SS command provides a long list without paging, and then the pager of choice can be used with all of the shell access available as necessary. However, this also makes it possible to use the grep command or sed command or perform other processing on the resulting output.

It is also possible to feed in input using echo to answer prompts as necessary.

The Caché Database

I’ve not spoke of the Caché database previously; this is a product from InterSystems that uses MUMPS (now called M) as its database engine. The M language and database are thoroughly intertwined; database accesses and so on are just variable assignments, etc.

MUMPS goes back to the 1960s, to the days of COBOL, LISP, and FORTRAN. However, in spite of its age, it is robust and thriving (quietly) today – especially in the finance and health care industries where Caché is widely used.

They have an excellent set of documentation available, including an online set.

On another note, it turns out that InterSystems is now being sued by a patent troll, and they’ve asked the Groklaw community to help find prior art that would invalidate the patent.