Hardware Hacking on the EeePC

This is an amazing piece on hacking the Asus EeePC (be patient for the link; not only is it image heavy, it suffered the Slashdot effect…. need I say more?).

If you’ve soldered before, this will be fairly easy; if you don’t know what solder is, you may not want to do this.

The amount of new capabilities that this user added are insane. He added the following:

• A USB hub
• GPS with antenna
• Bluetooth
• Card reader
• Flash drive
• Power switch
• Wifi (with 802.11N draft support)
• FM transmitter
• Modem

This is a truly amazing list of things to add to a small environment such as the EeePC. Each addition is accompanied by a screenshot of a Windows XP hardware listing of the particular item. The user stated that they wanted to be able to dual-boot into Linux as well; too bad they didn’t show the Linux support for these items.

Another thing: this article also includes a handsome list of links on the EeePC at the end; so go take a look!

Update: I just found this detailed review; it shows a lot about what the EeePC can do and is well-written.

Finding the operating system version installed and on DVD (Solaris)

I saw this post about finding the version of a Solaris DVD – very interesting. It relies on the fact that the DVD contains all of the packages in a unpacked format, and then goes looking for /etc/release contained within the SUNWsolnm package. His example looked like this:

# cd /Solaris_10/Product/SUNWsolnm/reloc/etc
# cat release
Solaris 10 3/05 HW2 s10s_hw2wos_05 SPARC
Copyright 2005 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 26 September 2005
#

This also translates into doing the same thing on an installed system:

# cat /etc/release
Solaris 10 6/06 s10s_u2wos_09a SPARC
Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 09 June 2006
#

FreeBSD on the fitPC and on the EeePC

FreeBSD is a nice environment, and I tend to gravitate to it (though I love Linux and Solaris as well). It does tend to work better in smaller environments than either Linux or Solaris.

There was recently a discussion of FreeBSD on the EeePC; it appears that while some items do not work (to be expected) it runs nicely and works nicely (including wireless). There was recently posted a simple introductory article which also refers to a comprehensive article on FreeBSD on the EeePC.

There was also an article (with followup) about running FreeBSD on the fitPC; in contrast to the EeePC, this sounds like it is not as good. However, the fitPC has less memory and a slower processor; it is unclear as to whether the processor is “fast enough” (I still use Pentium IIIs for my use!) or if it really is slow. It is, however, very surprising that the default Ubuntu install would be a graphical installation that swaps badly and comes without SSH.

The fitPC forums have a nice Linux on fitPC section, which also includes the BSDs as well. The biggest problem with FreeBSD seems to be its lack of a USB CDROM driver in the base kernel; however, apparently OpenBSD loads fine. Since the system has only 256M of memory, it perhaps should not be swamped with heavy desktop applications.

HP-UX binary paths (and PATH variable)

HP-UX tried using a different approach to loading packages and binaries to run. The traditional method uses /usr/local – the BSD variants use this directory to its fullest extent. The original idea behind /usr/local was that the system file system could be upgraded – or even replaced – and it would not affect added system programs. This extends to /home as well. If /home and /usr/local were separate file systems, then the operating system could be upgraded with impunity.

HP-UX adopted a newer version of this, the /opt directory (for optional). The way HP-UX adopted it (which seems to be not all that unusual) every added program stuffs all of its information under its own directory. For example, /opt/ldapux contains the etc files, lib files, man files, sbin files and so forth for HP-UX LDAP.

In addition to the “new” HP-UX standard (introduced about HP-UX 10) the /usr/local directories remain available and very occasionaly used – but most of what was not to go into /opt but into /usr/local actually wound up going into /usr/contrib/bin. This means that there were generally three locations for all HP-UX binaries:

• /usr/local
• /opt
• /usr/contrib

What’s more, because of the layout of the /opt directories (with a great many /opt/*/bin and /opt/*/sbin directories – as well as a great many /opt/*/lib directories as well) the system PATH variable could be very long.

The actual system PATH is stored in /etc/PATH; each added program option was to update /etc/PATH (and other files in /etc such as SHLIBPATH) to match their directories. The easiest way to reset the path in HP-UX is to use this command:

export PATH=$(cat /etc/PATH)

Then add in any extra locations you want (separated by colons).

FreeBSD 6.3 is OUT! (Armada E500 installation)

Just after installing 6.3-RC2, I discovered that 6.3 was officially released!

This didn’t take much work to update to. The basic steps were:

freebsd-update upgrade
freebsd-update install

Then – after a reboot (for kernel updates, I presume) another:

freebsd-update install

Relatively painless, throughout.

Then, continuing my install over the weekend, there were a few niggling things to fix. First off, the buttons up on the top of the keyboard didn’t work (no surprise there). Using the xev utility helps to pin down the actual keycodes for these keys, then use the xmodmap tool to add the appropriate actions to the keys. In my case, xev reported that the keys left to right were:

• 163 (info key)
• 142 (home key)
• 154 (search key)
• 143 (mail key)

These can be configured using xmodmap and a .Xmodmap file configured this way:

keycode 163 = Help
keycode 142 = XF86HomePage
keycode 154 = XF86Search
keycode 143 = XF86Mail

The values on the right (Help, XF86HomePage, XF86Search, XF86Mail) show their XFree86 heritage, but apparently do not change for X.org. These are activated by using the command:

xmodmap .Xmodmap

A good place for this line would be in the .xinitrc. However, once this is set, it is still necessary to tie applications to the shortcuts listed. In KDE, this is done in the Keyboard Shortcuts section of the Regional and Accessibility pane in Settings. In this dialog, select the “Command Shortcuts” tab, then the application you desire to use. For example, “Find Files/Folders” could be attached to the shortcut XF86Search. Once the xmodmap has been modified using the command above, then click on the Custom radio button, and click the shortcut button. Press the actual shortcut button to define the shortcut for the application.

Do this for all four buttons, and all will be well.

Then there was the problem of the mouse not being operational when waking up from a suspend. Turns out that the moused(8) daemon is the culprit. Sending a HUP signal to the daemon fixes it, but having to do this all the time is not a desirable outcome. The utility acpiconf(8) describes how it uses /etc/rc.suspend and /etc/rc.resume before and after suspending the system. I placed the command:

pkill -HUP moused

into this script, but I don’t yet know if it is truly having the desired effect or if other things are causing failures.

Another thing: the CD player would not play CDs in Amarok. Apparently, this is due to HAL and DBUS not being available. HAL depends on DBUS, so both are necessary. The following packages were needed:

hal-0.5.8.20070909
dbus-1.0.2_2
dbus-glib-0.74
dbus-qt3-0.70_1

I don’t know that all the dbus packages are necessary, but I decided not to chance it. Of course, dbus is part of GNOME, but whatever. Once these packages are installed, add the following to the startup configuration in /etc/rc.conf:

hald_enable="YES"
dbus_enable="YES"

Next time the system boots, these daemons will start.

Also, up until this point X.org had to be started using startx as a normal user. However, adding the login screen isn’t difficult. Edit the /etc/ttys file. In this file, there will be a line that specifies the command xdm.

Since kdm (the KDE display manager, the login screen) is actually a reworked xdm, switching one for the other is smooth and clean. Replace the xdm setting with /usr/local/bin/kdm (keeping the -nodaemon option) and set the tty to “on” (instead of “off”). Then the next time the system starts (or the ttys file is read) a login screen will activate on ttyv8 (if your file is like mine).

FreeBSD 6.3 RC-2 on a Compaq Armada E500

FreeBSD 6.2 has been on this machine for a while, but then I tried to upgrade all of the applications using the ports tree. This almost worked, except upgrading to Xorg turned out to be a massive headache and nothing worked.

It was then that FreeBSD 6.3 RC-2 was announced. I thought, why not? So off I went.

It installed well – if you don’t count my not providing enough room for /usr/local. With my “full-featured” (ha!) list of software, I wound up needing more than the original 2 Gb I originally alloted for /usr/local; with 4 Gb it worked. I also had to change the boot options, as it was still set to use 6.3-RC1 instead of 6.3-RC2. Changing the name in the options screen worked just fine.

Then after loading, I had to load the proper kernel – it couldn’t find the kernel. I selected /boot/GENERIC/kernel and all was well. At the boot loader prompt:

load /boot/GENERIC/kernel
boot

I had to configure Xorg. This was another headache. There was an excellent article from Julien Valroff about instaling Debian GNU/Linux on this machine. Despite the difference in operating systems, the fundamentals were similar. Another fantastic resource was this old page by Frank Steiner. Despite the age, the descriptions are relevant and useful (though, again, it is about Linux). There is a page on the Gentoo Wiki that describes the machine as well, though the other pages are more descriptive.

The screen display descriptions turned out to be the easiest; the problem was the mouse. Some descriptions suggest that the synaptics driver should work. However, this never did work for me. Using the standard PS/2 mouse driver and protocol worked just fine.

I also had to up the maximum files available, though for what reason I forget. Add this line to /etc/sysctl.conf to fix this problem:

kern.maxfiles=10000

Sound was another matter. It took a bit to figure out. First off, all the Linux directions suggested using lspci to see if it was there; this is Linux-specific. The FreeBSD counterpart is pciconf. Running pciconf -lv presents this:

pcm0@pci0:8:0:   class=0x040100 card=0xb1120e11 chip=0x1978125d rev=0x10 hdr=0x00
    vendor     = 'ESS Technology'
    device     = 'ES1978 Maestro-2E Audiodrive, ES1970 Canyon3D'
    class      = multimedia
    subclass   = audio

Thus, I knew that the sound was recognized. I just had to figure out how to get things to work with it. This means kernel support, da?

First attempts to load a driver turned up short; nothing is found in /boot/modules (!). The search path had to be changed to /boot/GENERIC:

kldconfig -i /boot/GENERIC

After this, load the snd_maestro driver:

kldload snd_maestro

After this, sound will work! Amarok is great…… and sound on this machine is excellent too!

Seeing as a I was trying to load KDE on here, the next step (once Xorg is working) is to add a startkde command to the .xinitrc file (in one’s home directory).

To make the system boot properly (and so you don’t have to load kernel modules manually all the time), the /boot/loader.conf file had to be created with this:

# Directory (in /boot) containing kernel and modules
kernel="GENERIC"
 
# Load maestro driver
snd_maestro_load="YES"

This then worked well.

I’m enjoying this machine again – though I am attempting to make it more of a usable desktop, which means more memory and all of the niggling setup work – like bootup splash screens, configuring kdm, and more – but hey, we’re system admins here, right?

Computer Forensics and the Chain of Custody

When a server is hacked, what do you do? That is the question. There are several questions:

• Do you want to prosecute the offender?
• Do you want to analyze the attack?
• Do you want to preserve the evidence?

For the moment, we’ll assume that you want to preserve the evidence in order to prosecute. Once this decision is made, the system administrator’s role starts to enter the legal realm. In order to preserve the evidence, a chain of custody must be maintained and preserved.

At its simplest, the chain of custody is a legal phrase that describes the provable knowledge of every one who handled the evidence and a provability that this evidence is the same as the one that was present at the crime scene. The question that the chain of custody attempts to answer is: Is this evidence (hard drive) the same one (and unaltered) that was at the crime scene (in the server)?

Before preserving the evidence, make a copy of the hard drive in order to have a copy for forensic analysis. This process should be noted on the evidence’s chain of custody, more than likely.

To make the chain of custody (and preserve the evidence), there are a number of things that can be done:

• Once the hard drive is no longer needed, wrap it and seal it (possibly along with an initial document stating its origins) – then sign across the seal itself (to prove that it hasn’t been opened since).
• Put the drive into a safe with a very limited access.
• Each time the drive is touched (i.e., moved, relocated, or reassigned) fill out an entry on a form with date, action, by who, and why the action was taken.

All through the process, there should always be a witness to what occurs. For example, when the system administrator removes a hard drive from a server, there should be additional witnesses present. Likewise, if anyone moves the hard drive – or gives it to someone, then witnesses should be present also. In all these cases, note the witnesses and other information in the chain of custody log (always with date).

Another excellent tip is to photograph the removal of the hard drive – specifically, photograph the location of the machine, what was done to the machine, and where was the hard drive physically located. The person removing the hard drive may also want to sign the hard drive itself, in order to prove that this was the actual hard drive removed from the system.

The wrap should be some sort of wrap that will show evidence of tampering – shrinkwrap may fit the bill.

The chain of custody is a legal phrase with a legal definition. However, I am not a lawyer (sheesh, do I really have to say that?). So for best advice, seek that of a lawyer (or forensic specialist).

In a big enough company, certainly you should consult with your in-house corporate attorney before taking any unnecessary action.

Here are some further resources:

• How to Keep a Digital Chain of Custody from CSO Online
• Proof of a Chain of Custody in a Case
• Forensic Science: the Basics by Jay A. Siegel (chapter 2)
• Computer Forensics: Computer Crime Scene Investigation by John R. Vacca (pp. 247 and following)
• Laws of Evidence by Thomas Buckles (pp. 81 and following)

Some of these resources are directed towards the legal profession instead of the computer professions, but if you have the willingness to learn more, they can only help.

Generating a coredump (gcore)

If you wish to examine a runaway program outside of its element, you may choose to use the utility gcore. This utility is found in Solaris, Linux, and HP-UX, and perhaps others. The program syntax is:

gcore [ -o corename ] pid

The pid is the process id of the process to dump core, and the corename is the base of the filename to use for the core dump – the full name is the base name plus period (“.”) and the process id number. The default is to use “core“.

HP-UX systems will accept multiple process ids instead of just one. Solaris has several additional flags (as well as multiple pids). The additional Solaris flags won’t be covered here.

Once core has been dumped, the program continues operation; it does not stop. Thus, gcore is especially useful for taking a snapshot of a running process.

For example, consider a program with the process id 6674:

gcore 6674

This command generates a core file in the current directory with the name “core.6674“. This file then can be read by the GNU debugger gdb. Solaris also provides the dbx(1), mdb(1), and pstack(1) utilities. HP-UX provides gdb as well as the HP adb(1) utility. Both Solaris and HP-UX provide a core management utility coreadm(1m) – which is a topic for another day.

This article has an excellent description of working with core files in Solaris.

OS Startup Process (HP-UX)

Each UNIX-based operating system seems to take a turn at creating a unique and different startup process, and HP-UX is no different than the others.

The /etc/inittab is a typical inittab file, but as installed doesn’t contain any comments to explain its entries. There is, however, a decent description in the man page inittab(4).

To configure the startup, all the parameters can be found in files in the /etc/rc.config.d directory; this directory is read by /etc/rc.config. Some startup programs read their configuration files directly, and some use /etc/rc.config to read all files.

The startup initialization files are found in /sbin/init.d (which is perhaps unlike most). /sbin/rc is run to begin the startup scripts. As mentioned, these files are in /sbin/init.d, but they are run from /sbin/rc[0-9].d directories. These directories will contain links to files in /sbin/init.d. Across all of these directories, there only needs to be two files: one to start the daemon in the right runlevel, and one in the next lower runlevel to stop it.

For example, consider the cron daemon:

# ls -1Fd /sbin/rc*.d/*cron
/sbin/rc1.d/K270cron@
/sbin/rc2.d/S730cron@

(First a note about the options to ls: -1 means list only filenames, -F adds the filetype designator at the end of the filename, and -d lists directories and not their contents.)

When HP-UX starts up (or changes to a higher runlevel) it runs start scripts (the S### scripts) as it goes from one level to the next until it reaches the desired level. When going to a lower runlevel, the system runs the kill scripts (the K### scripts) in order to stop processes.

The two listed files above (K270cron and S730cron) follow the [SK][0-9][0-9][0-9]script convention, and the file these link to is accordingly /sbin/init.d/cron. The numbers help to define the order that the scripts are run in; scripts are run in alphabetic order as defined by the (system) shell.

The startup scripts themselves are somewhat typical: a first parameter of start will start the service; a first parameter of stop will stop the service. Two other expected parameters are start_msg and stop_msg; these are used during the startup to provide a decently readable account of what is going on.

The scripts should return one of the following values as its exit value:

• 0. Success!
• 1. Failure.
• 2. Noop – translates to N/A (not applicable) during startup; script did nothing.
• 3. Script will cause a reboot (and will display /etc/rc.bootmsg if it contains anything).
• 4 and up. Treated same as 1.

All of the output from the startup and shutdown scripts is stored in /etc/rc.log (and the previous rc.log is moved to rc.log.old). Only the last two startup logs are kept. Here is a snippet of an rc.log:

Old /etc/rc.log moved to /etc/rc.log.old
 
**************************************************
HP-UX Start-up in progress
Thu Dec 13 13:56:28 CST 2007
**************************************************
 
Configure system crash dumps
Output from "/sbin/rc1.d/S080crashconf start":
----------------------------
crashconf: concurrent mode not supported on this platform
EXIT CODE: 0
 
Removing old vxvm files
Output from "/sbin/rc1.d/S090sw_clean_vxvm start":
----------------------------
 
VxVM INFO V-5-2-3360 VxVM device node check
Output from "/sbin/rc1.d/S091vxvm-nodes-check start":
----------------------------
 
VxVM INFO V-5-2-3362 VxVM general startup
Output from "/sbin/rc1.d/S092vxvm-startup start":
----------------------------

More information can be had on the rc(1m) and rc.config(4) man pages.

Dealing with “Hidden” Files

Files can be “hidden” on a filesystem in several ways; some are just attempts to “hide in plain sight.” This is not hiding data in files (such as steganography) but rather hiding the mere existence of files themselves.

There are several ways to do this. The basic ideas are:

• Make reporting tools lie about the existence of files. This falls into the realm of “rootkits.” Utilities such as ls (and even the kernel itself) may modified in order to not report the existance a particular file. A way to avoid this is to use programs (during examination) that are statically linked and precompiled and saved to read-only media: such programs are immune to future modifications, except for kernel resources (if the kernel is taken over, then all other resources are suspect).
• Make files “disappear” in file listings. Files of this sort include “.. ” (two dots followed by a space) and other non-standard characters. These files look normal but are not. One way to see files like these is to use the ls command and its -b and -q options. The -b option prints the octal equivalent of non-printing characters; the -q option uses the “?” character instead of non-printing characters. The -F option may also be of use: a file with a terminating space will show up when the -F places its “type” character at the end of the file name.
• Open the file, then delete it. As long as the file is in use, its disk blocks remain in use, and the file remains available. No other processes can access it – or even see it – but the program (or programs) that opened it still has access to it. Of course, this only works for running programs, but files disappear this way almost entirely. To see the files, use the utility lsof with the +L1 option. The +L option lists files with less than that many links; thus, +L1 lists files with less than one link (or zero links).

As mentioned previously, a static build of all relevant utilities to CDROM can be a very useful tool during investigation of a possible system break-in. All general reporting tools should be included: ls, ps, top, etc. Also included should be any desired programming interpreters: ksh, perl, ruby, tclsh, lua (whatever one finds useful). Also included should be any more intensive tools as well: tcpdump, nmap, etc.