Using Hamachi on Gentoo


Hamachi is a lightweight personal VPN connector that is a breeze to setup but there can be some pain if you don't know what to expect. As always Gentoo provides us with an ebuild that simplifies the installation process but getting up and running is still a little confusing.

Installation and Setup

User Specific Configurations

The obvious first step is emerge -av hamachi (this is only available to ~ARCH right now so add this to /etc/portage/package.keywords if desired). The following are the typical instructions to install hamachi on Gentoo using portage:

echo "net-misc/hamachi ~${ARCH}" >> /etc/portage/package.keywords # Optional
emerge -av hamachi
rc-update add tuncfg default

After these steps have been taken, you can run hamachi as any user on the system for ad-hoc VPN creation.

Server-Wide Configuration

If you prefer to do a system wide VPN configuration with hamachi, a different approach must be taken:

echo "net-misc/hamachi ~${ARCH}" >> /etc/portage/package.keywords # Optional
emerge -av hamachi
rc-update add hamachi default

Now all configuration should be placed inside /etc/hamachi for the VPN tunnels to be configured at startup or when the hamachi service starts.

Kernel Configuration

For hamachi to work correctly you do need the tun parameter in your kernel or loaded as a module. This parameter is located in Device Drivers → Network device support → Universal TUN/TAP device driver support.

Using Hamachi

Now that hamachi is on the system we need to start using it. The server-wide installation doesn't require this (but I'm sure you can use this method to create a configuration usable by the server-wide instance) but the user specific usage does.

Starting hamachi is as simple as the following:

hamachi start
hamachi login
hamachi create NETWORK [ PASSWORD ]
hamachi join NETWORK [ PASSWORD ]
hamachi go-online NETWORK

That's it. You're now connected to a private network named NETWORK. You can view who else is connected to your network with hamachi list and hamachi get-nicks.


Setting up a VPN can be daunting (see the OpenVPN configuration documentation) or it can be a breeze with hamachi. Need a quick VPN for LAN gaming or a VPN for performing maintenance over The Internet on a device behind a firewall? Hamachi may be the quick solution you're looking for.

How To: Deliver and Forward with Postfix


At times it would not only be nice but also might be necessary to have the ability to locally deliver mail for a domain as well as forward those same mail messages to another domain (migrations to other services, &c). How does one go about configuring postfix in such a way that fulfills these requirements?


Luckily the BCC field of e-mail allows us to accomplish this quite elegantly. The postfix parameter we'll take advantage of is recipient_bcc_maps. By setting this to a regexp lookup table, we're able to add a BCC to all messages destined for a particular domain passing through this instance of postfix:

recipient_bcc_map = regexp:PATH

After we've added the above to our postfix configuration, we still need to create the regexp mapping file with lines similar to the following:

/(.*) ${1}


Setting up a deliver and forward configuration on postfix is very simple and requires little consideration about the mail delivery process itself.


  1. Auto BCC

An Explanation of System Load Average


Often in administration, people make reference to a magic number known as load average but it's not always clear what this number actually means. Most of the time its simply an indicator of whether or not the server is under duress. This number doesn't have a whole lot of detail embodied in it but it's intended to be a one-glance check on server health.

Calculation of Load Average

The load average is an exponentially damped/weighted moving average that is similar to a running n² average. This number is calculated in accordance with the jiffies the kernel is tracking rather than the clock ticks. Every time slice (which can but shouldn't be tweaked in the kernel) the system load average is updated. This rolling average allows us to keep a minimal amount of information on hand and still have an average since boot.

The load that gets placed into this average algorithm is simply a count of the number of processes in the run queue at that instant. Thus, since processes who are waiting on I/O (those in the D state) need to periodically check in or wait to be woken up by the kernel; these processes can contribute to the count of processes in the run queue. Since these processes aren't taking CPU time but are taking space in the run queue they can increase the apparent load on the server (thus bringing in the I/O wait of the system into the load average) without raising the CPU usage time.


The load average doesn't tell you a whole lot of information but coupled with information from iostat or your CPU usage you can quickly use it to guage whether your server is falling down under it's load. Since it reports the number of processes in the run queue it is safe to assume a reaonably efficient use of hardware would dictate you want N + 1 (N being the number of cores in the system) as your load average.


More information, including source code for this calculation, can be found on the wikipedia page.

Anexcellent reference on the how the Linux process life cycle works is Robert Love's Linux Kernel Development. Another reference on the process life cycle is the KLDP Wiki

An Explanation of MySQL SELECT Statements


SQL SELECT statements have the following form:


The way SQL processes these directives is slightly different:


FROM Clause


This lets SQL know which tables are being used for the query and any aliases they might be referenced as in the query. The aliases are only necessary when an inner join is performed.


FROM table1, table2
FROM table1 AS t1, table2 AS t2
FROM table1 AS t1a, table1 AS t1b

WHERE Clause


This lets SQL know which rows should be selected from the table based on the CONDITIONS passed. The CONDITIONS can also be combined with the logical OR and AND operators. (Which should be properly parenthesized to demonstrate priority.)

The following operators can act on columns in a where clause: =, <> (also !=), >, <, >=, <= BETWEEN, LIKE, IN.


WHERE t1.c1 = "string"
WHERE t1.c1 = t2.c1 and t1.c2 <> t2.c2
WHERE t1.c1 = t2.c1 and (t1.c2 <> t2.c2 or t1.c3 <> t2.c3)
WHERE t1.c1 in (value set)

This last example demonstrates using either an explicit set or a subquery where the subquery returns a list of values that filter this main query.



This lets SQL know to group the table for aggregate filtering operations. For example, if you need to sort the results based on the max sales of your sales people you could GROUP BY your sales members' identifiers and then use a HAVING clause to filter based on their max(sales_amount). This clause is only really useful when a HAVING clause is useful.

Multiple GROUP BY columns just restricts the groupings to be tighter and tighter. For example, if you have three columns t1, t2, and t3 and you use a GROUP by t1, t2 you will end up with the following groupings:

  • t1 and t2 are all the same in this group; t3 varies
  • t1 and t2 are all the same in this group; t3 varies


GROUP BY c1, c2



This is very similar to the WHERE clause and the logical operators AND and OR can be used as in the WHERE clause. The difference here is that typically you'll be filtering based on an aggregate operation on an ungrouped column to filter out groups.


HAVING max(t3) > N
HAVING average(t3) BETWEEN x AND y



This lets SQL know you want to sort the specified columns in ascending or descending order. The sorting will be applied to the columns in the oder that they are specified. Thus, it works similar to the way groups work; it makes groups out of the first specification and the the second, &c. Performing the new operation only within the context of the previous. Thus, the following data would be sorted as shown:

  • Before Sorting:

    c1 c2 c3
    aa aa aa
    bb bb bb
    aa dd ee
    cc cc cc
    aa bb cc
  • After Sorting (ORDER BY c1, c2, c3):

    c1 c2 c3
    aa aa aa
    aa bb cc
    aa dd ee
    bb bb bb
    cc cc cc


ORDER BY c1, c2, c3



This lets SQL know which columns (or what projection) of the table to actually display. One can also specify aggregate functions here to perform functions such as counting, averaging, &c.


SELECT col1, col2
SELECT col1, t1.col2


Remember that SQL SELECT statements are not processed in the order that they are parsed. This will simplify the query building process to think of it as operations on a set of data (since that is what it is). The steps are as follows:

  1. Select the set to act on.
  2. Filter out the elements from the set.
  3. Group the remaining elements.
  4. Filter out groups of elements.
  5. Sort the elements.
  6. Get the projection of the elements' attributes.

All of this somehow translates to the SQL SELECT statement syntax we started this discussion with:



RPM Build: Gearman-0.14 for CentOS-5.5


I was recently tasked with working on getting PHP 5.3 installed with Gearman on CentOS 5.5. I've learned quite a few of the pains of working with RPMs and have reminded myself why I don't work with RHEL on a regular basis (for personal items anyway). I have had some success in getting everything working correctly and the following is the quick easy way to get this done on an x86_64 CentOS 5.5 install (although other RHELs should work as well).

Step 1: Upgrade PHP

First, we need to upgrade PHP from the IUS Community Repository. Once you have the EPEL and IUS repositories installed and working, you simply run these commands:

yum remove php
yum install php53

If the second yum command complains, simply remove all php packages installed (i.e. rpm -qa | grep ^php) and install the corresponding php53 packages.

Step 2: Install Gearman

I recompiled the Fedora source RPMs (with a slightly modified spec file) to get Gearman to play nicely with the CentOS 5.5 environment. These RPMs are available in RHELL. Install these RPMs and you'll have the gearman installation required to install the PHP bindings for Gearman.

If you have any issues, please let me, Alex Brandt, know.

Step 3: Install PHP's Gearman Bindings

The easiest part of this ordeal is to simply install the PECL package for the PHP Gearman bindings:

pear install channel://


If you must use RHEL, this guide should help you get gearman running with PHP.

Updating Bacula — Database Issues


Backups, a subject always talked about but rarely put into practice. If you do happen to have a backup solution, excellent, you probably sleep easier than most knowing your data is safe.

I've been using bacula for my backup solution for over a year now and one thing I've never had enough experience with has been upgrades. The upgrade process can be pretty hairy depending on your distribution, level of investment, &c.

Updating Bacula

Recently, bacula-5.0.2-r1 was marked stable in the portage tree (Gentoo). Upon finishing the emerge, I attempted to restart all of the bacula services to bring the new version live:

/etc/init.d/bacula-sd restart
/etc/init.d/bacula-fd restart
/etc/init.d/bacula-dir restart

The Problem

All went well until I performed the restart on the directory (bacula-dir). At that point things took a turn for the worse. The director didn't want to start and there were no messages on the screen indicating why this might be the case.

The Solution

The database needed some schema changes to be applied before the director could start again. In order to determine this I ran the following:

bacula -u root -g bacula -c /etc/bacula/bacula-dir.conf -fvm

This finally output the magical answer: update the database schema a couple of versions. It's important to know which versions you're dealing with as you must run each update to the database individually.


Back up the databases you're about to modify, just in case.

The database update scripts are located in /usr/libexec/bacula/updatedb/ and when run in the correct order will get you back up and running in short order. Afterwards, restart the bacula-dir service and everything should be up and running again.


When updating bacula don't forget the catalog database may have changes that need to be applied as well before the service will restart correctly.

How To: Boot Using GRUB from a Compaq Smart Array RAID Controller


When utilizing GRUB to boot from a Compaq Smart Array there are a few items that aren't commonly encountered and require special configuration. The configuration required is quite simple but can require hours to determine.


These instructions have been tested on Gentoo but should be generic enough to work on all distributions.

GRUB Configuration

I'm assuming that the system is installed and GRUB has been installed as well. A working GRUB configuration is shown below:

System Message: ERROR/3 (<string>, line 20)

Cannot find pygments lexer for language "grub"

.. code-block:: grub

    default 0
    timeout 3

        root (hd0,4)
        kernel /boot/bzImage-2.6.20-gentoo-r8 root=/dev/ida!c0d0p7


The boot partition for the above example is partition five on the first disk.

The interesting thing about this configuration is the exclamation point (!) between the first subdirectory and the device node in the device node's path. Because the device name in the configuration has an exclamation point we need to manually update the device map, /boot/grub/, accordingly. The corresponding device map for the above GRUB configuration is shown below:

System Message: ERROR/3 (<string>, line 41)

Cannot find pygments lexer for language "grub"

.. code-block:: grub

    (fd0) /dev/fd0
    (hd0) /dev/ida/c0d0


If you don't have a floppy it doesn't require a device mapping.

Installing GRUB to the MBR

The last requirement is to install GRUB (with the provided configuration) to the master boot record (MBR).

Begin by entering the GRUB command line interface:

/sbin/grub --batch --device-map=/boot/grub/ --config-file=/boot/grub/grub.conf --no-floppy

Once you're in, use the following commands to install GRUB:

System Message: ERROR/3 (<string>, line 64)

Cannot find pygments lexer for language "grub"

.. code-block:: grub

    grub> device (hd0) /dev/ida/c0d0
    grub> root (hd0,4)
    grub> setup (hd0)
    grub> quit


At this point GRUB should be installed and configured correctly. The only thing left is rebooting the machine to verify that it worked.

An Explanation of Xbox 360 NAT Types

I finally broke down and purchased an Xbox 360 with Halo 3 . It is an incredible improvement over the first system and is backwards compatible. The first intriguing thing about the system from a technologist perspective is their idea of network address translation (NAT) types: Open, Moderate, and Closed. I didn't realise this was possible but after a few packet captures I learned a couple of things:

  • Xbox Live uses port 3047
  • Xbox Live uses UDP
  • Xbox Live thinks Open NAT means UDP port 3047 is directly accessible on the Xbox in question behind your NAT device

This leads me to the conclusion that the Xbox Live and Vista ready devices are just engineered to provide holes in networks (utilizing Universal Plug 'n' Play (UPnP) of course). I guess acceptance of this as a requirement to play is something a security policy can help decide.

How To: Configure vpnc


vpnc is a handy Cisco virtual private network (VPN) client. This allows remote work, secure network connections, &c. Our university uses Cisco VPN services to allow remote access to internal servies (i.e. license servers).

Installing vpnc

vpnc is found in many distributions' package management systems. Before scouring for the sources try searching the package manager. Otherwise, install the source with the following standard build process:

su -c "make install"

The other thing to keep in mind is that vpnc requires tun/tap to be built into your kernel. vpnc will error with a reminder if it is missing when vpnc is first run.

Configuring vpnc

The configuration file needs a few pieces of information before vpnc can connect us to the remote network. Edit /etc/vpnc/default.conf to have the following information:

System Message: ERROR/3 (<string>, line 32)

Cannot find pygments lexer for language "vpnc"

.. code-block:: vpnc

    #Interface name tun0
    IPSec gateway
    IPSec ID wireless
    IPSec secret REMOVED
    Xauth ${USERNAME} # DragonMail user name
    Xauth ${PASSWORD} # DragonMail password (not required)


Only use the password line if you've secured this configuration and don't want to type a password everytime you connect.

Please, e-mail your name and DragonID to Alex Brandt to request the removed secret.

Thanks to Conor Shenk for decrypting the password.

Checking vpnc

Now, if everything is properly configured and installed then all you need to do is run vpnc as root. If this works, we can set it up so that certain users can run vpnc without becoming root. Otherwise, troubleshoot, check the vpnc homepage, &c.

Add Convenience to vpnc

  • Autostart vpnc:

    If we want vpnc to start on system boot we can add it to our rc.local file. We'll also want to add vpnc-disconnect to the shutdown scripts for the system.

  • Unprivileged vpnc:

    If we would like to run vpnc as a regular user, we'll need to setup a system like sudo. Using visudo, add a line similar to the following to the sudo configuration for your user:

    System Message: ERROR/3 (<string>, line 74)

    Cannot find pygments lexer for language "sudoers"

    .. code-block:: sudoers
        USERNAME HOSTNAME = NOPASSWD:/usr/bin/vpnc,/usr/bin/vpnc-disconnect
  • Aliasing vpnc:

    Using aliases can make life even easier. Simply add aliases like the following to a .bashrc file to simplify commands requiring the remote network:

    alias vpnc="sudo vpnc"
    alias vpnc-disconnect="sudo vpnc-disconnect"
    alias idl="vpnc && idl && vpnc-disconnect"


Using vpnc is quite simple and provides convenient access to remote resources securely.