To change the Apache log rotate frequency, you would want to change the apache2 configuration file located under /etc/logrotate.d. Configurations located in this directory overrides the default settings in /etc/logrotate.conf (the configurations in /etc/logrotate.d are included in logrotate.conf)

Here’s what my settings look like (note that I had commented out compress option so that all the logs can be easily browsed):

/var/log/apache2/*.log {
daily
missingok
rotate 365
#compress
#delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
/etc/init.d/apache2 reload > /dev/null
fi
endscript
}

Common problem

A common problem after changing the log rotate frequency is that the Apache logs do not appear to be rotating daily even though the configuration file had been changed correctly.

This seems to be a known issue with logrotate. If you read the manual for logrotate you’d see the following explanation:

-f, –force
Tells logrotate to force the rotation, even if it doesn’t think
this is necessary. Sometimes this is useful after adding new
entries to a logrotate config file, or if old log files have
been removed by hand, as the new files will be created, and
logging will continue correctly.

Since the default cron daily job does not force the log rotation, you may want to add -f parameter to /etc/cron.daily/logrotate:

#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate -f /etc/logrotate.conf

With this change, the logs will be forced to rotate daily according to the setup.

Since I had quite a few unofficial packages installed on my primary PC, I was not quite sure how the upgrade would turn out and I was prepared to see a few gotchas at some point. But the upgrade process was surprisingly smooth (albeit slow since all new versions of the installed packages had to be fetched first), and when the upgrade process is done I was able to boot into the new desktop. The only thing I needed to do was to re-install the CUDA driver (190.18), but this was to be expected since every major kernel updates would require the CUDA driver to be re-installed. I certainly hope that NVidia would make their proprietary binary drivers more robust by detecting kernel changes and reinstall automatically if necessary as opposed to require the end user’s manual intervention.

For my other machine running Ubuntu 9.04, the upgrade was equally smooth. One of the problems I ran into there was due to my VMWare Server installation. As it turned out, there are some issues with the default installation package that came with VMWare Server 2. The problem however can be resolved with Radu’s installation script.

So I followed the instructions given in these two blogs: Install Mono 2.4 on Ubuntu,
Building Mono and MonoDevelop from source on Ubuntu 9.04 64bit to re-install Monodevelop 2.0 and its dependencies from the source.

One thing you will need to pay special attention to is that for the monodevelop-debugger-mdb-2.0 to compile successfully, you will need mono-debugger version 2.4, instead of the latest version 2.4.1. Otherwise you will receive an error on Thread.AbortInvocation as there is a breaking change in mono-debugger 2.4.1 which added a parameter to the method (see mono debugger 2.4.1 change log).

I have a lot of important programs on my main PC and I wanted to be extra cautious, so instead of backing up the working harddrive and install Ubuntu 9.04 on top of it, I decided to install it on a new harddrive. This way, I could always fall back to my currently working Ubuntu 8.04 installation should something not working out the way I wanted.

The drive I chose to install Ubuntu 9.04 on was a Western Digital Caviar Black WD1001FALS (1TB). The installation went smoothly and ten minutes later I was able to use the brand new Ubuntu 9.04 desktop.

It took me a while to customize the desktop, install the applications I needed and migrate the data over. But everything pretty much worked out-of-box. And it certainly feels a lot faster than Ubuntu 8.04.

The only minor “glitch” I ran into was my dual sound card setup. As I mentioned in an earlier post, I needed to use a second sound card because my on board Intel sound board does not have a MIDI/Game port. I needed that port to hook up my old MIDI keyboard. After the installation though, the sound playback would only go through the Sound Blaster sound card, not the on board Intel HDA sound card. There are various posts (1,2,3,4,5) suggesting how to fix this issue but unfortunately none of them worked in my case. No matter which sound card I initialized first, I always ended up with the Sound Blaster Live card working but not the on-board Intel sound card. Here is how I fixed it: I removed the Sound Blaster Live card first and booted up with just the on board sound. After making sure that the on board sound worked properly, I plugged in the Sound Blaster PCI card and then everything worked as desired (the sound now comes from the on-board Intel HDA instead of the Sound Blaster card).

As a side note, it seems that WD1001FALS does have some performance advantage over WD6400AAKS (which is the harddrive I run Ubuntu 8.04 on) even though their platter densities are quite comparable (333MB/platter versus 320MB/platter).

This kind of enthusiasm is certainly encouraging as Linux matures. And certainly, this kind of enthusiasm is much needed for the open source community in general. Whether it is proprietary software or free software, the community acceptance is definitely key to the success of its future.

I have three systems running on Ubuntu at home right now. My WordPress server is running Ubuntu 8.04 server (32 bit). One of my working machine is running Ubuntu 8.04 64 bit and the other is running Ubuntu 8.10. For my server, I will stick to the 8.04 as it is an LTS release. But I am thinking of installing the latest Ubuntu desktops onto my workstations in the next couple of weeks…

Update 1 8:00 PM
I have just installed Ubuntu 9.04 using the ISO image I downloaded earlier onto a VM. The installation was made even easier (the time zone is now selected by default). The default installation took just about five minutes. The boot time is also very impressive. It took just about 20 seconds before showing the login screen, which is a big improvement over my 8.04 installation.

Update 2 9:00 PM
Even though a fresh installation might be ideal, I decided to run the update on my Ubuntu 8.10 machine anyway. This is mainly because it was used as a file and VM server and I had done very little customizations on it besides that. So I think the chances of getting a successful upgrade is pretty high. The distribution upgrade is pretty slow though, and it is downloading at about 30k/s right now…

Upgrading from Ubuntu 8.10 to 9.04

Update 3 4/24/2009 7:00 AM
The 8.10 to 9.04 upgrade went surprisingly well. The only problem I had has nothing to do with Ubuntu.

Since the machine I was doing the update on is a file and VMs server, I did not bother attach display or keyboard to it. So when I was doing the update yesterday used an SSH connection and forgot that the updates would need to shutdown SSH and other services in order to install the newer version of the daemons. Anyway, half way through the upgrade, the connection to my server was lost. At first I thought that I could just attach a monitor and keyboard to my server and continue the whole upgrade process. Of course it wouldn’t work since the upgrade process was tied to that particular terminal.

So I had to reboot the machine. For whatever reason, maybe because the machine was only half upgraded, the normal boot does not work and I could not log in via the graphical login screen. I restarted in “safe” mode dropping into a root shell and did a

sudo dpkg –configure -a

And the upgrade process picked up from where it was left at before. This is truly impressive! After the upgrade process finished, I rebooted into the new 9.04 installation:

sudo lsb_release -a

Distributor ID: Ubuntu
Description: Ubuntu 9.04
Release: 9.04
Codename: jaunty

The only thing that did not work after the upgrade was the VMWare Server (which is totally expected as it needs to link to the correct version of libraries). But after a simple re-config, the VMWare Server came back into life.

So naturally, I wanted to revert back to my previous working version (SVN 5489). There are some instructions on lgp203.free.fr that in theory should work to install any revisions of Code::Blocks from SVN. But the script provided there did not work for me as it was complaining about not able to find certain packages. So I decided to do it manually, and here are the steps I took to get SVN 5489 installed.

First, you will need to remove the installed version of Code::Blocks.

sudo apt-get remove codeblocks
sudo apt-get remove codeblocks-common

Then you will need to install the Debian packages one by one from lgp203.free.fr. using

wget http://lgp203.free.fr/pool/codeblocks/wxsmith-headers_8.02svn5489_all.deb
wget http://lgp203.free.fr/pool/codeblocks/libwxsmithlib0_8.02svn5489_amd64.deb
wget http://lgp203.free.fr/pool/codeblocks/codeblocks-headers_8.02svn5489_all.deb
wget http://lgp203.free.fr/pool/codeblocks/libcodeblocks0_8.02svn5489_amd64.deb
wget http://lgp203.free.fr/pool/codeblocks/codeblocks_8.02svn5489_amd64.deb
wget http://lgp203.free.fr/pool/codeblocks/codeblocks-common_8.02svn5489_all.deb
wget http://lgp203.free.fr/pool/codeblocks/codeblocks-contrib_8.02svn5489_amd64.deb
wget http://lgp203.free.fr/pool/codeblocks/codeblocks-contrib-common_8.02svn5489_all.deb

sudo dpkg -i wxsmith-headers_8.02svn5489_all.deb
sudo dpkg -i libwxsmithlib0_8.02svn5489_amd64.deb
sudo dpkg -i codeblocks-headers_8.02svn5489_all.deb
sudo dpkg -i libcodeblocks0_8.02svn5489_amd64.deb
sudo dpkg -i codeblocks_8.02svn5489_amd64.deb
sudo dpkg -i codeblocks-common_8.02svn5489_all.deb
sudo dpkg -i codeblocks-contrib_8.02svn5489_amd64.deb
sudo dpkg -i codeblocks-contrib-common_8.02svn5489_all.deb

You can find a full list of all the available Code::Blocks’ SVN versions here. Now, I am running version 5489 and everything is working again.

Since my MIDI keyboard uses the game port interface on my Sound Blaster Live! sound card, there is no need to load any firmware. After the sound card is recognized by the system, you will find a /dev/midi device (for some reason, my midi device would change between /dev/midi and /dev/midi1 once a while, but this only happens when I access the midi device using jack). Without any additional software, you should be able to do a cat /dev/midi and see some "random" characters printed on the console whenever you press a key on the MIDI keyboard. Even without touching the MIDI keyboard, you should still see characters get printed in the console at a rate of 3 to 5 characters per second. Alternatively, you could use amidi –dump and when a key is pressed some hex values should be displayed.

After confirmed that the MIDI device is working properly, we can move on to setting up the software environment. Most of the aforementioned articles are concentrated on connecting to MIDI devices using ALSA. What I have found out though is that using OSS is much easier.

Various sources have suggested that connecting MIDI devices using ALSA have very low-latency (e.g. the time between when a key is pressed and the note is played) and thus is the preferred method. But for modern computers, this kind of latencies have rarely been a problem. There might be some difference in terms of latency between using ALSA and using OSS but they are not observable in my case.

To use OSS, all you need to do is to choose the correct option in qsynth:

qsynth settings 1

qsynth settings 2

If you have multiple sound cards configured like in my case, you can select from which sound card you want the sound to be outputted (e.g. /dev/audio or /dev/audio1 etc.). The sound does not have to be produced on the same card where the MIDI device is connected. In fact, using a single sound card as the output device might be the most attractive solution since otherwise you will need to have one pair of speakers connects to each of the soundcard output. In my case, the Sound Blaster Live! card is only used for it’s game port to connect with my MIDI keyboard.

After you have successfully restarted qsynth, you should be able to hear sound when you press on the MIDI keyboard and you should see the indicator on qsynth flashing, indicating that MIDI signal was received.

qsynth

If you see activities from qsynth but there is no sound coming out then chances are that you have chosen the wrong sound card as the output device. Note that you will have to first load a Sound Font into qsynth, otherwise you will not hear any sound either. A few popular Sound Fonts can be downloaded from here.

MIDI Setup with ALSA requires a bit more work. First you need to start the jack daemon (jack -d alsa), or you could use the GUI Jack Control. Then you can start rosegarden. If you have timidity installed you you should be able to use timidity’s port (usually 128:0) as midi device port. If you are not sure, you could use pmidi -l to find out (this shows what my configuration looks like):

 Port     Client name                       Port name
 14:0     Midi Through                      Midi Through Port-0
 20:0     SBLive! Value [CT4780]            EMU10K1 MPU-401 (UART)
 21:0     Emu10k1 WaveTable                 Emu10k1 Port 0
 21:1     Emu10k1 WaveTable                 Emu10k1 Port 1
 21:2     Emu10k1 WaveTable                 Emu10k1 Port 2
 21:3     Emu10k1 WaveTable                 Emu10k1 Port 3
128:0     TiMidity                          TiMidity port 0
128:1     TiMidity                          TiMidity port 1
128:2     TiMidity                          TiMidity port 2
128:3     TiMidity                          TiMidity port 3

You can use either the Emu10k1 ports (21:x) or TiMidity ports (128:x) as output.

Since Sound Blaster Live! utlizates hardware based SoundFont, you must load appropriate SoundFont onto the board before you can hear any playback. Due to the on-board memory limitations, not all SoundFonts can be loaded onto Sound Blaster Live! Here is an 8MB version of the Creative Sound Font that you can use with Sound Blaster Live!. To load the sound font use the following command:

sfxload 8MBGMSFX.SF2 (assuming that the Sound Font is located in the current directory)

For more detailed MIDI setup using ALSA you can refer to the two articles I mentioned. What I have found is that the MIDI environment using jack is not always stable. For instance, I could instantiate the ALSA sound driver using jackd but not directly with jack GUI. And once a while jack would lock up. Another problem is that the MIDI device name seemed to change back and forth between /dev/midi and /dev/midi1 whenever I was using jack. This might be because the fact that I am using two sound cards though. Anyway, since the performance of using OSS turned out to be quite good I did not investigate the ALSA issue much further…

Ubuntu 8.04 recognized the second sound card with no problem. A cat /dev/sndstat confirmed that both sound cards are indeed installed:

Card config:
HDA Intel at 0xf9ff8000 irq 22
SBLive! Value [CT4780] (rev.7, serial:0×80221102) at 0xe880, irq 18

And lsmod | grep snd showed that midi modules were successfully loaded:

snd_rtctimer, snd_emux_synth, snd_seq_virmidi, snd_emu10k1, snd_ac97_codec, snd_seq_dummy, snd_seq_oss, snd_hda_intel, snd_pcm_oss, snd_mixer_oss, snd_pcm, snd_rawmidi, snd_hwdep, snd_seq, snd_timer, snd_seq_device

So the dual sound card setup is pretty effortless. Getting the MIDI keyboard setup correctly though turned out to be quite tricky and I will share my experience in a later post.

Typically a large computer cluster has a single head node and many compute nodes. These nodes are interconnected via a fast dedicated network (e.g. Gigabit Ethernet or Infiniband). The head node typically has two network adapters which bridges the WAN (or Internet) and the cluster.

For a two-node Beowulf cluster setup however, the above requirements (e.g. one head node and many compute nodes) can be greatly relaxed. Particularly in a two node environment, we do not have the luxury of having a dedicated head node and we have to use both nodes as compute nodes in the cluster. As a result, one of the two nodes will be both the head node and the compute node. As you will see later, the difference between the head node and the compute node in a two node cluster is not all that significant. Furthermore, since the cluster sits behind a firewall, there is no need to bridge the head node from the outside world via a second network adapter. The DSL router is sufficient to isolate the LAN from the Internet. This drops the two NIC requirements.

1. Choosing an MPI (Message Passing Interface) package
There are primarily two popular MPI implementations: MPICH and LAM/MPI. Some technical and performance differences exist between these two implementations (you can find more information in this research paper. This link has some high level information). Since MPICH2 implementation relies solely on SSH and does not require a daemon running on each machine in the cluster, it is presumably easier to setup in an environment that involves a large number of nodes. But for my particular setup, either implementation should serve the purpose. I chose to use MPICH2.

2. Setting up the MPI Environment
The two PCs (zeta, sigma) I use for the cluster both have a quad core processor (Q9450) and 8GB of RAM. The hardware configurations are slightly different (one Motherboard is based on Intel X38 chipset and the other one is based on P43) and the OSes are slightly different as well (one is running 64bit Ubuntu 8.04 and the other is running 64bit Ubuntu 8.10). None of these differences are material though.

In my setup, I used zeta (Ubuntu 8.04) as the head node since that is where my primary desktop is on. My new built (sigma, Ubuntu 8.10) runs in headless mode and serves as a compute node.

There is a lot of information on the Internet on how to setup an MPI environment. But most articles are rather dated. For Ubuntu cluster setup, this article is pretty much up-to-date (the article mainly targets Ubuntu 7.04 and there are some places that need to be updated and I will point those out as we move along). Some of the best references on MPI environment setup come from the MPICH2 distribution tar ball:

mpich2-1.0.8/doc/installguide/install.pdf

mpich2-1.0.8/doc/userguide/user.pdf

It definitely pays to read them through prior to setting up the environment. There are many useful trouble shooting tips in the installation guide as well, and they will come in handy should you run into any setup issues.

3. Setting up NFS on the Head Node
The installation guide that came with the MPICH2 distribution points out that NFS is not necessarily required for MPICH2 to function, but it would be necessary to duplicate the MPICH2 installation and all the MPI applications in the same directory structure on each node. Things become so much easier if all the nodes can use the same NFS directory where MPICH2 installation and MPI applications are. See Step 2 in the MPICH Cluster article.

4. /etc/hosts, mpd.hosts and SSH Setup
As indicated in the article I mentioned earlier, we need to change /etc/hosts on both nodes to contain the IP addresses of each server.

Pay special attention to the 127.0.1.1 address that points to the server name. For MPICH2 to work properly, this IP address must be changed to the real IP address of that server. For instance, here is the /etc/hosts on zeta:

127.0.0.1    localhost

#127.0.1.1    zeta

192.168.1.67    zeta

192.168.1.68    sigma

And similarly here is the /etc/hosts file on sigma:

127.0.0.1       localhost

#127.0.1.1      sigma

192.168.1.67    zeta

192.168.1.68    sigma

You will also need to create a mpd.hosts file on each of the nodes within the user’s home directory (note, this is the user you wish to run mpi under). Here is my mpd.hosts (I only have two machines in the cluster):

zeta

sigma

MPICH2 requires trusted SSH among compute nodes and head node. Please see Step 7 in this article I mentioned earlier. Note that in Ubuntu 8.04 and 8.10 the authorized keys file is called authorized_keys2 and the actual keys generated using ssh-keygen by default are id_dsa and id_dsa.pub. It does not really matter what type of keys you choose. Either DSA or RSA should be fine. This step should be repeated on each cluster node and the id_dsa.pub generated on one machine should be added to authorized_keys2 on the other machine. In my setup, I added the id_dsa.pub on zeta to the authorized_keys2 on sigma and vise versa. This is where we make distinction between  head node and compute nodes. Since head node needs to communicate with all the compute nodes, it needs to have all compute nodes’ public keys. Whereas each compute node only needs to have the head node’s public key. As you can see, in a two-node scenario, either node can be used as the head node.

5 Building and Installing MPICH2
The MPICH Cluster article on Ubuntu documentation has some great details on this so I am not going to repeat here. One thing you need to pay special attention is the .mpd.conf file (located in user’s home directory).

This file is used to make the communications among cluster nodes more secure. To ensure that computers within the same cluster can communicate with each other, each machine must have the same secret word. If the secret word does not match, you will receive a handle_mpd_output error (e.g. failed to handshake with mpd on <node name>; recvd output={})

6 Test Run
On zeta (head node) export the NFS node

exportfs -a

On sigma (compute node) mount the NFS exported from zeta on the same path so that the directory structures on both machines are the same.

sudo mount zeta:/mirror /mirror

On zeta (head node) start up the mpd on both nodes according to mpd.hosts

mpdboot -n 2

Right now running mpdtrace on either machine you should see both nodes get listed.

Running the interactive version of cpi on one core with an interval number of 100,000,000

mpiexec -n 1 ./icpi

it took 1.93 second to complete.

Running with all 8 cores with the same paramter:

mpiexec -n 8 ./icip

took only 0.31 second which is more than 6 times faster than the signle core version.

The following figure recaps the structure of my cluster setup:

The performance improvement in Flash 10 should bode well in the Linux community as native flash performance had been one of the major issues for users to switch from Windows to Linux.

To get the driver built on Ubuntu 8.04 however, I needed to make some minor changes to epcam.c. The kernel version I was compiling against is 2.6.24-19-generic. To get the 0.7.3 driver build successfully, I needed to comment out #include <linux/config.h> and then ran

KBUILD_NOPEDANTIC=1 make install

If you do not want to setup the environment variables for the build, you could just modify the #include statements to where the kernel header files are located:

After the build the driver is automatically installed and when the webcam is plugged in, it should be recognized by apps such as camorama (note: /dev/video0 is automatically created when the webcam is connected).

While in the past I have always had machines running Linux (or FreeBSD) at home, my primary PC had always been a Windows PC. But when I made the switch to a true 64-bit environment, Linux made perfect sense. My old PC was running Windows XP 32 bit, in order to take advantage of the 8 GB memory installed in my new machine, I had to move to a 64 bit OS. So my choices were either to buy a copy of Windows XP 64 bit (Vista? No thanks.) or use the freely available 64 bit Linux. And the choice was quite easy to make.

So, I decided to go with the most popular distro – Ubuntu. In fact I could have easily chosen openSUSE, or even FreeBSD (which is based on BSD UNIX) since my previous experience with them were quite positive as well. At first I did have some question whether or not I would be able to do the majority of my work on Linux, without having to constantly turning to a Windows PC. As it turned out, those worries were totally unnecessary. For the past two months, I had been using Linux almost exclusively at home except for a few moments when I needed to test something in Visual Studio .Net.

For web browsing, I use Firefox 64 bit, since there’s no 64bit Flash available for 64 bit Linux yet, I installed the 32 bit version as well. The default email client GNOME supplies is Evolution, which is quite powerful. Unfortunately, it could only import my outlook mail hierarchical Inbox folder structures into a single folder. So for my mail client, I chose Mozilla’s Thunderbird instead. For word editing, OpenOffice 3.0 is an excellent choice. In fact, I personally found it quite comparable to Microsoft Office 2007, and most Office documents can be opened/edited without any issues.

Since I like many of the tools provided in KDE, I installed the KDE environment as well (the alternative would be to install Kbuntu first and install GNOME based tools later, but it really doesn’t matter, this is the beauty of the versatilities of Linux). The default simple text editing tool (gedit in GNOME/kedit in KDE) is quite powerful compare to Windows’ notepad. The syntax highlighting capability is built in. For viewing PDF documents, I use kpdf, the KDE version of the PDF/PS file viewer. If you are used to the sluggish performance of Adobe’s Acrobat Reader, you’ll be surprised to find how snappy kpdf is.

I use K3b (KDE’s CD/DVD creator) to burn CD and DVD discs. K3b’s features rival most of the commercial CD/DVD authoring programs in the Windows world. In terms of media playback, Ubuntu handles almost all media formats out-of-box, even proprietary formats like wmv can be easily accommodated with a single mouse click. And for photo editing, I use GIMP, which is rather easy to use and almost as powerful as Photoshop.

For software development, I use Code::Blocks and KDevelop for C++ applications, MonoDevelop for C# programs. Occasionally I needed to research some work related programming issues. Since some of these issues are Windows specific, I would fire up an instance of Windows XP (or Windows Vista) within VMWare Server and use Visual Studio from there.

There hasn’t been a single instance where I couldn’t do something conveniently in Linux for the past two months. In fact, for the majority of the stuff I do I felt that Linux was actually easier. There are a few rough edges, of course. For example, copy and paste sometimes do not play well between GNOME and KDE applications and the Flash play back performance under Linux is far inferior then under Windows. But it does not matter to me that much (I block flash content on my Windows PC anyway). 

During the past couple of weeks, I have been playing with Ubuntu 8.04 (Hardy Heron) beta version and I was pretty happy with the results so as soon as Ubuntu 8.04 was released last Thursday, I downloaded the image and installed on my PC.

I am happy to report that Ubuntu 8.04 recognized all the hardware (see the list here) with no issues. And the 2.6.24 kernel fully supports Q9450 and X38 chipset. The included proprietary new nVidia driver supports my 8600 GT graphic card with no problem and I could run Beryl desktop smoothly.

There are a few rough edges however. In beta versions, when I booted Ubuntu, the startup logo was correctly stretched to fit the entire screen (wide screen), but the production code somehow swapped to a 4:3 mode which is kind of annoying. The major issue though is with nautilus. Everything works fine in default mode (icon view), but as soon as I change the settings to list view, dragging and dropping would cause the whole system to lock up. I noticed this behavior in beta builds and had thought that it would be resolved in the production version. So for now, I will use the default setting.

To use MATLAB 7 under WINE, all one has to do is to copy the harddrive serial number over into ~/.wine/drive_c/.windoes-serial. Here I assume that MATLAB has already been installed under Windows C drive.

The serial number is the disk serial number detected during MATLAB’s installation and can be found under [MATLAB root folder]/bin/win32/license.dat. The value after HOSTID=DISK_SERIAL_NUM= is the serial number.

After setting the disk serial number, MATLAB should be able to be invoked from the shell:

wine /windows/C/MATLAB701/bin/win32/MATLAB.exe

Here is a screen shot of MATLAB 7 running under Suse 10.3 (Gome):

And if the serial number is set incorrectly, the following message will be received instead:

However, sometimes it is desirable to have a standby instance of the machine. For example, in a production environment, if a standby server is hot-synced with the production server then in the event of a production server failure, the standby server can kick in right away and the downtime can be reduced to as little as a couple of minutes. Whereas if a system restore is required, the production server has to be re-imaged and that can take significantly longer.

Fortunately, it is relatively easy to achieve this goal. In this post I will show you how to backup a Ubuntu Server (7.04) system to a Virtual Server. It only requires a few more steps then the plain vanilla TAR backup/restore.

• Backup

The backup command is almost identical to the command used in a standard backup (e.g. the backup is used to restore to the same server). Because the backup machine might not have the same physical specs as the machine where we perform the backup, we need to exclude a few more items. Particularly,
/dev, /etc/fstab, /etc/hostname, /etc/iftab might be different on the backup machine and we need to be careful not to overwrite them when restoring. An example of the TAR command used for backup a system is listed here:

tar -czf /mnt/backup/sysbackup/backup.tgz –exclude=/proc –exclude=/dev –exclude=/lost+found –exclude=/mnt –exclude=/sys –exclude=/cdrom –exclude=/etc/fstab –exclude=/etc/hostname –exclude=/etc/iftab /

• Preparing the Backup VM

Here I use a virtual machine to illustrate the restore-to-a-different machine strategy. In practice though, this can also be a physical machine.

First, you will need to prepare a minimum VM that has the desired OS on it (in my case Ubuntu Server 7.04). The OS is needed because we are using TAR to do the backup and restore. It should be pretty easy though as most server version Linux images support unattended setup.

You do not need to install any security updates afterward as the system will be brought up to date during the restoration process.

• Resotring Backup Image to VM

 Now copy the backup image over to the VM. Because certain files can only be overwritten when they are not in use, we will perform the restoration in single user mode:

init 1

next we will restore the backup image:

tar -xpzf /backup.tgz -C /

DO NOT REBOOT AFTER RESTORE! If you reboot right after the restoration, the system will not be in a bootable state (e.g. system drive’s UUID has not been updated yet, if you reboot now you will not be able to boot into your system. In this case you will have to use a live CD to boot and fix your system from there).

First thing we need to do is to set the correct UUID for the partition we are booting from.

Do a df to see which partition is the root partition, e.g.

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda1            113892200   1007084 107099712   1% /

Find out the UUID of the partition:

dumpe2fs /dev/sda1 | grep UUID

Write down the UUID of the partition and double check to make sure that it is written down correctly as wrong UUID will lead to a boot failure.

Now, we need to edit /boot/grub/menu.lst to update the boot partition’s UUID.

In menu.lst, find entries like:

title           Ubuntu, kernel 2.6.20-16-server
root            (hd0,0)
kernel          /boot/vmlinuz-2.6.20-16-server root=UUID=401eeff8-c149-4db6-b4df-6cd899b880c6 ro quiet splash
initrd          /boot/initrd.img-2.6.20-16-server
quiet
savedefault

And change the UUID to the one you just copied down. Again, use extra caution to ensure that the UUID is changed correctly.

Depending on the system, there might be quite a few entries with the same UUID as you might have different kernels installed. If you are not certain, change them all.

 Now you should be able to reboot into the restored system. After reboot, you might not have access to the network any longer due to the MAC change of the ethernet during system restore. This can be fixed easily. Do a

ipconfig -a

you should see an entry with eth0 (or eth1). Copy down the HWaddr (e.g. 00:52:AA:C2:F9:CD) and use this value to update /etc/iftab

eth0 mac 00:52:AA:C2:F9:CD arp 1

Make sure that the ether net interface is updated to eth0.

Now you should be able to use ifup eth0 to bring up the network. see this thread for more detailed discussion on this issue.

One benefit of this backup strategy is that you are always guaranteed that the backup image works correctly, and you can have the backup system on standby using rsync to synchronize the backup machine and your production machine. And in the event of a disaster, your backup machine can kick in immediately.

Restoring the image back to the production machine is actually easier since you can just restore the image back without the need to change any configurations mentioned above (however, it is still a good idea to check).