Testing bunzip2 files quickly using hexdump

My work sometimes involves dealing with large amounts of data stored in .tar.bz2 files. Recently, we found that some of the files were corrupted. The files are large enough that a successful "bunzip2 -t" takes prohibitively long. As a shortcut, we can just check for the bzip magic numbers at the beginning of the file. The bzip2 format requires that each file starts with the string "BZh". Our files are stored in batches by calendar date (MMDD). I generate a log of the files' magic numbers like so:

(for d in ????;
do
  for f in $d/*bz2;
  do
    echo $f `head -1 $f | hexdump -c -n3 | grep '0000000' `;
  done;
done) > magic_numbers.txt

And then I can find the bad files like so:

grep -v 'B Z h' magic_numbers.txt

hexdump is a nice tool for manipulating binary files on the command line. This method of testing the files runs relatively fast.

Enabling hibernate on Ubuntu 12.04 LTS

I recently decided I would prefer to have the option of using hibernate on my laptop running Ubuntu 12.04 LTS. Not only does hibernate need to be enabled manually on any system running that OS; I also had insufficient swap space, so I had to resize my partitions.

Now hibernate would work, but resume would not! It turns out the reason is that I elected to encrypt my $HOME folder during installation. This requires encrypted swap; when you reboot, the swap is encrypted with a fresh, random key each time.

By insisting on using the same key each time, which requires you to enter a password during boot, you enable the boot system to detect and reload the hibernate image. If you forget the password, you boot with no swap partition.

Detailed instructions are here. It worked for me!

Formatting terminal output in Python

From Nadia Alramli's Blog: Terminal Controller for Python. This page provides a simple module for prenting formatted text to the terminal using low-level commands in the curses module.

JACK + Ardour + headphones on a laptop

I use JACK + Ardour for music recording and mixing. For listening, I typically just use a laptop's headphone jack. This can lead to strange and confusing behavior. Well, no more: at least on my laptop, this all can be avoided by connecting Ardour's master out to system:playback_3 and system:playback_4 in addition to system:playback_1 and system:playback_2. This way, the master out goes to all the software channels, and somewhere, Ubuntu decides whether the headphones are plugged in and only sends the output to the correct place.

Saving executable permissions with Subversion

After getting angry at Subversion for failing to save "u+x" permissions on my version-controlled scripts, I finally Google'd for the solution. Turns out, it's simple. Just as you use "svn move" rather than "mv" to move a versioned file, you use "svn propset" to make a script executable:

$ svn propset svn:executable ON [filename]
$ svn commit

Thanks to John M, who says he referred to this page (see also these details).

Tell me when!

Today I was waiting at school for a RAID to be ready to be mounted. I got tired of trying the "mount" command over and over; the computer should tell me when it's ready. I ended up writing this python script to tell me when it was ready:

#!/usr/bin/env python
# tell-me-when

import os
import sys
import time

from optparse import OptionParser

usage = '%prog {[options]} [command]'
parser = OptionParser (usage=usage)
parser.add_option ('-d', '--delay', dest='delay', default=5,
        type=float, metavar='T', help='delay T seconds between tries')
opts, args = parser.parse_args ()
command = args[0]

while True:
    result = os.system (command)
    if result == 0:
        zenity_command = 'zenity --info --title="Success!" ' \
            + '--text=\'Command "' + command + '" has completed successfully.\''
        os.system (zenity_command)
        break
    else:
        print '%s - "%s" still fails...' % (time.ctime (), command)
        time.sleep (opts.delay)

I called it like so:

$ sudo tell-me-when -d 10 'mount /mnt/i3store0/'

The script tries running the given command until it returns an exit status of 0, at which point it displays a message box informing the user that the command has succeeded. In the mean time, it prints a message in the shell with the current time for each failed try.

The program could easily be extended to play a sound upon success, but I haven't implemented that yet since my work computer doesn't have speakers.

Timing a shell script

Time some shell script code without the "time" command:

before=`date +%s`

# code goes here

after=`date +%s`
elapsed_seconds="$(expr $after - $before)"
echo Elapsed time: $(date -d $elapsed_seconds +%H:%M:%S)