Category Archives: Sysadmin

All about Linux Systems Administration

Backup Your Dropbox Files with rdiff-backup

The Problem

Teresa and I use a single Dropbox account to share files between our computers. I also use the same account to store (and sync) plain-text notes on my iPad and iPhone (I use the apps PlainText and iA Writer). In case things go wrong with these apps, the syncing, or with Dropbox itself, I want to backup my Dropbox files and keep past snapshots of the backups so I can go back in time.

The Solution

rdiff-backup can do this. It is a command-line tool written in Python that:

…backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup.

To make this all happen, I have Dropbox installed, signed-in, and running on my Linux desktop/server, which runs Ubuntu 11.04 Natty with Gnome 2.

Install rdiff-backup thusly:

    # aptitude install rdiff-backup

I use the directory /backup/ to hold all my backup targets, so I can run rdiff-backup like this:

    $ rdiff-backup  \
        --exclude $HOME/Dropbox/.dropbox \
        --exclude $HOME/Dropbox/.dropbox.cache \
        $HOME/Dropbox /backup/Dropbox

Every time I run rdiff-backup like this, it creates a new snapshot of my Dropbox files. Old snapshots are kept until I decide to purge them (if at all). To purge any snapshots older than two months, for example, I run this command:

    $ rdiff-backup --force --remove-older-than 2M /backup/Dropbox

I run the above two commands in an @hourly crontab script to keep this all happening automatically.

Browsing Past Snapshots

rdiff-backup has its own commands for digging into the files stored in the past snapshots, but it requires exactly knowing the filenames and backup times. Another tool, rdiff-backup-fs solves this problem by mounting the rdiff-backup backup directory as a FUSE filesystem, allowing me to grep and find my way through a directory tree of all snapshots.

After installing FUSE and rdiff-backup-fs, I mount my Dropbox snapshot tree with this command:

    $ rdiff-backup-fs ~/mnt /backup/Dropbox

Note that the order of the arguments for mounting source and target are backwards compared to the canonical mount command.

A long listing of my 10-oldest snapshots looks like this:

    $ ls -lF ~/mnt/ | head -10
    total 0
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T05:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T06:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T07:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T08:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T09:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T10:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T11:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T12:00:01/
    dr-xr-xr-x 1 root root 4096 2013-03-10 13:52 2013-01-06T13:00:01/

I can then explore all my snapshots at once with any tools wish.

When done, I unmount the rdiff-backup-fs filesystem with:

    $ /bin/fusermount -u ~/mnt

Emailing cron output to different users

When a cron job generates output on either stdout or stderr, the output gets mailed to the owner of the crontab. Now, you can specify an alternative email address by setting the MAILTO environment variable in the crontab, but this applies to all jobs in the crontab.

So, if you want the output of different jobs to get mailed to different users, then you can just redirect the stdout and stderr of each job to the mail command like this:

13 02 * * *     /bin/backup 2>&1 | /usr/bin/mail -s "Cron <root@exobox> /bin/backup" \

MTR as a combined traceroute and ping tool

mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.” And it’s really cool.

Like ping, it sends “echo” packets from your machine to the target machine to measure latency and packet loss along the network path, but it continuously displays updated statistics in real time as it operates.

Like traceroute, it shows the names or IP addresses of each machine along the network path, also updating these statistics for each machine.

Here’s some (frozen) sample output from the ncurses mode (terminal mode) mtr:

                                My traceroute  [v0.71]
exobox (                                             Thu Dec 21 16:15:01 2006
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                             Packets               Pings
 Host                                      Loss%   Snt   Last   Avg  Best  Wrst StDev
 1.                                         0.0%    20    0.4   0.3   0.2   0.7   0.1
 2.                                         0.0%    20    1.6   1.4   1.0   2.7   0.4
 3.                                         0.0%    19    1.3   1.3   1.0   2.7   0.4
 4.                               0.0%    19    1.1   1.2   1.0   2.7   0.4
 5.                           0.0%    19    1.4   2.6   1.2  16.0   3.3
 6.             0.0%    19    1.4   1.6   1.4   2.5   0.3
 7.                           0.0%    19  199.3  12.0   1.3 199.3  45.4
 8.                             0.0%    19    1.7  35.7   1.3 186.5  56.2
 9.                           0.0%    19    1.8   2.0   1.5   4.8   0.8
10.                            0.0%    19  284.6 292.2 278.3 305.8   8.7
11.        15.8%    19  286.2 291.9 278.6 325.2  11.5
12.        15.8%    19  313.2 323.8 311.3 340.5   8.6
13.          5.3%    19  319.6 331.1 314.1 404.6  20.3
14.         11.1%    19  696.0 718.7 693.3 797.2  33.7
15.         36.8%    19  347.4 354.9 342.5 368.7   8.6
16.        11.1%    19  421.5 423.6 411.1 438.2   7.8
17.        11.1%    19  433.9 438.9 422.4 514.1  22.0
18.                     33.3%    19  423.1 432.7 418.7 463.0  11.4
19.        27.8%    19  405.8 414.7 398.2 434.6  11.6
20.                         23.5%    18  404.8 418.1 399.9 485.8  21.9

Looking at the Avg column (units in ms), the above output shows that my network packets pass through Beijing Telecom’s routers to the U.S., then to the U.K., and finally to their destination in the Netherlands. A large latency increase occurs between lines 9 and 10 (presumably leaving P.R. China), and another between lines 13 and 16 (U.S. to U.K.).

mtr has some interesting display modes besides the above, where it shows the latency of each packet graphically according to a dynamic scale. In this way, the above points of really large latency can be easily detected.

mtr can be obtained from the mtr website, or it can be installed in Debian/Ubuntu by:

# apt-get install mtr-tiny


# apt-get install mtr

for the ncurses or X11 versions, respectively, although mtr-tiny appears to be installed by default in the Debian and Ubuntu machines I have tested. So you may already have it.

How to send an entire domain to /dev/null in Postfix

At Exoweb, our software developers use bogus email addresses of the form * (where I mean “” literally, not as an example) to test their software’s ability to send email. Since I don’t want our Postfix server to attempt to deliver these messages out on the Internet, I need Postfix to handle these messages and blackhole them (make them disappear, sent to /dev/null). So what follows are instructions on how to blackhole an entire domain in Postfix.

First, we add a virtual_alias_maps entry to /etc/postfix/ so that we can specify as one of our virtual domains:

virtual_alias_maps = hash:/etc/postfix/virtual_alias

Inside /etc/postfix/virtual_alias, add a catchall address:      blackhole@localhost

We have to use blackhole@localhost here and not /dev/null/ because virtual_alias_maps cannot run commands—it can only forward to real addresses. So we put an entry inside /etc/aliases to handle the blackhole:

blackhole:      /dev/null

This assumes that one of your mydestination domains in is localhost so that Postfix will actually consult the aliases file.

In order to make these changes take affect, you have to rebuild the aliases database, build the virtual_alias database, and reload your Postfix configuration. Respectively:

# newaliases
# postmap /etc/postfix/virtual_alias
# postfix reload

Now, any emails you send to blackhole@localhost will disappear, and so will any emails addressed to (provided they are relayed through your Postfix server).

Footnote: The top-level and second-level domain names that are reserved for testing can be found in RFC 2606.

Spaghetti Wiring

When you work in a software development shop, spaghetti code is most certainly frowned upon. What about spaghetti wiring?

[Spaghetti wiring]

So this was the remains of the small electrical fire that occurred 15 minutes before I arrived at the office today… the charged main circuit to the room that houses most of our developers, their computers, and the all-important Ice Box.

Do you think this is enough to convince the building management that the building’s wiring needs professional help? No, unfortunately, I think we’ll still have to fight them to get adequate electrical capacity and safety. 🙁

Stop using backup email spooling

Here’s a great opinion piece from the blog of my DNS provider entitled: Want to reduce email spam to your mail server? Stop using backup spooling.

It is with regret that we have come to the following conclusion, but here it is: Offsite backup SMTP spoolers and backup mail exchangers have become worse than useless.

The problem is spam and the software that delivers it exploiting the weak authentication schemes inherent in the SMTP protocol itself. It used to be an annoyance, then it became a concern, it is now an epidemic and has resulted in the death of the offsite backup MX handler.

The author then goes on to explain what the problem is and why you won’t really miss your backup spooling. It’s a very interesting point of view (in a good way) that’s worth considering.


[bsdtalk logo]I just discovered yesterday that there’s a BSD podcast out there. Awesome! It’s called bsdtalk, and has the tagline: “Talking about the BSD family of free operating systems.”

I’ve been using Linux for over a year and a half now, and I’m starting to miss my BSD. This might push me back…

A Hacker’s Vacation

[Head in the clouds]I took this week off work to enjoy a Hacker’s Vacation. That is, I’m planning to spend a lot of time hacking on my computer.

It’s more than that, actually. I desperately need some time to put my life back in order and catch up on things that I’ve been neglecting, such as housework, email, this website, hard drive spring cleaning, my Tchou Tchou’s website, the Swing website, a new server, and various little projects I have going on. Slowly, I’m getting parts of it all done. I’ll have to carry on some of the tasks later, but at least this week will give me a good foundation to work with.

The biggest thing I want to hack on is my brain. As I mentioned above, I’ve got a new server and I need to spend some time learning how it works. I’m intimately familiar with FreeBSD, but since it’s a virtual hosting solution, I’m constrained at this point to use Debian GNU/Linux on the new server. Since I’ve been using Ubuntu (which is based on Debian) on my desktop for over a year, it is fairly easy to manage. But there are lots of server-related configurations and tasks that I need to nail down for good security and management.

For general Linux information goodness, I’m following a set of tutorials from the IBM Developer Network entitled the Linux Professional Institute (LPI) exam prep, described as a “series of tutorials to help you learn Linux fundamentals and prepare for system administrator certification”. I’m not intending to write the exams—just learn the material. I’m finding that the tutorials give very good background information, covering things in enough detail to explain the process. I can then, of course, delve into the man pages and other documentation to learn more.

I’m enjoying it so far.