Automating Linux File Backups With Bash: part 1
“Always keep a backup” is one of the most fundamental rules in the digital age, but just like everyone else, I rarely follow it. Not too long ago, I faced the consiquences of my actions when my hard drive failed, and I lost a lot of important data because all my backups were outdated by about 1 year.
So I made this script to automate the process. It will watch for when I connect my external drive, when backup the needed files to that drive. I will be using luks to encrypt the drive because I don’t like to keep my private data openly on uncrypted drives.
Preparing drive
I will connect the drive to my laptop, and run lsblk to find the block device.
(I will be using a 16GB usb drive for the example)
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 931.5G 0 disk
├─sda1 8:1 0 2G 0 part /boot
└─sda2 8:2 0 929.5G 0 part
└─cryptlvm 253:0 0 929.5G 0 crypt
├─MyVolGroup-swap 253:1 0 8G 0 lvm [SWAP]
├─MyVolGroup-root 253:2 0 100G 0 lvm /
└─MyVolGroup-home 253:3 0 821.5G 0 lvm /home
sdb 8:16 1 14.6G 0 disk
└─sdb1 8:17 1 14.6G 0 part
sr0 11:0 1 1024M 0 rom
From the output, I can see that my usb drive is listed as sdb. Now I’m going to format the drive and encrypt it with luks. Before we encrypt it, we need to generate a key to decrypt it. We can use a password, but then we won’t be able to automate the process because it will prompt for the password.
To create the key file,
$ sudo dd if=/dev/urandom of=/root/luks.key bs=4096 count=1
# Set the permissions to read-only
sudo chmod 400 /root/luks.key
Before encrypting the drive, remember that this will completely wipe your data. So make sure to backup anything on that drive. Be careful not to format the wrong drive as well(it can happen sometimes).
$ sudo cryptsetup luksFormat /dev/sdb1 \
-c aes-xts-plain64 \
--hash sha512 \
--key-size 512 \
--iter-time 5000 \
--key-file /root/luks.key
Now that we have encrypted our drive, we can work on the backup script which will handle the decryption and the file backup.
Decrypting the drive
To decrypt the drive, we can use the command sudo cryptsetup luksOpen /dev/sdb1 luks_drive --key-file /root/luks.key. But there’s a problem. There is no guarantee that our encrypted partition will be named /dev/sdb1 after the next system reboot. So we need to identify the partition each time using the UUID of the partition.
$ ls -hAlF /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 10 Nov 17 18:13 29982e4e-ac5a-494a-9335-d542c3e6e379 -> ../../dm-2
lrwxrwxrwx 1 root root 10 Nov 17 18:13 3A12-B39A -> ../../sda1
lrwxrwxrwx 1 root root 10 Nov 17 18:13 499c4f61-9f38-4b0f-aef1-f0dd87bf2336 -> ../../dm-1
lrwxrwxrwx 1 root root 10 Nov 17 18:13 8e3c15bd-dfb8-4c6b-a031-552cebb7fc77 -> ../../sda2
lrwxrwxrwx 1 root root 10 Nov 18 09:57 ace19864-ea89-4db9-8da1-aa7956501198 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Nov 17 18:13 cc35e6cd-cbd3-41d4-9386-894f5f76cc59 -> ../../dm-3
As you can see, the UUID of the sdb1 partition is ace19864-ea89-4db9-8da1-aa7956501198. Now we can open the luks container.
$ sudo cryptsetup luksOpen \
/dev/disk/by-uuid/ace19864-ea89-4db9-8da1-aa7956501198 \
luks_drive \
--key-file /root/luks.key
Here, luks_drive is the name we are using to map the decrypted drive. This means that the decrypted drive is now available at /dev/mapper/luks_drive. Since this is the first time, we need to create a filesystem in our decrypted drive. So let’s format it with ext4.
$ sudo mkfs.ext4 /dev/mapper/luks_drive
Now we can mount the drive.
$ sudo mkdir /media/tmp_luks_drive
$ sudo mount /dev/mapper/luks_drive /media/tmp_luks_drive
We can access the decrypted drive at /media/tmp_luks_drive.
Making file backups with rsync
To copy the files/folders to our decrypted drive, we can use rsync. I will backup my notes folder in my home directory as an example.
$ rsync -a notes /media/tmp_luks_drive
It’s important to use notes instead of notes/ because the latter will copy the contents of notes to the output instead of copying the entire folder.
Closing the luks drives
Now that we have made the backup, it’s time to unmount the decrypted drive, then close the decrypted luks container.
$ sudo umount /dev/mapper/luks_drive
$ sudo cryptsetup luksClose /dev/mapper/luks_drive
$ sudo rmdir /media/tmp_luks_drive
Putting it all together and creating the backup script
To create the initial version of the backup script, let’s just put everything we did in the previous steps to one place. I will name the script as backup.sh. It should look like this so far,
#!/bin/bash
# Decrypt the luks container
sudo cryptsetup luksOpen \
/dev/disk/by-uuid/ace19864-ea89-4db9-8da1-aa7956501198 \
luks_drive \
--key-file /root/luks.key
# Mounting the decrypted drive
sudo mkdir /media/tmp_luks_drive
sudo mount /dev/mapper/luks_drive /media/tmp_luks_drive
# Making the backup
rsync -a notes /media/tmp_luks_drive
# Cleaning up
sudo umount /dev/mapper/luks_drive
sudo cryptsetup luksClose /dev/mapper/luks_drive
sudo rmdir /media/tmp_luks_drive
Make sure to add file permissions for executing the script with chmod +x backup.sh.
The code looks ok, but there is an issue here. Let’s say that we connected the drive and executed the script. What would happen if we remove the drive while files are being copied? This is even more of a problem when we automate it. Because what we will be doing is to watch for when our usb drive connects, and then execute this script. Let’s say if we connect and disconnect the drive immediately. The script will execute and fail at mounting the luks drive, but it will still rsync the files to /media/tmp_luks_drive.
At the end, when it tries to remove tmp_luks_drive, it will fail because rmdir will only remove empty directories. So we need to account for those errors.
I will be going over how to manage error handling in the next post. Then I will also cover how to use udev rules to execute the script based on hardware changes.