So, after playing with Bareos for a bit, I realized that I had chewed off a bit more than I could swallow. It isn't that it's a bad product, it's just more nuclear weapon and I only need a grenade.
I looked around for a bit and stumbled upon Attic which seemed to be a pretty great bit of software for what I needed, and then I was informed that it had lagged a bit behind in development.
So, I looked at its fork: Borg. It looked like everything that I could want, only needed port 22 since I would be using a central server to house the backups, and was an easy install since it was available via pip.
Note: I know that most of my readers are more Arch Linux than Ubuntu. Knowing this, a forewarning: there will be some Ubuntu stuff in here as well.
Getting the Backup Server
This is an easy one: build a server with lots of storage for all of your backups. We'll use Arch Linux for the backup server, so you will need a few things:
- Decent sized HDD
- A target user that servers can SSH into to save backups. (My example user is
jweatherly, I would recommend something more like
So, in short:
I'm assuming that the essential things are done such as acquiring a server (physical or VM) and installing Arch Linux on it, and that an account has been created with
- Log into the server.
borgon the server, and ensure that there's an
sudo pacman -S borg openssh
Start and enable the newly installed
sudo systemctl enable sshd sudo systemctl start sshd
Make a directory on the server to house all of these backups. I chose
/mnt/backupsas my root, so we'll use that. Notice that the
/mnt/backupsdirectory is owned by my backup user.
sudo mkdir -p /mnt/backups sudo chown jweatherly:jweatherly /mnt/backups chmod 0750 /mnt/backups
Installing the "Client"
We'll be pushing backups to the backup server via
cron jobs eventually, but there's a bit of work to do before then.
Installing the Prerequisites
So, for the clients on the servers we'll need most of the same things that the backup server had. You don't strictly need
openssh installed, but I'm guessing/assuming that it already is.
Procedure for Arch Linux
sudo pacman -S borg cronie
cronieis enabled and running.
sudo systemctl enable cronie sudo systemctl start cronie
Procedure for Ubuntu
sudo apt install borgbackup
Getting Things Set Up
This procedure should work for both Ubuntu and Arch Linux. We'll initialize the new backup, then conduct an initial full backup to the backup server.
Then, at the end, we'll set up a
cron job to make everything automatic.
First, let's generated some SSH keys for the
rootwill use these keys to connect to the backup server. We will use empty passphrases so that these keys can be used without user intervention.
ssh-keygen -t ed25519 Generating public/private ed25519 key pair. Enter file in which to save the key (/root/.ssh/id_ed25519): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_ed25519. Your public key has been saved in /root/.ssh/id_ed25519.pub.
Next, we're going to log into the backup server using our user account that we set up in the very first section. We don't want to SSH as
root. I'm going to use the
jweatherlyaccount from above.
ssh-copy-id [email protected] /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_ed25519.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys [email protected]'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.
Now, SSH into the backup server and make a directory for the server's backups to live in. I name them off the first part of their FQDN (i.e.
server1). Since the name of this server is
server1, we'll use that as the directory name.
hostname --short server1 ssh [email protected] Last login: Mon Jan 30 00:15:25 2017 from 10.11.12.13 mkdir /mnt/backups/server1 chmod 0750 /mnt/backup/server1 exit
Next, let's initialize the
/mnt/backups/server1directory on the backup server via the
borg initcommand. This is required before we start backup blasting to the remote server.
We won't assign a passphrase to the encryption key (thought you may if you want to, just remember that you'll have to add it somewhere to automate everything).
borg init --encryption=keyfile [email protected]:/mnt/backups/server1 Enter new passphrase: Enter same passphrase again: Do you want your passphrase to be displayed for verification? [yN]:
Okay, so let's start our first backup since everything is in place. I exclude several directories by default as they're not something you should try to backup:
The following commands look a bit messy, but we define three variables to clean it up slightly. The
REPOvariable just holds the location of our backups that we set up earlier. The
HOSTvariable gives us good information about our host (notice that it's the same as the
REPOdirectory) and the
DATEjust sets the date of the backup.
[email protected]:/mnt/backups/server1 HOST=$(hostname) DATE=$(date +%Y-%m-%d) borg create -v --stats $REPO::$HOST-$DATE / --exclude /dev --exclude /proc --exclude /sys --exclude /var/run --exclude /run --exclude /lost+found --exclude /mnt --exclude /var/lib/lxcfs
Now that the backup is complete, we will create a script that will backup our server and prune any backups that our outside of our requirements.
For pruning, we'll set it up so that we have 7 daily backups, 4 weekly backups, and 6 monthly backups, and save it in the
mkdir /etc/backup chmod 755 /etc/backup vim /etc/backup/backup_script.sh
LOCAL=$(hostname --short) [email protected]:/mnt/backups/$LOCAL HOST=$(hostname) DATE=$(date +Y%-%m-%d)
borg create -v --stats $REPO::$HOST-$DATE / --exclude /dev --exclude /proc --exclude /sys --exclude /var/run --exclude /run --exclude /lost+found --exclude /mnt --exclude /var/lib/lxcfs
Keep 7 daily backups, 4 weekly backups, and 6 monthly ones
borg prune -v --list $REPO --keep-daily=7 --keep-weekly=4 --keep-monthly=6
The script should be in place, and ready to go. We'll set it so that it's executable, then create the
cronjobthat will allow it to run everyday at 0100.
chmod 0755 /etc/backup/backup_script.sh crontab -e
0 1 * * * /etc/backup/backup_script.sh
- Basically, enjoy your backups!
This should show a decent way of configuring backups on servers without too much issue. I'm almost positive that there's a mistake somewhere in this document, but this procedure mirror's my own when getting everything up and running.
I would recommend using things like
puppet to help out with setting up these backups across multiple servers, or using some
nfs-type links to manage the
backup_scripts.sh file across the servers. Also, staggering the backup times in your
cronjobs will help out greatly depending on the number of servers you're backing up.
Hopefully, there won't be too many entries here. I'm probably wrong.
- Initial release.