How to create automated Linux/Unix backups

“A stitch in time saves nine”, goes an old saying. In the world of computers we refer to that stitch as backups. When running a production IT infrastructure, it is imperative that we achieve an uptime on our servers and applications of as close to a hundred percent as possible. While there are a number of ways to make an IT infrastructure solid, it is impossible to make it failure proof. So we make a plan B.

Servers are made of three broad layers – the hardware, the operating system and applications, and the data. In case of a hardware failure it is rather simple to replace the machine with a new one. Installing the operating system and applications too are relatively simple. The part that is hard to replace is the data and configuration that your applications use. In case of failure this is the part that can not be replaced. So we backup that data on a regular basis. We back it up in such a way that it becomes easy to retrieve in case of emergency.

I will guide you through writing a script to take daily backups on your system. For this exercise I will use a server that runs the Apache web server, a MySQL database server, and has a directory on the /home partition that contains some data that needs to be backed up. We will also backup the previous day’s Apache access logs in a separate backup directory. First, we need to create a directory where our backups will reside. As part of the exercise I have created a directory called BACKUP on the /opt partition. We will organize the backups in this directory so that each days backup resides in a directory formed like this – /opt/BACKUP/YEAR/MONTH/DATE/. So on the 11th of August 2008 the backups will go into /opt/BACKUP/2008/08/11/. Let’s create these directories. Run the following set of commands as the root user.

# mkdir -p /opt/backup/DATA/2008
# cd /opt/backup/DATA/2008
# mkdir 01 02 03 04 05 06 07 08 09 10 11 12
# cd /opt/backup/DATA/
# cp -r 2008 2009
# cd /opt/backup/
# cp -r DATA LOG

See note below before issuing this last command.

# chown -R calvin.calvin /opt/backup

We now have two sets of backup directories – one for data and another for log files. With the backups directories created we are now ready to start writing the script. Note: I’m using the user calvin throughout this article. Please replace calvin with your own username.

###########################################################
# CODE BEGINS HERE
###########################################################

#!/bin/bash

# SERVER DATA BACKUP V1.0

#########################
# BACKUP CONFIG
#########################

#Configure the month, date, and day
YEAR=`date +”%Y”` # 2008
MONTH=`date +”%m”`# 11
DAY=`date +”%d”` # 14
YESTERDAY=`date –date=”yesterday” +%Y-%m-%d` # 2008-11-13
YESTERDAY_DATE=`date –date=yesterday +%d` # 13
YESTERDAY_MONTH=`date –date=yesterday +%m` # 11
YESTERDAY_YEAR=`date –date=yesterday +%Y` # 2008
TODAY=`date +%Y-%m-%d` # 2008-11-14
TODAY_DATE=`date +%d` # 14

# directories to backup to
DATA_BACKUP_PARENT_DIR=/opt/backup/DATA/ # <-- Change this your data backup dir
LOG_BACKUP_PARENT_DIR=/opt/backup/LOG/ # <-- Change this your log backup dir

# Apache Document Root Directory
APACHE_DOCROOT=/var/www/html/

# MySQL Database config
DB_IP="localhost" # <-- Change this to the IP of your database server
DB_USER="calvin" # <-- Change this to your database username
DB_PASS="calvin_password" # <-- Change this to your database password
DB_NAME="clientlist" # <-- Change this to your database name

# Apache Log Dir
APACHE_LOG_DIR="/var/log/httpd/" # <-- Change this to the path data of your logs

# Application Files' Dir
APP_FILES="/home/calvin/application_files/" # <-- Change this to the path of the data directory

# Config Dir
CONFIG_DIR="/etc/" # <-- Change this to the path data directory

# Mail report address
EMAIL_ID="you@youremailid.com" # <-- Change this to your e-mail ID

###########################################################
# DATA BACKUP
###########################################################

# Create and go into backup directory
cd $DATA_BACKUP_PARENT_DIR/$YEAR/$MONTH
mkdir $DAY
cd $DAY

# Database backup
mysqldump -h ${DB_IP} -u ${DB_USER} -p${DB_PASS} ${DB_NAME} > ${DB_NAME}.db
tar -zcvf ${DB_NAME}.tar.gz ${DB_NAME}.db
rm -f ${DB_NAME}.db

# /etc backup
tar -zcf etc.tar.gz /etc

# Application backup
tar -zcf Apache_Doc_Root.tar.gz ${APACHE_DOCROOT}
tar -zcf App_Files.tar.gz ${APP_FILES}

###########################################################
# LOG BACKUP
###########################################################

# Create and go into backup directory
cd $LOG_BACKUP_PARENT_DIR/$YESTERDAY_YEAR/$YESTERDAY_MONTH
mkdir $YESTERDAY_DATE
cd $YESTERDAY_DATE

# HTTPD Log Backup
for i in ${APACHE_LOG_DIR}/access_log.${YESTERDAY}*; do cp $i .; done
for i in access_log.${YESTERDAY}*; do gzip $i; done

###########################################################
# Send out mail notifications
###########################################################

# Mail notify
du -shc $DATA_BACKUP_PARENT_DIR/$YEAR/$MONTH/$DAY/* | mail -s “Backup for ${HOSTNAME}::DATA done  ” ${EMAIL_ID}
du -shc $LOG_BACKUP_PARENT_DIR/$YESTERDAY_YEAR/$YESTERDAY_MONTH/$YESTERDAY_DATE/* | mail -s “Backup for ${HOSTNAME}::LOG done  ” ${EMAIL_ID}

###########################################################
# CODE ENDS HERE
###########################################################

Save this script in a directory such as /home/calvin/scripts/backup.sh. Give executable permissions to the file:

# chmod +x /home/calvin/scripts/backup.sh

Now you are good to go. Run the script form your command line:

# /home/calvin/scripts/backup.sh

You can also optionally add this script as a cron job so that it will be executed on a daily basis at a time of your choice. If you want to run this script at 4 AM daily do the following:

# crontab -e

Create a new line in the file that opens and add the following:

0 4 * * * /home/calvin/scripts/backup.sh > /dev/null

There you go. Your server will now backup your data, database, configuration files, application files, and log files to /opt/backup everyday at 4 AM. A smart thing to do now would be to export these backups to a remote machine so that you don’t lose your backups if your server faces problems. We will cover how to do that in my next article.