Disaster Recovery using amanda

Prerequisites

  • 2x Orange Pi +2E (or similar devices that offer at least two partitions, like SD Card and eMMC memory)
  • 1x SD Card (min 4GB)
  • External NFS server (like Synology NAS) to store backups

Goals

  • Production device:
    • eMMC runs a fully configured openhab installation
    • WHAT to backup?: Raw image of eMMC
    • WHEN to backup?: Automatic backups every night
    • WHERE to backup?: Backup to external NFS share
  • Recovery device:
    • SD CARD runs a plain Linux with just amanda client/server installed
    • WHEN / HOW to restore? Manual boot and restore trigger
    • WHERE to restore? Restore from external NFS share to internal eMMC

Production Device:

1. Make sure your NFS share is mounted during boot

go here

2. Install amanda

apt-get install amanda-server 
apt-get install amanda-client

3. Create the config directory

mkdir /etc/amanda/openhab-dir

4. Create the main config file

Show /etc/amanda/opehab-dir/amanda.conf

Hide /etc/amanda/opehab-dir/amanda.conf

Watch the parameters marked in red! To copy to clipboard, click the code

org "openHABian openhab-dir"                            # Organization name for reports
mailto "RECIPIENT@DOMAIN.TLD"                           # Email address to receive reports
netusage 10000 Kbps                                     # Bandwidth limit, 10M
dumpcycle 14 days                                       # when fulldump will be forced, inbetween incremental backups are possible (14 days = every 14 days fulldump)
runspercycle 14                                         # how often will a backup be started (using amdump cronjob) within a dumpcycle (14 / 14 days = once per day)
tapecycle 15 tapes                                      # recommended formula: value = (runspercycle + 1) * runtapes
runtapes 1                                              # just one tape / one slot per backup (don't use more than one tapes, even if tape is full -> generate error)
tpchanger "chg-disk:/mnt/NAS"                     # The tape-changer glue script
taper-parallel-write 2
autolabel "openHABian-openhab-dir-%%%" empty
changerfile "/etc/amanda/openhab-dir/storagestate"      # The tape-changer or SD- or disk slot or S3 state file
tapelist "/etc/amanda/openhab-dir/tapelist"             # The tapelist file
tapetype DIRECTORY
infofile "/var/lib/amanda/openhab-dir/curinfo"          # Database directory
logdir "/var/log/amanda/openhab-dir"                    # Log directory
indexdir "/var/lib/amanda/openhab-dir/index"            # Index directory

# tapetypes as defined in 'tapetype' above
define tapetype DIRECTORY {                             # Define our tape behaviour
        length 15360 mbytes                             # size of every virtual container (= max. usage per directory)
}

amrecover_changer "changer"                             # Changer for amrecover

#configurations (how to dump) for 'disklist' file
define dumptype global {                                # The global dump definition
        maxdumps 2                                      # maximum number of backups run in parallel
        holdingdisk no                                  # Dump to temp disk (holdingdisk) before backup to tape
        index yes                                       # Generate index. For restoration usage
}
define dumptype root-tar {                              # How to dump root's directory
        global                                          # Include global (as above)
        program "GNUTAR"                                # Program name for compress
        estimate server                                 # Estimate the backup size before dump
        comment "root partitions dumped with tar"
        compress none                                   # No compression
        index                                           # Index this dump
        priority low                                    # Priority level
}
define dumptype user-tar {                              # How to dump user's directory
        root-tar                                        # Include root-tar (as above)
        comment "user partitions dumped with tar"
        priority medium                                 # Priority level
}
define application-tool app_amraw {                     # how to dump the SD card's raw device /dev/mmcblk0
        plugin "amraw"                                  # uses 'dd'
}
define dumptype amraw {
        global
        program "APPLICATION"
        application "app_amraw"
}
#define dumptype comp-user-tar {                        # How to dump & compress user's directory
#       user-tar                                        # Include user-tar (as above)
#       compress client fast                            # Compress in client side with less CPU (fast)
#}
# vim: filetype=conf

5. Create client authentication file

/etc/amanda/openhab-dir/amanda-client.conf

auth "local"

6. Create the disklist (here mmcblk0 is the internal eMMC)

/etc/amanda/openhab-dir/dislist

smarthome  /dev/mmcblk0              amraw

7. Set user permissions for user backup

chown -R backup:backup /etc/amanda/openhab-dir/

8. Run the script below to

  • create the required target directory structure (slots)
  • enable a shell for user 'backup'
  • assign labels to the slots

Hint: Run the script with bash shell, not sh!

Show script

Hide script

⇛ To copy to clipboard, click the code
#!/bin/bash
# Created by Boris Manojlovic (modified by Stefan Haupt)
# License: Public Domain

# GLOBALS

CFGNAME=""
TAPENAME=""
MAXTAPES=""
VTAPESFOLDER=""

usage () {
        echo
        echo "USAGE:"
        echo "  $0 <CONFIG_NAME> <TAPENAME>             # you can provide the tapename here"
        echo "  $0 <CONFIG_NAME>                        # the script will ask you for the tape name"
        echo
        echo "Example:"
        echo "  $0 openhab-dir mytape"
        echo
        return 0
}

readconfig () {
        CFGNAME=$1
        TAPENAME=$2
        if [ ! -f /etc/amanda/$CFGNAME/amanda.conf ]; then
                echo "ERROR:No Config \"${CFGNAME}\" file found"
                return -1
        fi
        # Get number of tapes in "charger"
        MAXTAPES=`grep tapecycle /etc/amanda/${CFGNAME}/amanda.conf |grep -v ^#|cut -d " " -f 2`
        VTAPESFOLDER=`grep tpchanger /etc/amanda/${CFGNAME}/amanda.conf|grep -v ^#|cut -d '"' -f 2|cut -d ':' -f 2`
        if [ "x${VTAPESFOLDER}" == "x" ]; then
                echo "ERROR: Unable to read 'tpchanger' value from /etc/amanda/${CFGNAME}/amanda.conf"
                return -1
        fi
        echo "Please type in tape label text: "
        if [ "x${TAPENAME}" == "x" ]; then
                read TAPENAME
        fi
        if [ "x${TAPENAME}" == 'x' ] ; then
                echo "ERROR:TAPENAME cannot be empty"
                return -1
        fi
        echo "Name of tape which will be used is: ${TAPENAME}"
        return 0
}

if [ $# -lt 1 ]; then
        usage
else
        CONFIG=$1
        TAPENAME=$2
        readconfig ${CONFIG} ${TAPENAME}
        if [ $? -ne 0 ];then
                echo "Exiting"
                exit -1

        fi
        echo "DEBUG:CFGNAME=${CFGNAME} TAPENAME=${TAPENAME} MAXTAPES=${MAXTAPES} VTAPESFOLDER=${VTAPESFOLDER}"

        mkdir -p ${VTAPESFOLDER}
        cd ${VTAPESFOLDER}
        for i in `seq 1 ${MAXTAPES}`; do
                mkdir -p slot$i;
        done
        chown -R backup:backup ${VTAPESFOLDER}
        ls -al ${VTAPESFOLDER}
        usermod --shell /bin/bash backup
        for i in `seq 1 ${MAXTAPES}`; do
                su - backup -c 'amlabel $0 $1$2 slot $2' -- ${CONFIG} ${TAPENAME} ${i}
        done
fi
# end of script

9. Create /etc/cron.d/amanda

  • backup (amdump) runs every morning 1am. It sends an automatic eMail report.
  • every 30min a check (amcheck) will be performed (e.g. if target has enough disk space). An eMail will be sent in case of an error.
  • backup important amanda files (especially database/index) every day at 2am (important for restore)
Watch the parameters marked in red! To copy to clipboard, click the code
0 1 * * * backup /usr/sbin/amdump openhab-dir >/dev/null 2>&1
0 0/2 * * * backup /usr/sbin/amcheck -m openhab-dir >/dev/null 2>&1
0 2 * * * root (cd /; tar czf /mnt/NAS/amanda_data_$(date +\%Y\%m\%d\%H\%M\%S).tar.gz /etc/amanda /var/lib/amanda; find /mnt/NAS -name amanda_data_\* -mtime +10 -delete) >/dev/null 2>&1

10. Test your configuration

You might see some info messages that some files don't exist, which will be created the next time you start your backup.
cd /tmp
su backup
amcheck openhab-dir

Recovery Device:

One-Time preparation

1. At first you need to download Linux for your device - here Armbian for Orange Pi +2E - and write the image to your SD Card (e.g. with Win32DiskImager).

Then boot your device from the SD Card and install amanda:

apt-get update
apt-get install amanda-server
apt-get install amanda-client

2. Make sure your NFS share is mounted during boot

go here

3. Assign a shell to user backup

usermod --shell /bin/bash backup

4. Assign a shell to user backup

usermod --shell /bin/bash backup
Perform restore

1. Extract the latest backup amanda data from NFS share

cd /
tar xvzf /mnt/NAS/amanda_data_TIMESTAMP.tar.gz

2. Change to user backup

su backup

3. Create restore image from slot data

Syntax:  amfetchdump -p openhab-dir ORIGINALHOSTNAME ORIGINALDEVICE DATE_TO_RESTORE_TO_YYYYMMDD > /mnt/NAS/restoreimage
Example: amfetchdump -p openhab-dir smarthome /dev/mmcblk0 20180702 > /mnt/synology/restoreimage

4. Dump image to eMMC

Attention: When booted from SD Card the eMMC is /dev/mmcblk1

Remark: dd does not output any progress bar, you just have to wait

dd bs=4M if=/mnt/NAS/restoreimage of=/dev/mmcblk1

Appendix: How to reset Amanda and start from scratch

rm -rf /var/lib/amanda/openhab-dir/*
rm -rf /var/log/amanda/openhab-dir/*
rm -rf /var/log/amanda/client/openhab-dir/*
rm -rf /var/log/amanda/server/openhab-dir/*
rm /etc/amanda/openhab-dir/tapelist*