hatcreek68
Posts: 8
Joined: Tue Dec 26, 2017 7:54 pm

Automatic rsnapshot backup, rsync'd to QNAP NAS

Fri Jan 12, 2018 3:16 am

This is a long post to document 2.5 days of work to figure out an automatic backup of my pi.
I wanted to backup from pi to a QNAP NAS, which then gets mirrored to the cloud. 3 copies -2 local -1 remote.... 3-2-1 backup plan.
Sorry that the notes are a mess - i took notes of my steps in MS word, and just dumped it here for now to potentially help anyone doing similar things.



Backup Pi to QNAP NAS setup notes.docx
2017 12

Table of Contents
End goal / Process flow: 1
Rsnapshot Configuration steps 1
Establish SSH between Pi/server 1
Install apps on Pi, Folder setup (pi and QNAP NAS) 2
Determine an rsync command that does what you want (for pi SD ->pi SD snapshot) 3
Migrate the rsync command and options to the rsnapshot.conf 3
Create script to start rsnapshot and manage log file 5
Create cron jobs on PI to run the daily, weekly tasks 6
Rsync configuration to mirror snapshots to QNAP NAS 6
Test an rsync from pi to NAS (on pi) 6
Bonus (improvement or detriment depending) 6
Make a new script to run/log the rsync to NAS 6


End goal / Process flow:
Backup pi to QNAP NAS
- Pi runs rsnapshot to build incremental backups to it's own SD
- Rsnapshot will call an rsync command to mirror snaps to NAS
o Make compressed file then send, didn’t like adding 1M files to NAS
- Obsoleted -> rsync runs on QNAP NAS to pull pi rsnapshots folder to NAS backup folder
--------------------------------
Notes
1. I could only find synology guides w/ googling; so these notes capture a few changes needed for QNAP ie – unique cron tab and file path differences, and admin vs root usernames

2. Applicability: Setup on a Pi 2, Jessie Lite and QNAP 231+

3. I’m a rookie programmer, I ignored some of the advice and setup from listed websites, like not setting up special users and permissions, mainly don’t care, this took enough time as is, and this will all run on LAN. Perhaps not good excuses, just stating the thought.

4. Note 4: blasted case-sensitive OS; keep an eye out for case-sensitive errors (/backups vs /Backups)

Rsnapshot Configuration steps

Establish SSH between Pi/server
Commands from:
http://guficulo.blogspot.com/2015/04/ba ... ally.html
But keep in mind – we want to reverse the direction (pi connects to NAS)

Setup key logins vs PW
- Confirm that SSH keys are already present by running (on pi)

ls ~/.ssh

If you see a file called id_rsa.pub there (assuming you've already been through this once and are setting up a new Pi to be backed up) you can skip the next step

- If no keys are found, run

Code: Select all

  ssh-keygen -t rsa -C root@<Your Synology server's name>
  ssh-keygen -t rsa -C admin@192.168.2.2
vice versa for pi -> nas

Code: Select all

  ssh-keygen -t rsa -C root@192.168.2.3
- When prompted for the file in which to save the key, accept the default (hit <Enter>)

- When prompted for a passphrase hit <Enter> (no passphrase)


Push the public key to the Raspberry Pi

Code: Select all

	cat ~/.ssh/id_rsa.pub | ssh root@192.168.2.3 'cat >> .ssh/authorized_keys'
-or to push key to NAS-

Code: Select all

cat ~/.ssh/id_rsa.pub | ssh admin@192.168.2.2 'cat >> .ssh/authorized_keys'
(needed help from https://www.raspberrypi.org/documentati ... ordless.md)
If your Pi does not have an .ssh directory you will need to set one up so that you can copy the key from your computer.

Code: Select all

cd ~
install -d -m 700 ~/.ssh


Confirm that SSH key logins are now accepted by the Raspberry Pi by running this command (still from the Synology session, don't toggle back to the Pi session yet:)
ssh root@<Your Pi's IP Address>
ssh root@192.168.2.3

Reversed: pi -> nas
ssh admin@192.168.2.2


Delete root's PW
passwd -d root



Install apps on Pi, Folder setup (pi and QNAP NAS)
Step details here:
http://jeffskinnerbox.me/posts/2014/Feb ... snapshot/
sudo apt-get install rsync grsync rsnapshot

pi rsnapshot backups will go to /mnt/backups/

Code: Select all

mkdir /mnt/backups
chmod a+rwx /mnt/backups
chmod o-w /mnt/backups
On QNAP I already had a “Backups” shared folder, so I just added a “Pi-1” folder from a PC
Referencing this path required: /share/Backups/ vs. synology guides which mention /volume1/.../backups

Cron will call scripts and reference files (exclude txt file) to help keep the commands straight and add extra logging features vs. pasting a giant command in the cron file. Might as well add logs to the same folder?

Logs and scripts to go here:
/home/pi/backup-setup/ for the pi



Determine an rsync command that does what you want (for pi SD ->pi SD snapshot)
Rsnapshot just runs rsync but manages the destination file structure to allow iterations vs. 1 single instance; therefore we should determine an rsync command that creates the mirror we want, then setup rsnapshot to duplicate that command w/ the rsnapshot.conf file


Rsync trials:
Guide:
http://jeffskinnerbox.me/posts/2014/Feb ... snapshot/ ↑ hint use –dry-run during “practice”

Create the exclude text file - /home/pi/backup-setup/rsync-exclude.txt
Add these to the exclude, see other refs for why…
- /var/lib/pacman/sync/* - ignore? Don’t have this…
- /lost+found
- /media/*
- /cdrom/*
- /proc/*
- /mnt/*
- /run/*
- /tmp/*
- /sys/*
- /dev/*

This was a dry run that got things working and I got a “feel” for the command; so as I migrated to this to the rsnapshot.conf I modified this a bit. For example, the trial source of “/home/pi/.npm/” changed to root directory of just “/” and logging changed after creating “the wrapper.sh” coming next.

Trial rsync-ing:
/usr/bin/rsync -av --dry-run --delete --exclude-from=/home/pi/backup-setup/rsync-exclude.txt -e "ssh -o CheckHostIP=no -o StrictHostKeyChecking=no" /home/pi/.npm/ admin@192.168.2.2:/share/Backups/pi-1/ --log-file="/home/pi/backups/log-$(date +"%Y-%m-%d".log)"

- later we’ll migrate to pi SD -> pi SD for rnapshot (learned the hard way) but the above command options were adapted for the later rsync run from qnap, where we DO need the fancy command w/ remote addresses and ssh etc.


Migrate the rsync command and options to the rsnapshot.conf
Guide:
http://jeffskinnerbox.me/posts/2014/Feb ... snapshot/
I later learned that you can’t use rsnapshot to make a remote mirror, the mirror has to be on the maching that runs rsnapshot.
So in the above command it’s local source -> remote destination, as we migrate to this .conf it changed to local -> local (root -> /mnt/backups/)

Rolling w/ the punches, I decided it’s fine, step 1 will make snapshots to pi, and step 2 will rsync to the QNAP NAS. The NAS syncs to the cloud nightly… 3-2-1 backup plans… never lose anything*

Create/edit the config file
Sudo nano /etc/rsnapshot.conf (a template was already there for me)

And I ended up editing the following in it:

Code: Select all

#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULES:       #
#                                               #
# This file requires tabs between elements      #
#                                               #
# Directories require a trailing slash:         #
#   right: /home/                               #
#   wrong: /home                                #
#                                               #
#################################################

#
#REDACTED UNCHANGED STUFF
#


###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
# Can’t be remote……………. WHYYYYYYYYYYY I get hardlinks don’t work remote but… why. Dang.
#Actually, the real issue is lack of easy way to get rsnapshot onto qnap. Beside the point.
snapshot_root	/mnt/backups/



#########################################
#           BACKUP INTERVALS            #
# Must be unique and in ascending order #
# i.e. hourly, daily, weekly, etc.      #
# below keeps 7 days and 2 mo. Weekly for short term / longer term backups
#########################################


#retain		hourly	6
retain		daily	7
retain		weekly	8
#retain		monthly	3

# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
logfile	/home/pi/backup-setup/rsnapshot.log


# Default rsync args. All rsync commands have at least these options set.
# I removed z option (-aez  to → -ae) to save rpi CPU usage (didn’t test)
#a = archive e = use ssh, turns out ssh is not needed… as we’re going local to local for rsnapshot
rsync_short_args	-a
rsync_long_args	--delete --numeric-ids --relative --delete-excluded

# ssh has no args passed by default, but you can specify some here.
#
#ssh_args	-o CheckHostIP=no -o StrictHostKeyChecking=no

# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file	/path/to/include/file
exclude_file	/home/pi/backup-setup/rsync-exclude.txt


###############################
### BACKUP POINTS / SCRIPTS ###
###############################

# LOCALHOST
# this will backup root file structure at “/” to “/mnt/backups/daily.0/root/…”
#couldn’t use / to / apparently
backup	/	root/

Save the file,
Test it…

Code: Select all

sudo rsnapshot configtest
Create script to start rsnapshot and manage log file

Guide:
http://jeffskinnerbox.me/posts/2014/Feb ... snapshot/
I copied his script (the wrapper script for rsnapshot), but did not use his notification push, and changed log file path
We’ll call this script in the cron jobs vs. the actual command.

In the cron job, we’re sending a variable (daily, weekly, etc), and the script picks it up as var 1
- ie $@ = daily/weekly from cron task

Code: Select all

#!/bin/bash

# This script is a wrapper around rsnapshot, so status information is logged
# and notification are made via Pushover app.  It also makes sure the log file
# doesn't grow too large.

# Capture scripts start time
STARTTIME=`date +%s`

# Location of the Log file
LOG="/home/pi/backup-setup/rsnapshot.log"

# Number of lines in the log file required to trigger it's truncation
MAXLINES=7000

# Number of lines remaining in the log file after its truncated
MINLINES=2000

# Calculate the size of the Log file
LOGSIZE=$( wc $LOG | awk '{ print $1 }' )

# Place in the log file information concerning the execution of this script
echo -e "\n\n+++++ rsnapshot script started on `date` +++++" >> $LOG
echo "options passed to rsnapshot: " $@ >> $LOG

# Now execute the script
#/usr/bin/sudo /usr/bin/rsnapshot "$@" >> $LOG 2>&1
sudo rsnapshot "$@" >> $LOG 2>&1
EXITSTATUS=$?

# Capture scripts end time and calculate run time
ENDTIME=`date +%s`
INTERVAL=$((ENDTIME - STARTTIME))
RUNTIME="$(($INTERVAL / 60)) min. $(($INTERVAL % 60)) sec."

# Place time-stamped completion message in the Log
echo "rsnapshot exited with status $EXITSTATUS" >> $LOG
echo "+++++ rsnapshot ran" $RUNTIME "and script completed on `date` +++++" >> $LOG

# Send status notification to Pushover app
#/home/jeff/bin/apprise -t "Incremental Backup Status" -m "Filesystem backup took $RUNTIME and completed on `date` with exit status $EXITSTATUS.  Log file $LOG has $LOGSIZE lines."

# Truncate the log file if needed
if [ $LOGSIZE -ge $MAXLINES ]
then
    LINECUT=`expr $LOGSIZE - $MINLINES`
    sed -i "1,$LINECUT d" $LOG
    sed -i "1i ////////////////  File truncated here on `date`. //////////////// " $LOG
fi

file path for ref: /home/pi/backup-setup/snapshot-wrapper.sh

test and see if the files appear:
run:
sudo chmod +x /home/pi/backup-setup/snapshot-wrapper.sh
(have to make it executable first…) now
/home/pi/backup-setup/snapshot-wrapper.sh daily

It checked out, logging is better, cleaner w/ Jeff’s work and the files show on home/pi/…
Create cron jobs on PI to run the daily, weekly tasks

It’s “proper” to use crontab -e to append, as -e will check format before appending compared to direct input. We’ll add:

https://docs.acquia.com/article/cron-ti ... ng-format
crontab -e
now add:

0 13 * * 4 /home/pi/backup-setup/snapshot-wrapper.sh weekly
↑ Execute a command at 1 p.m. UTC every Thurs

And add
0 18 * * * /home/pi/backup-setup/snapshot-wrapper.sh daily
↑ Execute a command at 6pm daily


Rsync configuration to mirror snapshots to QNAP NAS
Test an rsync from pi to NAS (on pi)
Now that snapshots are working – we now need to get the snaps to the NAS for an off-device, and off site backup.

This seemed to work:
/usr/bin/rsync -av --dry-run --delete -e "ssh -o CheckHostIP=no -o StrictHostKeyChecking=no" /mnt/backups/ admin@192.168.2.2:/share/Backups/pi-1/

Bonus (improvement or detriment depending)
- rsync a zip of backups folder vs. lots of files. Benefit, backup 1 file to cloud nightly, detriment, zip and send big file possibly losing differential benefit of rsync (unclear if rsync can check differences within a zip?)

Guide:
https://www.techrepublic.com/article/ho ... ng-rsync/
A little zipping (Abandoned…)
A nifty trick you could do is zip your folders and then back them up with rsync. Say you want to zip the folder /data/MONDAY and back it up to a remote Linux server. To do this, you would employ the zip command like so:

zip /data/MONDAY.zip /data/MONDAY && rsync -av -e ssh /data/MONDAY.zip backup@192.168.1.228:/backup/

or tar:
tar -vczf /mnt/pi-backup.gz /mnt/backups

add before rsync, change rsync source:
tar -vczf /mnt/pi-backup.gz /mnt/backups && /usr/bin/rsync -av --delete -e "ssh -o CheckHostIP=no -o StrictHostKeyChecking=no" /mnt/pi-backup.zip admin@192.168.2.2:/share/Backups/pi-1/

Test speed on initial and secondary syncs:
Ran 2x
- speed 1
- speed 2
Added another daily backup, ran 2 more times:
- speed 1
- speed 2


Now we need to integrate this command into the rsnapshot process.
Make a new script to run/log the rsync to NAS

Create
/home/pi/backup-setup/nas-rsync.log
/home/pi/backup-setup/nas-rsync-wrapper.sh

This will be a copy of the script above but adjusting
- log file
- command
- log line limits

Code: Select all

#!/bin/bash

# This script is a wrapper around rsnapshot, so status information is logged
# and notification are made via Pushover app.  It also makes sure the log file
# doesn't grow too large.

# Capture scripts start time
STARTTIME=`date +%s`

# Location of the Log file
LOG="/home/pi/backup-setup/nas-rsync.log"

# Number of lines in the log file required to trigger it's truncation
MAXLINES=100000

# Number of lines remaining in the log file after its truncated
MINLINES=20000

# Calculate the size of the Log file
LOGSIZE=$( wc $LOG | awk '{ print $1 }' )

# Place in the log file information concerning the execution of this script
echo -e "\n\n+++++ qnap-rsync script started on `date` +++++" >> $LOG
echo "options passed to rsnapshot: " $@ >> $LOG

# Now execute the script
/usr/bin/rsync -av --delete -e "ssh -o CheckHostIP=no -o StrictHostKeyChecking=no" /mnt/backups/ admin@192.168.2.2:/share/Backups/pi-1/ >> $LOG 2>&1
EXITSTATUS=$?

# Capture scripts end time and calculate run time
ENDTIME=`date +%s`
INTERVAL=$((ENDTIME - STARTTIME))
RUNTIME="$(($INTERVAL / 60)) min. $(($INTERVAL % 60)) sec."

# Place time-stamped completion message in the Log
echo "rsnapshot exited with status $EXITSTATUS" >> $LOG
echo "+++++ rsnapshot ran" $RUNTIME "and script completed on `date` +++++" >> $LOG

# Truncate the log file if needed
if [ $LOGSIZE -ge $MAXLINES ]
then
    LINECUT=`expr $LOGSIZE - $MINLINES`
    sed -i "1,$LINECUT d" $LOG
    sed -i "1i ////////////////  File truncated here on `date`. //////////////// " $LOG
fi

test and see if the files appear on the nas
run:
sudo chmod +x /home/pi/backup-setup/nas-rsync-wrapper.sh
(have to make it executable first…) now
/home/pi/backup-setup/nas-rsync-wrapper.sh


Add the new rsync .sh to the rsnapshot conf option as shown:
/etc/rsnapshot.conf

# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec /path/to/postexec/script
cmd_postexec /home/pi/backup-setup/nas-rsync-wrapper.sh


check the config again

sudo rsnapshot configtest

Double check cronjobs are calling the right sh : Crontab --e

Return to “General discussion”