SD Card Backup and Custom Image...

[gunflame]

Here's the thing... I want to keep playing around with different software and configurations on my RPI... but I have already highly customized my Raspbian distribution.

I would like to make a backup (or an image) that I can use to reflash my SDcard in case everything goes wrong. I actually have 3 SD cards so I would also like to port the changes without having to do all the configurating from scratch.

So I would like to:
- Backup my SD card
- Restore (copy) to other SD Cards freely

Can anyone point me to a guide on how to do this? I don't even know what to write on a search bar in google/forum to get an answer.

[tdaemon]

You can use dd or Win32DiskImager, which are recommended to write SD images from download secton. With both tools you can also create image from SD card...

[billw]

If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS

[smallgod]

If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS

Thanks for this script!

[billw]

I made a small change to rpi-clone to avoid a possible error message when rsync tries and fails to copy a gnome .gvfs directory. If anyone has been getting this error, the error is harmless since .gvfs is a fuse mount point which should not be copied anyway, but I'm now explicitly excluding it to keep rsync quiet. If you don't want to re download rpi-clone, you can just edit it with this patch:

Code: Select all

--- rpi-clone.old0	2013-01-13 19:10:15.220909998 -0600
+++ rpi-clone	2013-01-13 19:09:40.174680138 -0600
@@ -311,10 +311,14 @@
 
 START_TIME=`date '+%H:%M:%S'`
 
+# Exclude fuse mountpoint .gvfs, various other mount points, and tmpfs
+# file systems from the rsync.
+#
 sync
 echo "Starting the filesystem rsync to $DST_DISK"
 echo -n "(This may take several minutes)..."
 rsync $RSYNC_OPTIONS --delete \
+		--exclude '.gvfs' \
 		--exclude '/dev' \
 		--exclude '/media' \
 		--exclude '/mnt' \

In case anyone wants to try it, you can do the local edit of your rpi-clone file using patch. If you create a patch file (say rpi-clone.patch) containing the above patch text, you can cd to
the bin directory where you installed rpi-clone, and apply the patch with:

Code: Select all

patch -p0 -l rpi-clone < rpi-clone.patch

[kumarnarain]

If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS

Hi
The script throws an error on Line 26 'Function not found' What is missing possibly? I have a 16 gb card inserted for back up purposes. Tried looking at the log ......

pi@raspberrypi /var/log $ tail rpiclone.log
tail: cannot open `rpiclone.log' for reading: No such file or directory

Regards
Kumar

[billw]

Do you have the bash shell installed? You could get that error if for some reason dash is being used instead of bash. If bash is installed, try running bash explicitly:

Code: Select all

bash /pathto/rpi-clone

If that still fails, add the -x flag to bash to get a trace.

If you don't have bash installed, one thing you can try is edit rpi-clone and change line 26 from:

Code: Select all

function usage

to

Code: Select all

usage()

Regardless, that's a change I should probably make anyway.

[kumarnarain]

Thank you. Will try that and post back. Did not know it needed bash.
Regards
Kumar

[sqrt3]

Thanks for this script from me too.
Since i'm running a different locale where the decimal seperator is ',' the following lines did not work for me:

Code: Select all

PART2_START=$(parted /dev/mmcblk0 -ms unit MB p | grep "^2" \
				| cut -f 2 -d: | sed s/MB// | cut -f 1 -d.)

I changed it to:

Code: Select all

PART2_START=$(parted /dev/mmcblk0 -ms unit MB p | grep "^2" \
				| cut -f 2 -d: | cut -f 1 -d. | cut -f 1 -d,)

which worked.

[kumarnarain]

Which locale is this? Just curious..

[sqrt3]

It's german ',' and '.' are interchanged in decimal numbers. BTW i omitted the

Code: Select all

sed s/MB//

but it's probably better to keep that in case of numbers without decimals.

[billw]

Thanks for letting me know about that locale problem and I've put a fix for it into the rpi-clone script. Also, I've now put rpi-clone on github and you can get it with:

Code: Select all

git clone https://github.com/billw2/rpi-clone.git 

For now, it's still on that pastebin link, but I think github is a better place for it.

[1HzCoder]

Billw
Downloaded manually and put in proper folder, git doesn't seem to work on Wheezy, ran it and after trying to use the drive,
got it to copy sda1 successfully, but when it gets to

"starting the file system rsync to sda
(This may take several minutes) .../user/local/sbin/rpi-clone: line 323: rsync: command not found
../user/local/sbin/rpi-clone: line 371 /mnt/clone//var/log/rpi-clone.log: no such file or directory

***Done with clone to /dev/sda ***
started:16:24:56 Finished: 16:25:01"

Do I need to create the log file, and should I redownload the zip file?
Thanks for the script, it's been years since I messed with Linux and it pretty much guided me through all my screw-ups.
TomJ

[billw]

Billw
Downloaded manually and put in proper folder, git doesn't seem to work on Wheezy, ran it and after trying to use the drive,
got it to copy sda1 successfully, but when it gets to

"starting the file system rsync to sda
(This may take several minutes) .../user/local/sbin/rpi-clone: line 323: rsync: command not found
../user/local/sbin/rpi-clone: line 371 /mnt/clone//var/log/rpi-clone.log: no such file or directory

***Done with clone to /dev/sda ***
started:16:24:56 Finished: 16:25:01"

Do I need to create the log file, and should I redownload the zip file?
Thanks for the script, it's been years since I messed with Linux and it pretty much guided me through all my screw-ups.
TomJ

The first error you got was because the script tried to run the rsync command but could not find it because it looks like rsync is not installed by default on the Raspbian image. I did not remember that I must have installed it myself at some point and should have made a point of mentioning that rsync needs to be installed ... sorry about that.

Anyway, you need to install rsync:

Code: Select all

apt-get update
apt-get install rsync

The second error you got about the log file was because rsync did not run and create the necessary directory structure needed to write the log file. So just get rsync installed and try running rpi-clone again. The script was running so I'm pretty sure you downloaded and installed it OK.

[1HzCoder]

Thanks Bill,
Will check that in the morning,there are So-o-o many little things like that that I don't remember.

TomJ

[dallas1951]

I tried this today and the script failed.
root@incrediblepbx:~# cd /usr/local/sbin/rpi-clone/
root@incrediblepbx:/usr/local/sbin/rpi-clone# ./rpi-clone sda -v

Clone destination disk : sda
Clone destination rootfs : /dev/sda2 (no label) on /mnt/clone
Clone destination bootfs : /dev/sda1 on /mnt/clone/boot
Verbose mode : on
===============================
Final check, is it Ok to proceed with the clone (yes/no)?: y
=> Mounting /dev/sda2 (no label) on /mnt/clone
mount: special device /dev/sda2 does not exist
Mount failure of /dev/sda2, aborting!

Prior to running the script cat /prog/partitions gave me this.
root@incrediblepbx:~# cat /proc/partitions
major minor #blocks name

179 0 7761920 mmcblk0
179 1 76800 mmcblk0p1
179 2 7683072 mmcblk0p2
8 0 7782400 sda
8 1 76800 sda1
8 2 7703552 sda2
After it failed I got this.
root@incrediblepbx:~# cat /proc/partitions
major minor #blocks name

179 0 7761920 mmcblk0
179 1 76800 mmcblk0p1
179 2 7683072 mmcblk0p2

As you can see I'm running incredible pbx which is raspian wheezy December 2012.
To get the sda partitions back I have to power down the pi and restart - that seems odd.

Does anyone have any suggestions?

[gridrun]

Windows users might like HDDGURU's free 'HDD Raw Copy Tool' to fully clone and/or image their SD Cards. Most other Windows utilities work with partitions and can't see anything non-Windows, while this one works on the device layer and does not care about partitions nor filesystems.

You need to take special care of what devices you actually select for 'source' and 'destination' as to not accidentally overwrite the wrong drive. But the tool works flawlessly and does exactly as advertized. File sources (image files) are supported as either 'source' or 'destination', letting you create and restore images of an entire SD card's content at once.

URL: http://hddguru.com/software/HDD-Raw-Copy-Tool/

[mcgyver83]

I'm interested in this tool to have a always up to date SD backup.
I plug in the usb hub a sd card reader with a card.
I run ./rpi-clone but this is the output:

Code: Select all

sudo ./rpi-clone sdb -v
A destination partition is busy (mounted).  Mount status:
    /dev/sdb2:  /media/usb1
    /dev/sdb1:  /media/usb0
Aborting!

Ok, it is clear
But my usb stuff is automounted so I can use also my external usb hdd with xbmc.
How can I edit the github script to umount these devices before run the backup?

Second question: I left 6GB of empty (not partitioned) space at the end of the usb disk, how can I use this space to backup my running rasp?In this way I can schedule a backup each night forgetting sd wearing problem.

[billw]

Ok, I updated rpi-clone to try unmounting mounted destination partitions instead of just aborting. But of course it will still abort if the unmount fails.
Above I said to get rpi-clone you can do a:

Code: Select all

git clone https://github.com/billw2/rpi-clone.git

But you can also go to: https://github.com/billw2/rpi-clone and download the zip file.

[keble]

Many thanks for that Bill. I have just stumbled across the same problem.
The updated script is running now.
Cheers!

[mcgyver83]

Nice!
Many thanks!

[Radics]

The script on the git site is working for me it is running right now. My question is do I have to restore this somewhere or will this card now boot up like the one it is a clone of?

[spamdan]

Isn't it true that using dd to copy an active filesystem can lead to data corruption? I've read this in several places and it seems to make sense as some of the files will be open at the time (and possibly being written to). This tool looks great but I want to make sure I'm backing up my Pi in a way that won't cause data corruption.

[chorlton2080]

If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS

Bill: this is a superb tool and much more efficient/flexible than using Win32DiskImager. Thanks for putting the time and effort into creating and sharing it.

[Eradicatore]

Yes, I second chorlton2080's comments!! Thank you so much Bill for sharing your script and putting the time into this! I just started reading the linux man pages on "parted" and partitioning in general, and then I finally stumbled on this thread. It was hard to find for some reason. All the noise out there about using the windows tool or dd....

I see your comment about being able to back up a booted RPI. That's what I was always worried about with using "dd". It sounds like you're saying this script is safe to run on a booted/mounted drive. Is that right? I'll try to answer that question for myself too. Thanks!

Justin