micerinos
Posts: 74
Joined: Fri Nov 09, 2012 11:15 am
Location: Madrid, Spain

Root union filesystem on raspian

Sun Nov 18, 2012 10:41 am

Hi all,
I'd like to explain how I managed to run rapbian with a unionfs root partition. Possible uses are related with keeping the internal sd card in read only mode and use writable layer placed either in a different partition of the same device or in a different medium (such as usb, nfs, sshfs or tmpfs if you have a 512MB raspi). With this, we can avoid detroying our root filesystem (or even the device itself), and also performance of sd media is vastly improved with respect to read-write mounts.

I'm currently using unionfs because aufs gave me problems on patching. I didn't explored this issue because unionfs worked like a charm, but it is often recommended to use the more modern and stable aufs. One interesting feature in aufs that you miss with unionfs is the possibility of syncing ro and rw filesystems within a running system. Never do it in unionfs! The script currently assumes you have your sd card formatted with at least 3 partitions (a 4th one to hold home directory contents maybe also of interest for some setups, like mine, to store media files). Here is my partition layout:

Code: Select all

unionfs(/ro & /rw)  /       rw
/dev/mmcblk0p1      /boot   ro  (150MB)
/dev/mmcblk0p2      /ro     ro  (4GB)
/dev/mmcblk0p3      /rw     rw  (1GB)
/dev/mmcblk0p4      /home   rw  (3GB)
tmpfs               /tmp    rw  (30MB)
tmpfs               /var/log rw (30MB)

Configuration of tmpfs filesystem is done within raspbian itself (see http://www.raspberrypi.org/phpBB3/viewt ... 40#p213440), but we could do everything from the initramfs. Another interesting configuration could be:

Code: Select all

unionfs(/ro & /rw)  /       rw
/dev/mmcblk0p1      /boot   ro  (150MB)
/dev/mmcblk0p2      /ro     ro  (4GB)
tmpfs               /rw     rw  (100MB)


Anyway, these are the steps I followed:

0-Prepare crosscompiling environment (Build Machine)
Follow steps 1 to 3 in the following post http://www.raspberrypi.org/phpBB3/viewt ... 29&t=22498

1- Get and patch kernel sources to support unionfs (Build Machine, home directory)

Code: Select all

cd
git clone --depth 1 git://github.com/raspberrypi/linux.git
cd linux
wget http://download.filesystems.org/unionfs/stable/unionfs-2.5.11_for_3.2.2.diff.gz
gunzip unionfs-2.5.11_for_3.2.2.diff.gz
patch -p1 < unionfs-2.5.11_for_3.2.2.diff
2- Create the initramfs (Raspbian Machine)

Code: Select all

sudo apt-get install busybox
mkdir initramfs 
cd initramfs 
fakeroot
mkdir -p bin lib dev etc proc sbin sys mnt/union mnt/ro mnt/rw
cp -a /bin/busybox bin/ 
ln -s busybox bin/sh 
touch etc/mdev.conf 
cp -a /lib/ld-* lib/
cp -a /lib/arm-linux-gnueabihf/{ld-*,libc-*,libc.so*,libdl*} lib/arm-linux-gnueabihf
cp -a /lib/arm-linux-gnueabihf/{libm-*,libm.so*,libpam.so*,libpam_misc*} lib/arm-linux-gnueabihf
Now create a file called init in the root of your initramfs with the folling code inside:

Code: Select all

#!/bin/busybox sh 

# Mount the /proc and /sys filesystems. 
mount -t proc none /proc 
mount -t sysfs none /sys 

busybox --install -s 

mknod /dev/null c 1 3 
mknod /dev/tty c 5 0 

# Select partitions here. This hardcodes the partitions inside the initramfs. It is also possible to parse
# kernel's command line and let the user choose from the the file cmdline.txt.
echo "This script mounts rootfs RO with an unionfs RW layer." 
mdev -s 
ROPART="/dev/mmcblk0p2"
RWPART="/dev/mmcblk0p3"

if [ ! -b "${ROPART}" ]; then 
  echo "No partition ${ROPART} has been found"
  exec /bin/sh 
  exit 0 
fi 

if [ ! -b "${RWPART}" ]; then 
  echo "No partition ${RWPART} has been found"
  exec /bin/sh 
  exit 0 
fi 

# Mount the partitions 
# 1) mount the ro partition 
mount -t ext2 -o ro,noatime ${ROPART} /mnt/ro

# 2) mount the rw partition, it could be any writable filesystem
# but you should make sure the needed modules are either 
# compiled built-in or present inside the initramfs and explicitly loaded before
# this line 
mount -t ext4 -o rw,noatime ${RWPART} /mnt/rw

# 3) mount the writable overlay to the static image 
mount -t unionfs -o dirs=/mnt/rw=rw:/mnt/ro=ro none /mnt/union

# Clean up. 
mkdir -p /mnt/union/rw /mnt/union/ro
mount --move /mnt/rw /mnt/union/rw
mount --move /mnt/ro /mnt/union/ro
umount /proc 
umount /sys 

# Boot the real thing
exec switch_root /mnt/union /sbin/init 

echo "Failed to switch_root, dropping to a shell" 
exec /bin/sh 
Make init executable for all

Code: Select all

chmod a+x init
Create initramfs.cpio and config for kernel compilation and copy them to build server

Code: Select all

find . | cpio -H newc -o > ../initramfs.cpio
cd ..
zcat /proc/config.gz > config.rpi
scp initramfs.cpio config.rpi USER@SERVER:BUILDIR
3- Compile kernel (Build Machine)
Configure kernel compilation issuing the following commands

Code: Select all

cp config.rpi .config
make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- menuconfig
There, you should choose the following options:

Code: Select all

Select "General Setup-->Initial RAM filesystem and RAM disk (initramfs/initrd) support"
Enter "initramfs.cpio" in Initramfs source file(s)
Enable unionfs in "File Systems-->Miscellaneous filesystems-->Union file system"
It's also a good moment to enable other modules we may have missed (usblp in my case). And finally, compile:

Code: Select all

make -j6 ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- -k
4- Copy kernel and modules to raspbian
Follow intructions in these post, steps 9 to 12: http://www.raspberrypi.org/phpBB3/viewt ... 29&t=22498
You should also remove any offending lines in /etc/fstab (any mention to root fs should be avoided). Mine looks like this:

Code: Select all

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    ro          0       2
/dev/mmcblk0p4  /home           ext4    defaults,noatime  0       1
tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777    0    0
tmpfs           /var/log        tmpfs   nodev,nosuid,size=30M,mode=1777    0    0
5-Reboot and enjoy!
To revert the system to pure rw mode we only need to rename the kernel.img and copy back the stock kernel.img. To sync the read only partition with the read write one (for instance, after a system upgrade), we have to extract the card and do it from another linux host. I'd recommend rsync for that.

Cheers

micerinos
Posts: 74
Joined: Fri Nov 09, 2012 11:15 am
Location: Madrid, Spain

Re: Root union filesystem on raspian

Tue Nov 20, 2012 12:23 am

Here you can find the compiled kernel and modules:
http://dl.dropbox.com/u/5684427/misc/kernel_unionfs.img
http://dl.dropbox.com/u/5684427/misc/mo ... .27%2B.tar

To make it work, save kernel_unionfs.img in /boot directory and modify cmdline.txt accordingly, cd to /lib/modules and extract modules tar. Edit /etc/fstab as explained above. Make sure you have at least the following partions (size is orientative):
/dev/mmcblk0p1 /boot fat (150MB)
/dev/mmcblk0p2 /ro ext2 (4GB)
/dev/mmcblk0p3 /rw ext4 (1GB)

A fourth home partition is also a good idea to store media files and user docs in general.

After a big system update, its a good idea to resync the ro partion and empty rw. With unionfs it cannot be done on the fly. You have to power off the raspi and mount the SD card in another computer and enter the commands:

Code: Select all

sudo rm -r /media/root-rw/var/lib/apt/lists/*
sudo rm -r /media/root-rw/var/cache/apt/archives/*
sudo rsync -arv /media/root-rw/ /media/root-ro/
sudo rm -r /media/root-rw/*
Where /media/root-ro and /media/root-rw are the mount points of the partions in the other computer.

Cheers


micerinos
Posts: 74
Joined: Fri Nov 09, 2012 11:15 am
Location: Madrid, Spain

Re: Root union filesystem on raspian

Tue Nov 20, 2012 11:33 am

I had fs corruption problems while syncing ro and rw with method explained above. Not sure which was the problematic step and not much time to test now. Maybe replacing rsync with "cp -Rup" is more conservative and safe. I forgot to mention, but It's always a good idea to have a full back up of your ro partition.

BMH
Posts: 4
Joined: Mon Jul 01, 2013 4:30 pm

Re: Root union filesystem on raspian

Mon Jul 01, 2013 4:34 pm

Micerinos, when you say
modify cmdline.txt accordingly
what does that actually mean? What inside of there needs to be changed?

micerinos
Posts: 74
Joined: Fri Nov 09, 2012 11:15 am
Location: Madrid, Spain

Re: Root union filesystem on raspian

Tue Jul 02, 2013 9:52 am

Hi,
If I'm not mistaken, cmdline.txt should contain the intended configuration of rw and ro partitions. config.txt should contain -in the example above- at least this line:

Code: Select all

kernel=kernel_unionfs.img
Cheers

User avatar
rikas
Posts: 17
Joined: Sun Jul 31, 2011 1:28 pm
Location: Austria y Argentina

Re: Root union filesystem on raspian

Sun Jul 07, 2013 10:05 am

micerinos,

I followed your excelent description ... but:

already at doing the patch for unionfs I get errors:

Code: Select all

rfk@deb02:/var/www/ARM-RPI/raspberrypi/linux$ wget http://download.filesystems.org/unionfs/stable/unionfs-2.5.11_for_3.2.2.diff.gz
–2013-07-07 09:37:18–  http://download.filesystems.org/unionfs/stable/unionfs-2.5.11_for_3.2.2.diff.gz
Resolving download.filesystems.org (download.filesystems.org)… 130.245.126.1
Connecting to download.filesystems.org (download.filesystems.org)|130.245.126.1|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 80196 (78K) [application/x-gzip]
Saving to: `unionfs-2.5.11_for_3.2.2.diff.gz'

rfk@deb02:/var/www/ARM-RPI/raspberrypi/linux$ gunzip unionfs-2.5.11_for_3.2.2.diff.gz


rfk@deb02:/var/www/ARM-RPI/raspberrypi/linux$ patch -p1 < unionfs-2.5.11_for_3.2.2.diff
patching file Documentation/filesystems/00-INDEX
patching file Documentation/filesystems/unionfs/00-INDEX
. . .
Hunk #7 succeeded at 1379 (offset 8 lines).
patching file fs/stack.c
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED – saving rejects to file fs/stack.c.rej
patching file fs/unionfs/Kconfig
. . .
patching file include/linux/namei.h
Hunk #1 FAILED at 84.
1 out of 1 hunk FAILED – saving rejects to file include/linux/namei.h.rej
patching file include/linux/splice.h
. . .
and - of course - compiling fails :

Code: Select all

. . .
  CC      mm/pagewalk.o
fs/namei.c:509:1: error: ‘release_open_intent’ undeclared here (not in a function)
fs/namei.c: In function ‘lookup_one_len_nd’:
fs/namei.c:2135:2: warning: passing argument 3 of ‘__lookup_hash’ makes integer from pointer without a cast [enabled by default]
fs/namei.c:1275:23: note: expected ‘unsigned int’ but argument is of type ‘struct nameidata *’
make[1]: *** [fs/namei.o] Error 1
  CC      fs/fcntl.o
. . .
  CC [M]  fs/unionfs/subr.o
In file included from fs/unionfs/subr.c:19:0:
fs/unionfs/union.h: In function ‘lookup_lck_len’:
fs/unionfs/union.h:579:2: error: implicit declaration of function ‘lookup_one_len_nd’ [-Werror=implicit-function-declaration]
fs/unionfs/union.h:579:4: warning: assignment makes pointer from integer without a cast [enabled by default]
cc1: some warnings being treated as errors
make[2]: *** [fs/unionfs/subr.o] Error 1
  CC [M]  fs/unionfs/dentry.o

I assume, I did something wrong doing a

Code: Select all

git pull
on the sources.

Could you please advise, which version of the kernel sources to use?

Thanks
Richard
-- rikas

yusiang1998
Posts: 22
Joined: Fri Nov 23, 2012 1:10 pm
Location: Singapore, Singapore, Asia, Earth, Solar System, Milky Way, Universe.

Re: Root union filesystem on raspian

Tue Jul 09, 2013 10:41 am

How about using zRam or similar for dumping the R/W partition? I don't need the contents to stay on reboot, but i do need the ability to just pull the plug any second and have the pi reboot into a fresh state.

micerinos
Posts: 74
Joined: Fri Nov 09, 2012 11:15 am
Location: Madrid, Spain

Re: Root union filesystem on raspian

Tue Jul 09, 2013 1:16 pm

Hi,

rikas, I haven't tested the unionfs method in a while, but the sources haven't been updated in a while, so it's likely that it won't work in recent kernels. For this tutorial I used kernel 3.2.27. I strongly recommend you to use aufs for the time being:
http://www.raspberrypi.org/phpBB3/viewt ... p?p=228099

yusiang, eventhough zram may be a good functionallity for some environments, you can right now use tmpfs for the rw partition without zram. Please, have a look at the aufs method.

Cheers

User avatar
rikas
Posts: 17
Joined: Sun Jul 31, 2011 1:28 pm
Location: Austria y Argentina

Re: Root union filesystem on raspian

Tue Jul 09, 2013 7:59 pm

micerinos,

gracias para contestarme - I did not see the mentioned thread ;)

Anyway, I tried this ... and failed again :(

Is there any "idiot-proof" guide to "how to checkout the right version" of this and that ?
I desperately need a kernel with One Wire, i2c and any kind of overlay FS (well unionfs might be a drawback)

Thanks anyway

Saludos
Richard
-- rikas

Return to “Advanced users”