using multiarch to cross-build for raspbian

[plugwash]

This topic will show my attempts to setup a multiarch based environment to cross-compile code for raspbian. I hope eventually this will turn into a tutorial but right now it will be more of a set of notes which may or may not actually work.

First we debootstrap a normal wheezy (amd64 in my case) chroot.

Code: Select all

debootstrap wheezy /chroots/wheezy-raspbian-crossbuild/

mount /proc and /dev/pts and chroot into it as normal

Code: Select all

mount -t proc proc /chroots/wheezy-raspbian-crossbuild/proc
mount -t devpts devpts /chroots/wheezy-raspbian-crossbuild/dev/pts
chroot /chroots/wheezy-raspbian-crossbuild

Create policy-rc.d to stop daemons starting and disable installing reccomends.

Code: Select all

echo '#!/bin/sh' > /usr/sbin/policy-rc.d
echo 'echo "************************************" >&2' >> /usr/sbin/policy-rc.d
echo 'echo "All rc.d operations denied by policy" >&2' >> /usr/sbin/policy-rc.d
echo 'echo "************************************" >&2' >> /usr/sbin/policy-rc.d
echo 'exit 101' >> /usr/sbin/policy-rc.d
chmod 755 /usr/sbin/policy-rc.d
echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf

setup sources.list, add armhf to dpkgs architectures, install the raspbian archive keyring and run apt-get update

Code: Select all

echo 'deb [arch=amd64] http://ftp.uk.debian.org/debian wheezy main' > /etc/apt/sources.list
echo 'deb [arch=armhf] http://archive.raspbian.org/raspbian wheezy main' >> /etc/apt/sources.list
echo 'deb-src http://archive.raspbian.org/raspbian wheezy main' >> /etc/apt/sources.list
dpkg --add-architecture armhf
wget http://archive.raspbian.org/raspbian/pool/main/r/raspbian-archive-keyring/raspbian-archive-keyring_20120528.2_all.deb
dpkg -i raspbian-archive-keyring_20120528.2_all.deb
apt-get update

Now install build-essential

Code: Select all

apt-get install build-essential

Next we need to build and install a native gcc-4.7 from the raspbian sources with the multiarch cross-build patches. This is needed to avoid version conflicts and other problems when we come to install raspbian libraries.

Code: Select all

mkdir gccnativebuild
cd gccnativebuild
export DEB_BUILD_OPTIONS=nocheck
apt-get build-dep gcc-4.7
apt-get source gcc-4.7
cd gcc-4.7-4.7.1
for patch in debian/thibgpatches/*.patch; do patch -p1 < $patch ; done
debian/rules control
dpkg-buildpackage -b
cd ..
dpkg -i *4.7.1-8+rpi1_amd64.deb
cd ..

Next we need to build our cross-binutils

Code: Select all

apt-get source binutils
cd binutils-2.22/
export TARGET=$(dpkg-architecture -aarmhf -qDEB_HOST_GNU_TYPE 2>/dev/null)
dpkg-buildpackage -b
cd ..
unset TARGET
dpkg -i binutils-arm-linux-gnueabihf_2.22-6.1_amd64.deb

now install binutils-multiarch

Code: Select all

apt-get install binutils-multiarch

now we need to get an armhf version of linux-libc-dev installed. We can't easily use the raspbian version because of version skew. Fortunately linux-libc-dev only contains headers, so it's safe to use the debian armhf version of it.

Code: Select all

wget http://ftp.uk.debian.org/debian/pool/main/l/linux/linux-libc-dev_3.2.23-1_armhf.deb
dpkg -i linux-libc-dev_3.2.23-1_armhf.deb

Now we can install the raspbian libc6-dev, this is needed before we can build the cross-compilers

Code: Select all

apt-get install libc6-dev:armhf libstdc++6-4.7-dev:armhf

now for gcc itself, for this we need to do some patching

Code: Select all

mkdir gcccrossbuild
cd gcccrossbuild
apt-get source gcc-4.7
cd gcc-4.7-4.7.1
for patch in debian/thibgpatches/*.patch; do patch -p1 < $patch ; done
export DEB_CROSS_NO_BIARCH=yes
echo armhf > debian/arch
debian/rules control
dpkg-buildpackage -b
unset DEB_CROSS_NO_BIARCH
cd ..
dpkg -i *4.7.1-8+rpi1_*.deb
cd ..

Now we need to install pkg-config and create some symlinks

Code: Select all

cd /usr/bin
ln -s arm-linux-gnueabihf-gcc-4.7 arm-linux-gnueabihf-gcc
ln -s arm-linux-gnueabihf-g++-4.7 arm-linux-gnueabihf-g++
ln -s ../share/pkg-config-crosswrapper arm-linux-gnueabihf-pkg-config
cd /

We need to hack some things in the dpkg database to avoid dependency issues. Open /var/lib/dpkg/status in your favourite text editor (if it's not installed install it), find the block for pkg-config (that is starts with the line Package: pkg-config) and add "Multi-Arch: foreign" to it's block and add a plus to the end of it's version number. Do the same for python. Now find the block for libffi5 and change it's version from 3.0.10-3 to 3.0.10-3+b3. Finally find the block for gcc-4.7-arm-linux-gnueabihf and add gcc-4.7:armhf to the provides line.

Now lets try cross-building something, i'm going to use pidgin as an example of a moderately tricky package.

Code: Select all

mkdir pidgin
cd pidgin
apt-get build-dep -aarmhf pidgin
[code]

This fails with an error about tcl-dev. It turns out this is arch all which confuses the multiarch support in apt-get build-dep. Looking further it turns out to be a dependency package depending on tcl-8.5-dev which we want the raspbian version of. Unfortunately this currently requires some hackery to make dpkg/apt accept it. tk turns out to have the same issue.

First we install tcl-dev and tk-dev
[code]
apt-get install tcl-dev tk-dev

At this point we will have the amd64 tcl8.5-dev, tcl8.5, tk8.5 and tk8.5-dev installed which is not what we want. To fix this open /var/lib/dpkg/status in your favourite text editor, find the entries for tcl-dev, tcl, tk8.5 an tk8.5-dev and change their architecture from "all" to "armhf"

We can now replace the amd64 tcl8.5 and tcl8.5-dev packages with the raspbian ones.

Code: Select all

aptitude install tcl8.5:armhf tcl8.5-dev:armhf tk8.5:armhf tk8.5-dev:armhf tcl8.5:amd64- tcl8.5-dev:amd64- tk8.5:amd64- tk8.5-dev:amd64-

It will want to remove the amd64 versions of libexpat1-dev, libfontconfig1-dev, libfreetype6-dev and libxft-dev, this is fine.

Unfortunately it's still not possible to install the pidgin build-deps, i'm still looking into this.

[plugwash]

Just to let you guys know the current status i've succcessfully built hello world in c in my multiarch crossbuild environment and run the resulting binary successfully on a Pi.

I'm currently rebuilding the native gcc again with the multiarch patches (originally I built it without them) as that seems nessacery to get a multiarch libstdc++6-4.7-dev for amd64 which we need so we can get a libstdc++6-4.7-dev for raspbian installed.

[twolife]

hi !
is there any progress with the "final" rebuild of the native gcc ?

[plugwash]

Yes, rebuilding the native gcc from the raspbian sources with the multiarch cross patches applied worked to allow the libstdc++6-dev packages to be parellell installed.

I haven't actually managed to crossbuild any raspbian packages yet though. I should probablly have chosen something simpler than pidgin to start with. I think in general multiarch cross-building is still a pretty fragile buisness and I intend to bring this up on the debian side.

I think I will setup a dedicated repository for this as i've come accross a few different packages requiring various hacks.

[simplesi]

One day I hope to understand >10% of the words used

Simon

[plugwash]

Still working on this.

[twolife]

I have successfully rebuild gcc (3 times) :
- "native" amd64
- "native" armhf
- amd64 -> arm-linux-gnueabihf

helloworld in c & c++ builds fine & run fine on the pi
but...

so far it as been a complete faillure on the debian side
- pkg-config not multiarch aware
+ lack of "Multi-Arch: foreign" in control file
+ typo in pkg-config-crosswrapper ( /usr/${triplet}/lib/ instead of '/usr/lib/${triplet}/' )
- too much libs not multiarch aware
- all the binNMU buisness is a complete anti-multiarch mess

i will stop this experiment here, i think it's too much work to get this in good shape for wheezy

[plugwash]

I have successfully rebuild gcc (3 times) :

Only 3?

I must be on about the tenth build of gcc since starting this multiarch cross stuff.

[lueschem]

Some years later I tried a similar approach (now based on the jessie release). Here are my results:
- kernel compilation works
- recompilation of low level libraries such as openssl works
- compilation of executables with a manageable number of dependencies works
- I gave up on rebuilding pidgin (too much work due to mismatching binNMU versions)

Here are the details including an automated setup of the toolchain within an lxc/lxd container:
http://www.get-edi.io/Cross-Compiling-for-Raspbian/