Raspbian Kernel Source

[bertr2d2]

Hi,

I'm using the actual Raspbian 2013-09-25-wheezy-raspbian.zip and tried to get the kernel
source. Does anybody know where I can get it ? apt does provide some:

Code: Select all

linux-source - Linux kernel source (meta-package)
linux-source-2.6 - Linux kernel source (dummy package)
linux-source-3.2 - Linux kernel source for version 3.2 with Debian patches
linux-source-3.6 - Linux kernel source for version 3.6 with Debian patches

but these are not the source of the used kernel:

Code: Select all

pi@raspberrypi ~ % uname -a
Linux raspberrypi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l GNU/Linux

I compiled a module from vanilla 3.6.11 Kernel with 'linux-headers-3.6.11+' and got
a stalled system after installing. I want to use the same source for further testing.

Regards

Gerd

[Oakham]

http://elinux.org/RPi_Kernel_Compilation

[bertr2d2]

http://elinux.org/RPi_Kernel_Compilation
is helpful but doesn't provide any info about getting the exact kernel sources.

It's a little bit tricky - with a litle help from somebody else:

Code: Select all

$  uname -a
  Linux pi3 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l GNU/Linux

  $ zgrep "* firmware as of" /usr/share/doc/raspberrypi-bootloader/changelog.Debian.gz | head -1 |
awk '{ print $5 }'
   d4f5315cfac4e

On the build host fetch the equivalent R-PI firmware.

  $ mkdir -p /embedded/raspbian/build/

  $ cd /embedded/raspbian/build/
  $ git clone https://github.com/raspberrypi/firmware
  $ cd /embedded/raspbian/build/firmware
  $ git checkout d4f5315cfac4e
  $ git reset --hard

Determine the kernel hash for this firmware.

  $ cat extra/git_hash
  1587f775d0a3c437485262ba951afc5e30be69fa

Fetch the kernel source for this hash.

  $ cd /embedded/raspbian/build/
  $ git clone https://github.com/raspberrypi/linux.git
  $ cd /embedded/raspbian/build/linux
  $ git checkout 1587f775d0a3c437485262ba951afc5e30be69fa
  $ git reset --hard

Does anybody know, why the source is not in the Raspbian image ?

Regards

Gerd

[Zeta]

Hello Gerd,

I found a shorter way to get the kernel commit hash without needing to clone locally the firmware git repository, using direct download of a file from github website:

Code: Select all

FIRMWARE_HASH=$(zgrep "* firmware as of" /usr/share/doc/raspberrypi-bootloader/changelog.Debian.gz | head -1 | awk '{ print $5 }')

# get git hash for this kernel
KERNEL_HASH=$(wget https://raw.github.com/raspberrypi/firmware/$FIRMWARE_HASH/extra/git_hash -O -)

# Then replace the git checkout command by this one:
git checkout $KERNEL_HASH

Still a bit complicated, but it works faster.

But this doesn't answer the question why the kernel sources are not in the packages.
I have not yet found where the source packages and recipes are stored for the raspbian build system to take a look. If someone found them I would be interested !

Zeta

[plugwash]

The problem is the kernel you are using is NOT from the raspbian project, it is from the raspberry pi foundation. The raspbian project provides source packages for all their binary packages but unfortunately the raspberry pi foundation does not.

The "firmware" package is built from a "debianised source tree" available from https://github.com/asb/firmware but IIRC when I tried I had to make some hacks before I could actually get a workable "source package" out of the thing. Also despite being a "debianised source tree" it doesn't actually contain source, the binaries are pre-built and the debian package build process just puts them in the right place (much like how packages for closed source software in debian non-free work)
Some of the packages in the foundation repo appear to have been hacked together manually without having a debianised source tree at all.

You can find some stuff at https://github.com/asb?tab=repositories

[Zeta]

Hello plugwash !

As we are talking, I first would like to thank you for your great work on Raspbian !

About the kernel packaging, I made one package this night, and while I didn't yet tested everything with it, it seems to work correctly.

However, it is probably not the "debian way"

Here are the steps I followed, which would probably need to be changed:

• Find the git hash from the current raspbian kernel using the method above
• checkout that kernel from the raspberrypi github
• apply some patches (my original goal was to add CAN bus functionality, but this discussion goes beyond that)
• get the current raspbian .config file and make some adjustments related to the patches above
• package thewhole as a debian package using the command "make-kpkg --append-to-version -can --revision 3.6.11 --initrd --rootcmd fakeroot kernel_image modules_image"

I am quite new to debian packaging (at least the maintener side), so this is only my first steps, and a lot could probably be improved from this.
From what I understood, a proper package should add the debian/ subdir containing all the recipe, documentation, ...
I am not sure if it works the same for kernel package, or if the make-kpkg works differently, or even if it is a good idea to use this command...

Anyway, I will take example on the link you provided for asb's github, but porting that to the raspberrypi's linux repository and not the firmware repository which is already compiled. As I need to compile it with some changes, I can't use the precompiled firmware, so that would be a good test for how to also provide a source package.

As you already knows this well, if you have some other tips or links to improve this process in a good debian way, I can try to do it.
Who knows, if we manage to make a good kernel packaging for a specific need, maybe popcornmix and the raspberrypi foundation could be also interested to use it also for the official kernel they provide ?

Thanks again !
Zeta

[walt09241984]

If you are using a linux system for development aside from you raspberry pi.
To the the kernel source.
github.com/raspberrypi is where kernel source, etc are found.
mkdir ~/src
cd ~/src
git clone -b rpi-3.10.y https://github.com/raspberrypi/linux.git

This will clone 3.10.y branch of the raspberry pi linux kernel, then get the .config of your current kernel. Then compile
Down load the tool chain for raspberry pi.
Apply you patches, etc.

make ARCH=arm CROSS_COMPILE=arm-bcm2708hardfp-linux-gnueabi- oldconfig
Answer only Y to the modules you know otherwise N.
make ARCH=arm CROSS_COMPILE=arm-bcm2708hardfp-linux-gnueabi- -j3
or follow the instructions of rpihub



Walt

[Zeta]

Hello Walt,

Thanks for your help, but the problem I am trying to solve here is not how to compile or cross-compile the kernel, but how to package it correctly as a debian source package, and a debian binary package for the compiled version.

The source package would allow to later compile some modules against the exact kernel source we are running on the pi, otherwise it will fail to load.

The binary package allows to use the package manager to install or update the kernel. So in the case of the custom kernel with CAN bus support, we could distribute updates when the kernel used by the stock raspbian is updated, or when some modules are changed (like when the new mcp2515a driver will be finished).

Zeta

[plugwash]

BTW there is a properly packaged kernel in the raspbian repositories though it sometimes lags a bit behind the foundation's packages.

The package is called linux-image-rpi-rpfv

[walt09241984]

To create a .deb packages

make ARCH=arm CROSS_COMPILE=arm-bcm2708hardfp-linux-gnueabi- deb-pkg

Walt

[Bingo600]

BTW there is a properly packaged kernel in the raspbian repositories though it sometimes lags a bit behind the foundation's packages.

The package is called linux-image-rpi-rpfv

@plugwash

I had to register in order to thank you for providing a full debian (compliant) kernel with headers.
I have spent several days trying to get linux-gpib to work on my RasPI , but every time i loaded the module.
My RasPI quit working , and only a power reset would get it up again.

I have tried countless tricks to get some working headers and module.symvers , but none worked.
The specific "git checkout" used by others have an open ticket that indicates it has been broken since late september.

But then i discovered this posting , and i had linux-gpib up and running in no time
Well i just had to make a small patch .....

I have described the process here.
http://www.eevblog.com/forum/reviews/us ... b-adapter/

Will you please keep on updating this metapackage , this makes compiling modules act like on all my debian/mint workstations.

Once again thanx to the raspbian.org team for picking up where the "Foundation" left off.

/Bingo

[jamesh]

BTW there is a properly packaged kernel in the raspbian repositories though it sometimes lags a bit behind the foundation's packages.

The package is called linux-image-rpi-rpfv

@plugwash

I had to register in order to thank you for providing a full debian (compliant) kernel with headers.
I have spent several days trying to get linux-gpib to work on my RasPI , but every time i loaded the module.
My RasPI quit working , and only a power reset would get it up again.

I have tried countless tricks to get some working headers and module.symvers , but none worked.
The specific "git checkout" used by others have an open ticket that indicates it has been broken since late september.

But then i discovered this posting , and i had linux-gpib up and running in no time
Well i just had to make a small patch .....

I have described the process here.
http://www.eevblog.com/forum/reviews/us ... b-adapter/

Will you please keep on updating this metapackage , this makes compiling modules act like on all my debian/mint workstations.

Once again thanx to the raspbian.org team for picking up where the "Foundation" left off.

/Bingo

Unfortunately, with a small number of volunteers and staff, the Foundation are unable to fix every problem instantly. But hey, thanks for the implied insult. Much appreciated, really makes us want to work harder, since you are all so grateful.

[Bingo600]

Unfortunately, with a small number of volunteers and staff, the Foundation are unable to fix every problem instantly. But hey, thanks for the implied insult. Much appreciated, really makes us want to work harder, since you are all so grateful.

@jamesh
It's not that i don't appreciate what you (the foundation) do/"have done".
But i have seen the kernel-header package being wanted almost since day 1 of the RasPI release.
And there still is no kernel source/header package available , so the "instantly" isn't the situation here.
I have been involved in FOSS , and know the felling when the users "are all so gratefull".
And do understand the sarcasm in your answer, and prob. why it's there.

Basically i wanted to tell plugwash that there are someone out there that uses his kernel metapackage , and that we hope the raspbian.org will keep on maintaining this package.

I'm sorry that you feel insulted , that certainly wasn't my intention. I'd never be personal about a FOSS project.

Maybe you could add the raspbian.org kernel as a selection to NOOBS , witch i think is a neat package.

Happy computing.
/Bingo

[dom]

You've got to understand that no one who works for raspberry pi is a debian packaging expert (and the fact that there are very few people who work for raspberry pi).
I already spend all hours of the day solving issues and with 2 million customers and a handful of people trying to support them it's always going to be an uphill struggle.

When deciding what I should work on, a key point is "who is best placed to solve this?"
For a lot of issues, this involves fixing bugs or adding feature to the firmware, which no one else can do. These issues take precedence.
Then there are issues in the kernel, where the source is available, and technically anyone could fix it, but I know the hardware better than most, so these are quite important.
Then there are arm applications that rely on the GPU, like omxplayer or xbmc. Again, anyone could fix bugs there, but I know the APIs. Attaching a debugger to the GPU can be helpful, so again I'm likely to fix these.

But something like packaging the kernel source is something I know nothing about. I'm not in the top thousand people most suitable to do the job.
It is something that requires no internal knowledge of the Pi.
It is problem that has a perfectly adequate workaround (download the source from github).
It is something that only a handful of people care about.

You see? It's not high priority compared to other issues.

However if you want to help solve this, then we need someone who knows exactly what is needed and who is willing to help.

Find out how kernel source or headers are packaged and provide a script that creates a deb file from the github tree.
Decide which repository is more suitable for the packages. There is the raspbian repo that plugwash maintains, and there is a raspberry pi repo that asb maintains.
If anything necessary is missing from the existing github trees, then let me know.

There is also the question of whether the kernel headers should match the firmware from the latest "apt-get upgrade", or the latest rpi-update (or the "next" rpi-update tree).

[Bingo600]

You've got to understand that no one who works for raspberry pi is a debian packaging expert (and the fact that there are very few people who work for raspberry pi).

@dom
Your answer makes sense.
Especially about doing the things you are best at , and not everyone has access to the Broadcomm NDA docs.

But i thought you (the foundation) had a "university" behind you , and a lot of .deb package knowledge.

That said ....
I had a feeling that one of the reactions here would be ... It's FOSS so please contribute to make it better.
My standard reply was "patches are welcome"

I'd say thanx for the offer but no thanx.
Even though i know how to make .deb packages, i'm not in the top thousand of that task.

So i'll stop whining here
And just use the raspbian.org kernels , they're quite easy to install after having installed the standard raspbian from NOOBS

Btw: The "debian way" of getting the correct headers, has to my best knowledge always been
sudo apt-get install linux-headers-$(uname -r)
Meaning that the headers was always dependant on the kernel vers. you were running when you asked for them.

I installed and activated my raspbian.org kernel just after doing a fresh raspbian (NOOBS) install.

Here are my notes for others

Code: Select all

We get the Raspbian package "metapackage linux kernel" , that has corresponding headers.

1: sudo apt-get update
2: sudo apt-get install linux-image-rpi-rpfv

3: In boot/config.txt append this at end of file , or it will boot the default "Foundation kernel"

************* SNIP ***********************
# Set params for "raspbian debian-style kernel" boot
#kernel=vmlinuz-3.2.0-4-rpi
#initramfs initrd.img-3.2.0-4-rpi followkernel
kernel=vmlinuz-3.6-trunk-rpi
initramfs initrd.img-3.6-trunk-rpi followkernel
************* SNIP ***********************

4: reboot
5: login
6: apt-get install linux-headers-rpi-rpfv

Done Kernel stuff

On my current raspian.org kernel the above "linux-headers-rpi-rpfv" should pull in the linux-headers-3.6-trunk-rpi package.
Based on this output.

Code: Select all

pi@raspberrypi-3 ~ $ uname -r
3.6-trunk-rpi

pi@raspberrypi-3 ~ $ sudo apt-cache search linux-headers-
linux-headers-3.2.0-4-all - All header files for Linux 3.2 (meta-package)
linux-headers-3.2.0-4-all-armhf - All header files for Linux 3.2 (meta-package)
linux-headers-3.2.0-4-common - Common header files for Linux 3.2.0-4
linux-headers-3.2.0-4-rpi - Header files for Linux 3.2.0-4-rpi
linux-headers-3.6-trunk-all - All header files for Linux 3.6 (meta-package)
linux-headers-3.6-trunk-all-armhf - All header files for Linux 3.6 (meta-package)
linux-headers-3.6-trunk-common - Common header files for Linux 3.6-trunk
linux-headers-3.6-trunk-rpi - Header files for Linux 3.6-trunk-rpi
linux-headers-rpi - Header files for Linux rpi configuration (meta-package)
linux-headers-rpi-rpfv - This metapackage will pull in the headers for the raspbian kernel based on

While i was porting versaloon (an arm jtag software) to the STM32VL-Discovery , the project owner always released OpenOCD patches against the current OOCD github tree , but never wrote witch OOCD git-revision the patch was made against.

This usualy meant that the patches failed shortly after his latest patch, as OOCD like the "Foundation kernel" is an evolving target.

So one thing that might help people that compiles modules against the "Foundation" kernel , would be to write what git-revision the current foundation kernel was pulled from.

Thanx to all for your efforts , the RasPI is a neat little board.

/Bingo

[jamesh]

But i thought you (the foundation) had a "university" behind you , and a lot of .deb package knowledge.

Nope.

These are the people I know about...(I will have missed some - sorry)

Dom does a hell of a lot of work, but doesn't work for the Foundation, gsh (Gordon) does work for the Foundation and has fixed the USB and recently the SD card issues (along with a couple of other Foundation employees - these were big jobs and needed serious manpower), asb, rdb and others worked on NOOBS. I work on the camera apps and camera related GPU code (I don't work for the Foundation).

There are certainly others working away, but not very many, and there are occasionally contributors from the community on various items. But not a huge number.

[Loufa]

Hello everyone,

i followed your posts but i have a different question if you can help me with. I’m trying to collect some info from raspberry pi using streamline came with DS-5. I’m currently trying to figure a way to create gator and deamon but i noticed that raspberry pi is not recognized with the adb devices command. also it is not supported by Arm ds-5 but the arm 11 is supported.
MY question is how to use it with streamline how to create the deamon and gator?
i saw Walt post have some details but can someone tell me how to get the .config file and what tool-chain to use?
thanks